CN111382696A - Method and apparatus for detecting boundary points of object - Google Patents
Method and apparatus for detecting boundary points of object Download PDFInfo
- Publication number
- CN111382696A CN111382696A CN202010153383.1A CN202010153383A CN111382696A CN 111382696 A CN111382696 A CN 111382696A CN 202010153383 A CN202010153383 A CN 202010153383A CN 111382696 A CN111382696 A CN 111382696A
- Authority
- CN
- China
- Prior art keywords
- target
- feature
- boundary
- panoramic image
- radial scanning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 56
- 238000001514 detection method Methods 0.000 claims abstract description 43
- 238000013528 artificial neural network Methods 0.000 claims abstract description 39
- 238000010586 diagram Methods 0.000 claims description 32
- 238000005070 sampling Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 9
- 230000008569 process Effects 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 4
- 238000012706 support-vector machine Methods 0.000 description 4
- 238000011176 pooling Methods 0.000 description 3
- 238000004140 cleaning Methods 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 210000004556 brain Anatomy 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/04—Context-preserving transformations, e.g. by using an importance map
- G06T3/047—Fisheye or wide-angle transformations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the disclosure discloses a method and a device for detecting boundary points of a target, and relates to the technical field of automatic driving, in particular to the technical field of autonomous parking. The method comprises the following steps: acquiring a panoramic image to be processed; inputting the panoramic image to be processed into a target detection neural network to obtain a feature map which is output by the target detection neural network and contains the feature points of the target; and performing radial scanning from the center point to the edge of the feature map, and determining the feature point with the maximum feature value in each radial scanning as the feature point of the boundary of the target. The method can effectively reduce the operation time of the feature points of the boundary of the detection target, accelerate the detection speed, reduce the requirement on the computing capacity of hardware and reduce the cost of the hardware.
Description
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to the field of image recognition technologies, and in particular, to a method and an apparatus for detecting boundary points of a target.
Background
In current intelligent automobile scene, in order to can detect the vehicle region of can traveling, installed four ways fisheye camera additional on the car, respectively one to the front fisheye camera, each side direction fisheye camera about, and to the fisheye camera of back.
In the process of vehicle advancing, the four paths of fisheye cameras transmit the obtained video data to a Field Programmable Gate Array (FPGA), then in the FPGA, four models (respectively corresponding to the four paths of cameras) trained offline are adopted to respectively infer the image data of the four paths of cameras, and finally four results are returned to downstream for decision making.
Disclosure of Invention
The embodiment of the disclosure provides a method and a device for detecting boundary points of a target.
In a first aspect, an embodiment of the present disclosure provides a method for detecting a boundary point of an object, including: acquiring a panoramic image to be processed; inputting the panoramic image to be processed into a target detection neural network to obtain a feature map which is output by the target detection neural network and contains the feature points of the target; and performing radial scanning from the center point to the edge of the feature map, and determining the feature point with the maximum feature value in each radial scanning as the feature point of the boundary of the target.
In some embodiments, acquiring the panoramic image to be processed comprises: acquiring a plurality of fisheye images collected at the same time; splicing a plurality of fisheye images collected at the same time to obtain a spliced panoramic image; and downsampling the spliced panoramic image to obtain the panoramic image to be processed.
In some embodiments, the method further comprises: and performing up-sampling on the characteristic points of the boundary of the target to obtain the boundary of the target.
In some embodiments, performing radial scanning from the center point to the edge of the feature map, and determining the feature point with the largest feature value at each radial scanning as the feature point of the boundary of the target includes: dividing the characteristic diagram at equal angles along a circular direction around the central point of the characteristic diagram to obtain a plurality of areas; and simultaneously, carrying out radial scanning on each area, and determining the characteristic point with the maximum characteristic value in each radial scanning as the characteristic point of the boundary of the target.
In some embodiments, the target comprises a travelable region.
In a second aspect, an embodiment of the present disclosure provides an apparatus for detecting boundary points of an object, including: an image acquisition unit configured to acquire a panoramic image to be processed; the characteristic diagram output unit is configured to input the panoramic image to be processed into a target detection neural network to obtain a characteristic diagram which is output by the target detection neural network and contains characteristic points of a target; and the radial scanning unit is configured to perform radial scanning from the center point to the edge of the feature map, and determines the feature point with the maximum feature value in each radial scanning as the feature point of the boundary of the target.
In some embodiments, the image acquisition unit comprises: an image acquisition subunit configured to acquire a plurality of fisheye images acquired at the same time; the image splicing subunit is configured to splice a plurality of fisheye images collected at the same time to obtain a spliced panoramic image; and the downsampling subunit is configured to downsample the spliced panoramic image to obtain the panoramic image to be processed.
In some embodiments, the apparatus further comprises: and the up-sampling unit is configured to up-sample the characteristic points of the boundary of the target to obtain the boundary of the target.
In some embodiments, the radial scanning unit comprises: the region dividing subunit is configured to divide the characteristic diagram at equal angles along a circular direction around the central point of the characteristic diagram to obtain a plurality of regions; and the area scanning subunit is configured to perform radial scanning on the areas simultaneously, and determine the characteristic point with the maximum characteristic value in each radial scanning as the characteristic point of the boundary of the target.
In some embodiments, the targets in the feature map output unit and the radial scanning unit include travelable regions.
In a third aspect, an embodiment of the present disclosure provides an electronic device/terminal/server, including: one or more processors; storage means for storing one or more programs; when executed by one or more processors, cause the one or more processors to implement a method for detecting boundary points of an object as described in any above.
In a fourth aspect, the embodiments of the present disclosure provide a computer-readable medium, on which a computer program is stored, which when executed by a processor, implements the method for detecting boundary points of an object as any one of the above.
The method and the device for detecting the boundary point of the target provided by the embodiment of the disclosure firstly acquire a panoramic image to be processed; then, inputting the panoramic image to be processed into a target detection neural network to obtain a feature map which is output by the target detection neural network and contains the feature points of the target; and then, radial scanning is carried out from the center point to the edge of the feature map, and the feature point with the maximum feature value in each radial scanning is determined as the feature point of the boundary of the target. In the process, an existing target detection neural network can be adopted to extract a feature map containing the feature points of the target from the panoramic image to be processed so as to improve the efficiency of generating the feature map containing the feature points of the target, and then for the feature map containing the feature points of the target, as the rows or the columns of the feature map comprise more than one boundary points of the target, the boundary points of the target cannot be accurately identified in the scanning process of the rows or the columns, the radial scanning can be carried out from the center point to the edge of the feature map, and the feature point with the maximum feature value in each radial scanning process is determined as the feature point of the boundary of the target, so that the operation time of detecting the feature points of the boundary of the target is effectively reduced, the detection speed is increased, the requirement on the computing power of hardware is reduced, and the cost of the hardware is reduced.
Drawings
Other features, objects, and advantages of the disclosure will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, with reference to the accompanying drawings in which:
FIG. 1 is an exemplary system architecture diagram in which the present disclosure may be applied;
FIG. 2 is a schematic flow chart diagram illustrating one embodiment of a method for detecting boundary points of an object, in accordance with an embodiment of the present disclosure;
FIG. 3 is an exemplary application scenario of a method for detecting boundary points of an object according to an embodiment of the present disclosure;
FIG. 4 is a schematic flow chart diagram illustrating yet another embodiment of a method for detecting boundary points of an object in accordance with an embodiment of the present disclosure;
FIG. 5 is an exemplary block diagram of one embodiment of an apparatus for detecting boundary points of a target of the present disclosure;
FIG. 6 is a schematic block diagram of a computer system suitable for use with a server embodying embodiments of the present disclosure.
Detailed Description
The present disclosure is described in further detail below with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that, in the present disclosure, the embodiments and features of the embodiments may be combined with each other without conflict. The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
Fig. 1 illustrates an exemplary system architecture 100 to which embodiments of the method for detecting boundary points of an object or the apparatus for detecting boundary points of an object of the present disclosure may be applied.
As shown in fig. 1, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The user may use the terminal devices 101, 102, 103 to interact with the server 105 via the network 104 to receive or send messages or the like. Various applications such as an image processing application, a travel reminder application, a travel control application, and the like may be installed on the terminal apparatuses 101, 102, 103.
The terminal apparatuses 101, 102, and 103 may be hardware or software. When the terminal devices 101, 102, 103 are hardware, they may be various electronic devices that support browser applications, including but not limited to automobile brains, smart terminals, tablets, laptop portable computers, desktop computers, and the like. When the terminal apparatuses 101, 102, 103 are software, they can be installed in the electronic apparatuses listed above. It may be implemented, for example, as multiple software or software modules to provide distributed services, or as a single software or software module. And is not particularly limited herein.
The server 105 may be a server providing various services, such as a background server providing support for the terminal devices 101, 102, 103. The background server can analyze and process the received data such as the request and feed back the processing result to the terminal equipment.
The server may be hardware or software. When the server is hardware, it may be implemented as a distributed server cluster formed by multiple servers, or may be implemented as a single server. When the server is software, it may be implemented as multiple pieces of software or software modules, for example, to provide distributed services, or as a single piece of software or software module. And is not particularly limited herein.
In practice, the method for detecting the boundary point of the target provided by the embodiment of the present disclosure may be performed by the terminal device 101, 102, 103 and/or the server 105, 106, and the apparatus for detecting the boundary point of the target may also be disposed in the terminal device 101, 102, 103 and/or the server 105, 106.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
With continued reference to fig. 2, fig. 2 illustrates a flow 200 of one embodiment of a method for detecting boundary points of a target according to the present disclosure. The method for detecting the boundary point of the target comprises the following steps:
In the present embodiment, an execution subject (for example, a terminal or a server shown in fig. 1) of the method for detecting the boundary point of the target may acquire the panoramic image to be processed from a local or remote album, a database, or via a local or remote image processing service.
The panoramic image to be processed can adopt a plurality of fisheye images collected at the same time to be spliced, and the spliced panoramic image is used as the panoramic image to be processed; after the spliced panoramic image is obtained by adopting a plurality of fisheye images collected at the same time, the spliced panoramic image is further processed (for example, data cleaning, data enhancement and the like) to obtain a panoramic image to be processed.
In some optional implementations of the present embodiment, acquiring the panoramic image to be processed may include: acquiring a plurality of fisheye images collected at the same time; splicing a plurality of fisheye images collected at the same time to obtain a spliced panoramic image; and downsampling the spliced panoramic image to obtain the panoramic image to be processed.
In the implementation mode, after the spliced panoramic images are obtained based on the plurality of fisheye images collected at the same time, the to-be-processed panoramic images are obtained by down-sampling the spliced panoramic images, and the data volume required to be processed when the follow-up to-be-processed panoramic images are subjected to data processing can be reduced.
The down-sampling means that for an image I with the size of M × N, the down-sampling is performed by s (s >1) times, namely, a resolution image with the size of (M/s) × (N/s) is obtained, wherein s is the common divisor of M and N, if the panoramic image subjected to the down-sampling is an image in a matrix form, the image in the window of an original image s × s is changed into one pixel, and the value of the pixel is the average value of all pixels in the window.
In this embodiment, the target detection neural network is a trained machine learning network that takes the panoramic image to be processed as an input and takes the feature map containing the feature points of the target as an output, and may be a neural network used in the prior art or a technology developed in the future to detect and mark the feature points of the target in the feature map, which is not limited in this application.
For example, a target detection neural network may be implemented based on R-CNN, a spatial pyramid pooling network SPP, FAST-RCNN, FASTER-RCNN, and the like; on occasions where high real-time performance is needed, the target detection neural network can be realized based on a target detection algorithm YOLO series, YOLO + SSD, a Support Vector Machine (SVM), a deep convolution neural network deep Lab series and the like.
The target detection neural network can be obtained by the following method: and taking the panoramic image sample as input, taking a feature map containing the feature points of the target corresponding to the panoramic image sample as expected output, and training an initial network of the target detection neural network to obtain the target detection neural network.
The target included in the feature map is a target surrounding the central point of the panorama, which is identified from the panorama according to the needs of the application scene. E.g., a particular entity in the panoramic view. In one specific example, the targets included in the feature map may be: in the case of autonomous driving, a drivable region is identified from the panorama to be processed.
The feature map including the feature points of the target output by the target detection neural network outputs the maximum response value at the boundary points of the target, that is, the boundary points of the target have the maximum feature values.
And step 203, performing radial scanning from the center point to the edge of the feature map, and determining the feature point with the maximum feature value in each radial scanning as the feature point of the boundary of the target.
In this embodiment, for the feature map including the feature points of the target obtained in step 202, the execution subject performs radial scanning in the edge direction of the feature map starting from the center point of the feature map. Wherein, the radial scanning starts from the central point of the characteristic diagram, scans to the edge of the characteristic diagram along the radial direction, returns to the central point of the characteristic diagram again, generates another radial line adjacent to the previous radial line, and scans the newly generated radial line to the edge of the characteristic diagram along the radial direction, and returns to the central point of the characteristic diagram again, so that one radial line appears after the previous radial line as if each radial line rotates around the central point of the characteristic diagram.
When the execution body performs radial scanning, the feature point with the largest feature value at each radial scanning may be determined, and the feature point with the largest feature value may be determined as the feature point of the boundary of the target.
In some optional implementations of this embodiment, the method may further include: and performing up-sampling on the characteristic points of the boundary of the target to obtain the boundary of the target.
In this implementation, the stitched panoramic image is downsampled to obtain the panoramic image to be processed, and after the feature map including the feature points of the target boundary is obtained, the feature points of the target boundary are upsampled to obtain the restored target boundary. The upsampling may be an upsampling method in the prior art or a technology developed in the future, such as bilinear interpolation, transposed convolution, upsampling (upsampling), and pooling (upsampling), and thus, will not be described herein again.
In the implementation mode, the definition of the obtained boundary of the target can be improved by up-sampling the characteristic points of the boundary of the target.
The method for detecting boundary points of an object according to the above-mentioned embodiments of the present disclosure may employ an existing object detection neural network to extract a feature map including feature points of the object from a panoramic image to be processed, to improve the efficiency of generating a feature map containing feature points of the object, and then for the feature map containing feature points of the object, because the rows or columns of the feature map comprise more than one boundary point of the target, the boundary points of the target cannot be accurately identified when the rows or columns are scanned, therefore, radial scanning can be carried out from the center point to the edge of the feature map, and the feature point with the maximum feature value in each radial scanning is determined as the feature point of the boundary of the target, therefore, the operation time of the feature points of the boundary of the detection target is effectively reduced, the detection speed is accelerated, the requirement on the computing capacity of hardware is reduced, and the cost of the hardware is reduced.
An exemplary application scenario of the method for detecting boundary points of an object of the present disclosure is described below in conjunction with fig. 3.
As shown in fig. 3, fig. 3 illustrates one exemplary application scenario of the method for detecting boundary points of an object according to the present disclosure.
As shown in fig. 3, a method 300 for detecting boundary points of an object operates in an electronic device 310, and may include:
firstly, acquiring a panoramic image 301 to be processed;
secondly, inputting the panoramic image 301 to be processed into a target detection neural network 302 to obtain a feature map 303 which is output by the target detection neural network and contains the feature points of the target;
finally, radial scanning is performed from the center point 304 of the feature map to the edge 305 of the feature map, and the feature point 306 having the largest feature value at each radial scanning is determined as the feature point 307 of the boundary of the target.
It should be understood that the application scenario of the method for detecting boundary points of an object illustrated in fig. 3 is only an exemplary description of the method for detecting boundary points of an object, and does not represent a limitation of the method. For example, the steps shown in fig. 3 above may be implemented in further detail. Other steps for generating an image may be further added to the above-described fig. 3.
With further reference to fig. 4, fig. 4 shows a schematic flow chart of yet another embodiment of a method for detecting boundary points of an object according to the present disclosure.
As shown in fig. 4, the method 400 for detecting boundary points of an object of the present embodiment may include the following steps:
In this embodiment, an execution subject (for example, a terminal or a server shown in fig. 1) of the method for detecting the boundary point of the target may acquire the panoramic image to be processed from a local or remote album, a database, or via a local or remote image processing service.
The panoramic image to be processed can adopt a plurality of fisheye images collected at the same time to be spliced, and the spliced panoramic image is used as the panoramic image to be processed; after the spliced panoramic image is obtained by adopting a plurality of fisheye images collected at the same time, the spliced panoramic image is further processed (for example, data cleaning, data enhancement and the like) to obtain a panoramic image to be processed.
And 402, inputting the panoramic image to be processed into a target detection neural network to obtain a feature map which is output by the target detection neural network and contains the feature points of the target.
In this embodiment, the target detection neural network is a trained machine learning network that takes the panoramic image to be processed as an input and takes the feature map containing the feature points of the target as an output, and may be a neural network used in the prior art or a technology developed in the future to detect and mark the feature points of the target in the feature map, which is not limited in this application.
For example, a target detection neural network may be implemented based on R-CNN, a spatial pyramid pooling network SPP, FAST-RCNN, FASTER-RCNN, and the like; on occasions where high real-time performance is needed, the target detection neural network can be realized based on a target detection algorithm YOLO series, YOLO + SSD, a Support Vector Machine (SVM), a deep convolution neural network deep Lab series and the like.
The feature map including the feature points of the target output by the target detection neural network outputs the maximum response value at the boundary points of the target, that is, the boundary points of the target have the maximum feature values.
Those skilled in the art will appreciate that steps 401 to 402 described above correspond to steps 201 to 202 in the embodiment shown in fig. 2. Thus, the operations and features described above for step 201 to step 202 in the embodiment shown in fig. 2 are also applicable to step 401 to step 402, and are not described again here.
And 403, dividing the feature map at equal angles along a circular direction around the center point of the feature map to obtain a plurality of regions.
In this embodiment, the execution body may divide 360 degrees into a plurality of angles in a circumferential direction around a center point of the feature map, thereby dividing the angles such as the feature map into a plurality of regions.
And step 404, simultaneously performing radial scanning on each area, and determining the characteristic point with the maximum characteristic value in each radial scanning as the characteristic point of the boundary of the target.
In this embodiment, the execution subject performs radial scanning in the direction of the edge of the feature map from the center of the feature map for each of the plurality of divided regions. Wherein, the radial scanning starts from the central point of the characteristic diagram, scans to the edge of the characteristic diagram along the radial direction, returns to the central point of the characteristic diagram again, generates another radial line adjacent to the previous radial line, and scans the newly generated radial line to the edge of the characteristic diagram along the radial direction, and returns to the central point of the characteristic diagram again, so that one radial line appears after the previous radial line as if each radial line rotates around the central point of the characteristic diagram.
When the execution body performs radial scanning, the feature point with the largest feature value at each radial scanning may be determined, and the feature point with the largest feature value may be determined as the feature point of the boundary of the target.
Compared with the embodiment shown in fig. 2, the method for detecting the boundary point of the target according to the above embodiment of the present disclosure may divide the feature map at equal angles along the circular direction around the central point of the feature map to obtain a plurality of regions, perform radial scanning on each of the plurality of regions from the central point to the edge of the feature map, and determine the feature point with the largest feature value in each radial scanning as the feature point of the boundary of the target, thereby further effectively reducing the operation time for detecting the feature point of the boundary of the target, increasing the detection speed, reducing the requirement on the computing capability of hardware, and reducing the cost of hardware.
As an implementation of the methods shown in the above figures, the embodiment of the present disclosure provides an embodiment of an apparatus for detecting a boundary point of an object, where the embodiment of the apparatus corresponds to the method embodiments shown in fig. 2 to fig. 4, and the apparatus may be specifically applied to the terminal or the server shown in fig. 1.
As shown in fig. 5, the apparatus 500 for detecting a boundary point of an object of the present embodiment may include: an image acquisition unit 501 configured to acquire a panoramic image to be processed; a feature map output unit 502 configured to input the panoramic image to be processed into a target detection neural network, and obtain a feature map including feature points of a target output by the target detection neural network; the radial scanning unit 503 is configured to perform radial scanning from the center point to the edge of the feature map, and determines a feature point having the largest feature value at each radial scanning as a feature point of the boundary of the target.
In some optional implementations of the present embodiment, the image acquisition unit 501 includes (not shown in the figure): an image acquisition subunit configured to acquire a plurality of fisheye images acquired at the same time; the image splicing subunit is configured to splice a plurality of fisheye images collected at the same time to obtain a spliced panoramic image; and the downsampling subunit is configured to downsample the spliced panoramic image to obtain the panoramic image to be processed.
In some optional implementations of this embodiment, the apparatus further comprises: the upsampling unit 504 is configured to upsample the feature points of the boundary of the target, so as to obtain the boundary of the target.
In some optional implementations of the present embodiment, the radial scanning unit 503 includes (not shown in the figure): the region dividing subunit is configured to divide the characteristic diagram at equal angles along a circular direction around the central point of the characteristic diagram to obtain a plurality of regions; and the area scanning subunit is configured to perform radial scanning on the areas simultaneously, and determine the characteristic point with the maximum characteristic value in each radial scanning as the characteristic point of the boundary of the target.
In some alternative implementations of the present embodiment, the targets in the feature map output unit 502 and the radial scanning unit 503 include travelable regions.
It should be understood that the various elements recited in the apparatus 500 correspond to the various steps recited in the method described with reference to fig. 2-4. Thus, the operations and features described above for the method are equally applicable to the apparatus 500 and the various units included therein and will not be described again here.
Referring now to fig. 6, a schematic diagram of an electronic device (e.g., the server or terminal device of fig. 1) 600 suitable for use in implementing embodiments of the present disclosure is shown. Terminal devices in embodiments of the present disclosure may include, but are not limited to, devices such as notebook computers, desktop computers, and the like. The terminal device/server shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 6, electronic device 600 may include a processing means (e.g., central processing unit, graphics processor, etc.) 601 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage means 608 into a Random Access Memory (RAM) 603. In the RAM603, various programs and data necessary for the operation of the electronic apparatus 600 are also stored. The processing device 601, the ROM 602, and the RAM603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
Generally, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 607 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 608 including, for example, tape, hard disk, etc.; and a communication device 609. The communication means 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. While fig. 6 illustrates an electronic device 600 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided. Each block shown in fig. 6 may represent one device or may represent multiple devices as desired.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication means 609, or may be installed from the storage means 608, or may be installed from the ROM 602. The computer program, when executed by the processing device 601, performs the above-described functions defined in the methods of embodiments of the present disclosure. It should be noted that the computer readable medium described in the embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In embodiments of the disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In embodiments of the present disclosure, however, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: acquiring a panoramic image to be processed; inputting the panoramic image to be processed into a target detection neural network to obtain a feature map which is output by the target detection neural network and contains the feature points of the target; and performing radial scanning from the center point to the edge of the feature map, and determining the feature point with the maximum feature value in each radial scanning as the feature point of the boundary of the target.
Computer program code for carrying out operations for embodiments of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present disclosure may be implemented by software or hardware. The described units may also be provided in a processor, and may be described as: a processor includes an image acquisition unit, a feature map output unit, and a radial scanning unit. Here, the names of these units do not constitute a limitation on the unit itself in some cases, and for example, the image acquisition unit may also be described as a "unit that acquires a panoramic image to be processed".
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention in the present disclosure is not limited to the specific combination of the above-mentioned features, but also encompasses other embodiments in which any combination of the above-mentioned features or their equivalents is possible without departing from the inventive concept as defined above. For example, the above features and (but not limited to) the features disclosed in this disclosure having similar functions are replaced with each other to form the technical solution.
Claims (12)
1. A method for detecting boundary points of an object, comprising:
acquiring a panoramic image to be processed;
inputting the panoramic image to be processed into a target detection neural network to obtain a feature map which is output by the target detection neural network and contains the feature points of the target;
and performing radial scanning from the center point to the edge of the feature map, and determining the feature point with the maximum feature value in each radial scanning as the feature point of the boundary of the target.
2. The method of claim 1, wherein the obtaining the panoramic image to be processed comprises:
acquiring a plurality of fisheye images collected at the same time;
splicing the plurality of fisheye images collected at the same time to obtain a spliced panoramic image;
and downsampling the spliced panoramic image to obtain the panoramic image to be processed.
3. The method of claim 2, wherein the method further comprises:
and up-sampling the characteristic points of the boundary of the target to obtain the boundary of the target.
4. The method according to claim 1, wherein the radial scanning is performed from the center point to the edge of the feature map, and the determining the feature point with the largest feature value at each radial scanning as the feature point of the boundary of the target comprises:
carrying out equal-angle division on the feature map along a ring direction around the center point of the feature map to obtain a plurality of regions;
and simultaneously, carrying out radial scanning on each area, and determining the characteristic point with the maximum characteristic value in each radial scanning as the characteristic point of the boundary of the target.
5. The method of any of claims 1-4, wherein the target comprises a travelable region.
6. An apparatus for detecting boundary points of an object, comprising:
an image acquisition unit configured to acquire a panoramic image to be processed;
the characteristic diagram output unit is configured to input the panoramic image to be processed into a target detection neural network to obtain a characteristic diagram which is output by the target detection neural network and contains characteristic points of a target;
and the radial scanning unit is configured to perform radial scanning from the center point to the edge of the feature map, and determine the feature point with the maximum feature value in each radial scanning as the feature point of the boundary of the target.
7. The apparatus of claim 6, wherein the image acquisition unit comprises:
an image acquisition subunit configured to acquire a plurality of fisheye images acquired at the same time;
the image splicing subunit is configured to splice the plurality of fisheye images acquired at the same time to obtain a spliced panoramic image;
and the downsampling subunit is configured to downsample the spliced panoramic image to obtain the panoramic image to be processed.
8. The apparatus of claim 7, wherein the apparatus further comprises:
and the up-sampling unit is configured to up-sample the characteristic points of the boundary of the target to obtain the boundary of the target.
9. The apparatus of claim 6, wherein the radial scanning unit comprises:
the region dividing subunit is configured to divide the feature map at equal angles along a circular direction around the center point of the feature map to obtain a plurality of regions;
and the area scanning subunit is configured to perform radial scanning on the areas simultaneously, and determine the characteristic point with the maximum characteristic value in each radial scanning as the characteristic point of the boundary of the target.
10. The apparatus according to any one of claims 5 to 9, wherein the targets in the feature map output unit and the radial scanning unit comprise travelable regions.
11. An electronic device/terminal/server comprising:
one or more processors;
storage means for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-5.
12. A computer-readable medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010153383.1A CN111382696A (en) | 2020-03-06 | 2020-03-06 | Method and apparatus for detecting boundary points of object |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010153383.1A CN111382696A (en) | 2020-03-06 | 2020-03-06 | Method and apparatus for detecting boundary points of object |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111382696A true CN111382696A (en) | 2020-07-07 |
Family
ID=71221491
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010153383.1A Pending CN111382696A (en) | 2020-03-06 | 2020-03-06 | Method and apparatus for detecting boundary points of object |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111382696A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114092559A (en) * | 2021-11-30 | 2022-02-25 | 中德（珠海）人工智能研究院有限公司 | Training method and device for panoramic image feature point descriptor generation network |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090148037A1 (en) * | 2007-12-05 | 2009-06-11 | Topcon Corporation | Color-coded target, color code extracting device, and three-dimensional measuring system |
| CN104318238A (en) * | 2014-11-10 | 2015-01-28 | 广州御银科技股份有限公司 | Method for extracting crown word numbers from scanned banknote images in banknote detection module |
| CN104463851A (en) * | 2014-11-19 | 2015-03-25 | 哈尔滨工业大学深圳研究生院 | Automatic shoe sole edge line tracking method based on robot |
| CN105190693A (en) * | 2013-03-01 | 2015-12-23 | 波士顿科学国际有限公司 | Systems and methods for lumen border detection in intravascular ultrasound sequences |
| CN106296571A (en) * | 2016-07-29 | 2017-01-04 | 厦门美图之家科技有限公司 | A kind of based on face grid reduce wing of nose method, device and calculating equipment |
| CN107507208A (en) * | 2017-07-12 | 2017-12-22 | 天津大学 | A kind of characteristics of image point extracting method based on Curvature Estimation on profile |
| JP2019029002A (en) * | 2017-07-28 | 2019-02-21 | 株式会社リコー | Re-positioning method to be used in panoramic images, device thereto, and electronic device |
| CN109866684A (en) * | 2019-03-15 | 2019-06-11 | 江西江铃集团新能源汽车有限公司 | Lane departure warning method, system, readable storage medium storing program for executing and computer equipment |
| CN109934110A (en) * | 2019-02-02 | 2019-06-25 | 广州中科云图智能科技有限公司 | A kind of river squatter building house recognition methods nearby |
-
2020
- 2020-03-06 CN CN202010153383.1A patent/CN111382696A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090148037A1 (en) * | 2007-12-05 | 2009-06-11 | Topcon Corporation | Color-coded target, color code extracting device, and three-dimensional measuring system |
| CN105190693A (en) * | 2013-03-01 | 2015-12-23 | 波士顿科学国际有限公司 | Systems and methods for lumen border detection in intravascular ultrasound sequences |
| CN104318238A (en) * | 2014-11-10 | 2015-01-28 | 广州御银科技股份有限公司 | Method for extracting crown word numbers from scanned banknote images in banknote detection module |
| CN104463851A (en) * | 2014-11-19 | 2015-03-25 | 哈尔滨工业大学深圳研究生院 | Automatic shoe sole edge line tracking method based on robot |
| CN106296571A (en) * | 2016-07-29 | 2017-01-04 | 厦门美图之家科技有限公司 | A kind of based on face grid reduce wing of nose method, device and calculating equipment |
| CN107507208A (en) * | 2017-07-12 | 2017-12-22 | 天津大学 | A kind of characteristics of image point extracting method based on Curvature Estimation on profile |
| JP2019029002A (en) * | 2017-07-28 | 2019-02-21 | 株式会社リコー | Re-positioning method to be used in panoramic images, device thereto, and electronic device |
| CN109934110A (en) * | 2019-02-02 | 2019-06-25 | 广州中科云图智能科技有限公司 | A kind of river squatter building house recognition methods nearby |
| CN109866684A (en) * | 2019-03-15 | 2019-06-11 | 江西江铃集团新能源汽车有限公司 | Lane departure warning method, system, readable storage medium storing program for executing and computer equipment |
Non-Patent Citations (4)
| Title |
|---|
| JOSEPH REDMON 等: "YOLOv3: An Incremental Improvement", pages 1 - 6 * |
| 周显恩 等: "基于机器视觉的瓶口缺陷检测方法研究", vol. 30, no. 30, pages 702 - 713 * |
| 梁乐颖: "基于深度学习的车道线检测算法研究", 《中国优秀硕士论文全文数据库信息科技辑》, no. 07, pages 138 - 1537 * |
| 王国文: "基于全向视觉的足球机器人目标识别与跟踪研究", no. 2020, pages 138 - 2063 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114092559A (en) * | 2021-11-30 | 2022-02-25 | 中德（珠海）人工智能研究院有限公司 | Training method and device for panoramic image feature point descriptor generation network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112184738B (en) | Image segmentation method, device, equipment and storage medium | |
| KR20200044108A (en) | Method and apparatus for estimating monocular image depth, device, program and storage medium | |
| KR20200087808A (en) | Method and apparatus for partitioning instances, electronic devices, programs and media | |
| CN112668588B (en) | Parking space information generation method, device, equipment and computer readable medium | |
| CN110516678B (en) | Image processing method and device | |
| CN110298851B (en) | Training method and device for human body segmentation neural network | |
| JP7249372B2 (en) | Methods and apparatus, electronic devices, computer readable storage media and computer programs for labeling objects | |
| WO2020062494A1 (en) | Image processing method and apparatus | |
| CN112631947B (en) | Test control method and device for application program, electronic equipment and storage medium | |
| CN111783777B (en) | Image processing method, apparatus, electronic device, and computer readable medium | |
| CN112907628A (en) | Video target tracking method and device, storage medium and electronic equipment | |
| CN111382695A (en) | Method and apparatus for detecting boundary points of object | |
| CN112418249A (en) | Mask image generation method and device, electronic equipment and computer readable medium | |
| CN110310293B (en) | Human body image segmentation method and device | |
| CN111311609B (en) | Image segmentation method and device, electronic equipment and storage medium | |
| CN111382696A (en) | Method and apparatus for detecting boundary points of object | |
| CN110223220B (en) | Method and device for processing image | |
| CN115115836B (en) | Image recognition method, device, storage medium and electronic equipment | |
| CN113688928B (en) | Image matching method and device, electronic equipment and computer readable medium | |
| CN115375656A (en) | Training method, segmentation method, device, medium, and apparatus for polyp segmentation model | |
| CN110633595B (en) | Target detection method and device by utilizing bilinear interpolation | |
| CN114399696A (en) | Target detection method and device, storage medium and electronic equipment | |
| CN115345931B (en) | Object attitude key point information generation method and device, electronic equipment and medium | |
| CN112215774B (en) | Model training and image defogging methods, apparatus, devices and computer readable media | |
| CN116704473B (en) | Obstacle information detection method, obstacle information detection device, electronic device, and computer-readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |