CN110458805A  A kind of plane monitoringnetwork method calculates equipment and circuit system  Google Patents
A kind of plane monitoringnetwork method calculates equipment and circuit system Download PDFInfo
 Publication number
 CN110458805A CN110458805A CN201910605510.4A CN201910605510A CN110458805A CN 110458805 A CN110458805 A CN 110458805A CN 201910605510 A CN201910605510 A CN 201910605510A CN 110458805 A CN110458805 A CN 110458805A
 Authority
 CN
 China
 Prior art keywords
 sub
 plane
 image
 image data
 cloud
 Prior art date
 Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
 Pending
Links
 230000000875 corresponding Effects 0.000 claims abstract description 435
 239000008264 clouds Substances 0.000 claims abstract description 376
 238000000605 extraction Methods 0.000 claims abstract description 283
 238000005457 optimization Methods 0.000 claims abstract description 25
 230000011218 segmentation Effects 0.000 claims description 90
 241001269238 Data Species 0.000 claims description 38
 230000004927 fusion Effects 0.000 claims description 23
 239000012141 concentrates Substances 0.000 claims description 16
 238000004220 aggregation Methods 0.000 claims description 11
 230000002776 aggregation Effects 0.000 claims description 11
 238000003860 storage Methods 0.000 claims description 11
 230000001131 transforming Effects 0.000 claims description 7
 238000004590 computer program Methods 0.000 claims description 4
 239000000203 mixtures Substances 0.000 claims description 4
 238000000034 methods Methods 0.000 description 125
 238000004422 calculation algorithm Methods 0.000 description 37
 238000010586 diagrams Methods 0.000 description 20
 238000004891 communication Methods 0.000 description 11
 238000007499 fusion processing Methods 0.000 description 10
 238000005530 etching Methods 0.000 description 9
 281000009961 Parallax, Inc. (company) companies 0.000 description 7
 239000010410 layers Substances 0.000 description 7
 230000001537 neural Effects 0.000 description 6
 230000003190 augmentative Effects 0.000 description 5
 230000003628 erosive Effects 0.000 description 5
 238000000638 solvent extraction Methods 0.000 description 5
 230000001429 stepping Effects 0.000 description 5
 238000010276 construction Methods 0.000 description 4
 281000053831 Porter (beer) companies 0.000 description 3
 238000004364 calculation methods Methods 0.000 description 3
 239000000686 essences Substances 0.000 description 3
 230000000051 modifying Effects 0.000 description 3
 230000003287 optical Effects 0.000 description 3
 230000005236 sound signal Effects 0.000 description 3
 230000015572 biosynthetic process Effects 0.000 description 2
 238000006243 chemical reactions Methods 0.000 description 2
 238000005520 cutting process Methods 0.000 description 2
 238000000151 deposition Methods 0.000 description 2
 230000000694 effects Effects 0.000 description 2
 238000005755 formation reactions Methods 0.000 description 2
 238000003709 image segmentation Methods 0.000 description 2
 238000003384 imaging method Methods 0.000 description 2
 230000000670 limiting Effects 0.000 description 2
 239000004973 liquid crystal related substances Substances 0.000 description 2
 239000000155 melts Substances 0.000 description 2
 230000002093 peripheral Effects 0.000 description 2
 238000000513 principal component analysis Methods 0.000 description 2
 230000002829 reduced Effects 0.000 description 2
 238000005070 sampling Methods 0.000 description 2
 239000007787 solids Substances 0.000 description 2
 238000006467 substitution reactions Methods 0.000 description 2
 230000001702 transmitter Effects 0.000 description 2
 210000000988 Bone and Bones Anatomy 0.000 description 1
 210000004556 Brain Anatomy 0.000 description 1
 210000000481 Breast Anatomy 0.000 description 1
 102100007513 Complement C4B Human genes 0.000 description 1
 101710065027 Complement C4b Proteins 0.000 description 1
 280000412365 I Level companies 0.000 description 1
 280000605883 Image Group companies 0.000 description 1
 280000141919 Network Communication companies 0.000 description 1
 240000004678 Panax pseudoginseng Species 0.000 description 1
 235000003140 Panax quinquefolius Nutrition 0.000 description 1
 280000399038 Republic companies 0.000 description 1
 230000001133 acceleration Effects 0.000 description 1
 239000002585 bases Substances 0.000 description 1
 230000006399 behavior Effects 0.000 description 1
 230000002457 bidirectional Effects 0.000 description 1
 230000005540 biological transmission Effects 0.000 description 1
 210000004027 cells Anatomy 0.000 description 1
 230000019771 cognition Effects 0.000 description 1
 230000000052 comparative effects Effects 0.000 description 1
 230000002596 correlated Effects 0.000 description 1
 235000019800 disodium phosphate Nutrition 0.000 description 1
 238000009826 distribution Methods 0.000 description 1
 235000013399 edible fruits Nutrition 0.000 description 1
 238000005516 engineering processes Methods 0.000 description 1
 230000002349 favourable Effects 0.000 description 1
 244000144992 flock Species 0.000 description 1
 235000005035 ginseng Nutrition 0.000 description 1
 235000008434 ginseng Nutrition 0.000 description 1
 230000005484 gravity Effects 0.000 description 1
 230000003450 growing Effects 0.000 description 1
 210000003702 immature single positive T cell Anatomy 0.000 description 1
 238000007689 inspection Methods 0.000 description 1
 230000002452 interceptive Effects 0.000 description 1
 210000002569 neurons Anatomy 0.000 description 1
 RZVAJINKPMORJFUHFFFAOYSAN pacetaminophenol Chemical compound data:image/svg+xml;base64,PD94bWwgdmVyc2lvbj0nMS4wJyBlbmNvZGluZz0naXNvLTg4NTktMSc/Pgo8c3ZnIHZlcnNpb249JzEuMScgYmFzZVByb2ZpbGU9J2Z1bGwnCiAgICAgICAgICAgICAgeG1sbnM9J2h0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnJwogICAgICAgICAgICAgICAgICAgICAgeG1sbnM6cmRraXQ9J2h0dHA6Ly93d3cucmRraXQub3JnL3htbCcKICAgICAgICAgICAgICAgICAgICAgIHhtbG5zOnhsaW5rPSdodHRwOi8vd3d3LnczLm9yZy8xOTk5L3hsaW5rJwogICAgICAgICAgICAgICAgICB4bWw6c3BhY2U9J3ByZXNlcnZlJwp3aWR0aD0nMzAwcHgnIGhlaWdodD0nMzAwcHgnIHZpZXdCb3g9JzAgMCAzMDAgMzAwJz4KPCEtLSBFTkQgT0YgSEVBREVSIC0tPgo8cmVjdCBzdHlsZT0nb3BhY2l0eToxLjA7ZmlsbDojRkZGRkZGO3N0cm9rZTpub25lJyB3aWR0aD0nMzAwJyBoZWlnaHQ9JzMwMCcgeD0nMCcgeT0nMCc+IDwvcmVjdD4KPHBhdGggY2xhc3M9J2JvbmQtMCcgZD0nTSAxMy42MzY0LDE1Ny45NCBMIDU4LjE2OTUsMTQ2LjAwOCcgc3R5bGU9J2ZpbGw6bm9uZTtmaWxsLXJ1bGU6ZXZlbm9kZDtzdHJva2U6IzNCNDE0MztzdHJva2Utd2lkdGg6Mi4wcHg7c3Ryb2tlLWxpbmVjYXA6YnV0dDtzdHJva2UtbGluZWpvaW46bWl0ZXI7c3Ryb2tlLW9wYWNpdHk6MScgLz4KPHBhdGggY2xhc3M9J2JvbmQtMScgZD0nTSA2Mi42MjI5LDE0Ny4yMDEgTCA2Ni4wMDYxLDEzNC41NzUnIHN0eWxlPSdmaWxsOm5vbmU7ZmlsbC1ydWxlOmV2ZW5vZGQ7c3Ryb2tlOiMzQjQxNDM7c3Ryb2tlLXdpZHRoOjIuMHB4O3N0cm9rZS1saW5lY2FwOmJ1dHQ7c3Ryb2tlLWxpbmVqb2luOm1pdGVyO3N0cm9rZS1vcGFjaXR5OjEnIC8+CjxwYXRoIGNsYXNzPSdib25kLTEnIGQ9J00gNjYuMDA2MSwxMzQuNTc1IEwgNjkuMzg5NCwxMjEuOTQ4JyBzdHlsZT0nZmlsbDpub25lO2ZpbGwtcnVsZTpldmVub2RkO3N0cm9rZTojRTg0MjM1O3N0cm9rZS13aWR0aDoyLjBweDtzdHJva2UtbGluZWNhcDpidXR0O3N0cm9rZS1saW5lam9pbjptaXRlcjtzdHJva2Utb3BhY2l0eToxJyAvPgo8cGF0aCBjbGFzcz0nYm9uZC0xJyBkPSdNIDUzLjcxNjIsMTQ0LjgxNSBMIDU3LjA5OTUsMTMyLjE4OCcgc3R5bGU9J2ZpbGw6bm9uZTtmaWxsLXJ1bGU6ZXZlbm9kZDtzdHJva2U6IzNCNDE0MztzdHJva2Utd2lkdGg6Mi4wcHg7c3Ryb2tlLWxpbmVjYXA6YnV0dDtzdHJva2UtbGluZWpvaW46bWl0ZXI7c3Ryb2tlLW9wYWNpdHk6MScgLz4KPHBhdGggY2xhc3M9J2JvbmQtMScgZD0nTSA1Ny4wOTk1LDEzMi4xODggTCA2MC40ODI4LDExOS41NjEnIHN0eWxlPSdmaWxsOm5vbmU7ZmlsbC1ydWxlOmV2ZW5vZGQ7c3Ryb2tlOiNFODQyMzU7c3Ryb2tlLXdpZHRoOjIuMHB4O3N0cm9rZS1saW5lY2FwOmJ1dHQ7c3Ryb2tlLWxpbmVqb2luOm1pdGVyO3N0cm9rZS1vcGFjaXR5OjEnIC8+CjxwYXRoIGNsYXNzPSdib25kLTInIGQ9J00gNTguMTY5NSwxNDYuMDA4IEwgNjcuODY3MiwxNTUuNzA1JyBzdHlsZT0nZmlsbDpub25lO2ZpbGwtcnVsZTpldmVub2RkO3N0cm9rZTojM0I0MTQzO3N0cm9rZS13aWR0aDoyLjBweDtzdHJva2UtbGluZWNhcDpidXR0O3N0cm9rZS1saW5lam9pbjptaXRlcjtzdHJva2Utb3BhY2l0eToxJyAvPgo8cGF0aCBjbGFzcz0nYm9uZC0yJyBkPSdNIDY3Ljg2NzIsMTU1LjcwNSBMIDc3LjU2NDksMTY1LjQwMycgc3R5bGU9J2ZpbGw6bm9uZTtmaWxsLXJ1bGU6ZXZlbm9kZDtzdHJva2U6IzQyODRGNDtzdHJva2Utd2lkdGg6Mi4wcHg7c3Ryb2tlLWxpbmVjYXA6YnV0dDtzdHJva2UtbGluZWpvaW46bWl0ZXI7c3Ryb2tlLW9wYWNpdHk6MScgLz4KPHBhdGggY2xhc3M9J2JvbmQtMycgZD0nTSAxMDYuMzYyLDE3NC40MzEgTCAxMjAuODMzLDE3MC41NTMnIHN0eWxlPSdmaWxsOm5vbmU7ZmlsbC1ydWxlOmV2ZW5vZGQ7c3Ryb2tlOiM0Mjg0RjQ7c3Ryb2tlLXdpZHRoOjIuMHB4O3N0cm9rZS1saW5lY2FwOmJ1dHQ7c3Ryb2tlLWxpbmVqb2luOm1pdGVyO3N0cm9rZS1vcGFjaXR5OjEnIC8+CjxwYXRoIGNsYXNzPSdib25kLTMnIGQ9J00gMTIwLjgzMywxNzAuNTUzIEwgMTM1LjMwMywxNjYuNjc2JyBzdHlsZT0nZmlsbDpub25lO2ZpbGwtcnVsZTpldmVub2RkO3N0cm9rZTojM0I0MTQzO3N0cm9rZS13aWR0aDoyLjBweDtzdHJva2UtbGluZWNhcDpidXR0O3N0cm9rZS1saW5lam9pbjptaXRlcjtzdHJva2Utb3BhY2l0eToxJyAvPgo8cGF0aCBjbGFzcz0nYm9uZC00JyBkPSdNIDEzNS4zMDMsMTY2LjY3NiBMIDE0Ny4yMzYsMTIyLjE0Mycgc3R5bGU9J2ZpbGw6bm9uZTtmaWxsLXJ1bGU6ZXZlbm9kZDtzdHJva2U6IzNCNDE0MztzdHJva2Utd2lkdGg6Mi4wcHg7c3Ryb2tlLWxpbmVjYXA6YnV0dDtzdHJva2UtbGluZWpvaW46bWl0ZXI7c3Ryb2tlLW9wYWNpdHk6MScgLz4KPHBhdGggY2xhc3M9J2JvbmQtNCcgZD0nTSAxNDYsMTYyLjM4MiBMIDE1NC4zNTMsMTMxLjIwOScgc3R5bGU9J2ZpbGw6bm9uZTtmaWxsLXJ1bGU6ZXZlbm9kZDtzdHJva2U6IzNCNDE0MztzdHJva2Utd2lkdGg6Mi4wcHg7c3Ryb2tlLWxpbmVjYXA6YnV0dDtzdHJva2UtbGluZWpvaW46bWl0ZXI7c3Ryb2tlLW9wYWNpdHk6MScgLz4KPHBhdGggY2xhc3M9J2JvbmQtMTAnIGQ9J00gMTM1LjMwMywxNjYuNjc2IEwgMTY3LjkwNCwxOTkuMjc2JyBzdHlsZT0nZmlsbDpub25lO2ZpbGwtcnVsZTpldmVub2RkO3N0cm9rZTojM0I0MTQzO3N0cm9rZS13aWR0aDoyLjBweDtzdHJva2UtbGluZWNhcDpidXR0O3N0cm9rZS1saW5lam9pbjptaXRlcjtzdHJva2Utb3BhY2l0eToxJyAvPgo8cGF0aCBjbGFzcz0nYm9uZC01JyBkPSdNIDE0Ny4yMzYsMTIyLjE0MyBMIDE5MS43NjksMTEwLjIxJyBzdHlsZT0nZmlsbDpub25lO2ZpbGwtcnVsZTpldmVub2RkO3N0cm9rZTojM0I0MTQzO3N0cm9rZS13aWR0aDoyLjBweDtzdHJva2UtbGluZWNhcDpidXR0O3N0cm9rZS1saW5lam9pbjptaXRlcjtzdHJva2Utb3BhY2l0eToxJyAvPgo8cGF0aCBjbGFzcz0nYm9uZC02JyBkPSdNIDE5MS43NjksMTEwLjIxIEwgMjI0LjM3LDE0Mi44MScgc3R5bGU9J2ZpbGw6bm9uZTtmaWxsLXJ1bGU6ZXZlbm9kZDtzdHJva2U6IzNCNDE0MztzdHJva2Utd2lkdGg6Mi4wcHg7c3Ryb2tlLWxpbmVjYXA6YnV0dDtzdHJva2UtbGluZWpvaW46bWl0ZXI7c3Ryb2tlLW9wYWNpdHk6MScgLz4KPHBhdGggY2xhc3M9J2JvbmQtNicgZD0nTSAxOTAuMTM5LDEyMS42MiBMIDIxMi45NTksMTQ0LjQ0MScgc3R5bGU9J2ZpbGw6bm9uZTtmaWxsLXJ1bGU6ZXZlbm9kZDtzdHJva2U6IzNCNDE0MztzdHJva2Utd2lkdGg6Mi4wcHg7c3Ryb2tlLWxpbmVjYXA6YnV0dDtzdHJva2UtbGluZWpvaW46bWl0ZXI7c3Ryb2tlLW9wYWNpdHk6MScgLz4KPHBhdGggY2xhc3M9J2JvbmQtNycgZD0nTSAyMjQuMzcsMTQyLjgxIEwgMjM4Ljg0LDEzOC45MzMnIHN0eWxlPSdmaWxsOm5vbmU7ZmlsbC1ydWxlOmV2ZW5vZGQ7c3Ryb2tlOiMzQjQxNDM7c3Ryb2tlLXdpZHRoOjIuMHB4O3N0cm9rZS1saW5lY2FwOmJ1dHQ7c3Ryb2tlLWxpbmVqb2luOm1pdGVyO3N0cm9rZS1vcGFjaXR5OjEnIC8+CjxwYXRoIGNsYXNzPSdib25kLTcnIGQ9J00gMjM4Ljg0LDEzOC45MzMgTCAyNTMuMzExLDEzNS4wNTYnIHN0eWxlPSdmaWxsOm5vbmU7ZmlsbC1ydWxlOmV2ZW5vZGQ7c3Ryb2tlOiNFODQyMzU7c3Ryb2tlLXdpZHRoOjIuMHB4O3N0cm9rZS1saW5lY2FwOmJ1dHQ7c3Ryb2tlLWxpbmVqb2luOm1pdGVyO3N0cm9rZS1vcGFjaXR5OjEnIC8+CjxwYXRoIGNsYXNzPSdib25kLTgnIGQ9J00gMjI0LjM3LDE0Mi44MSBMIDIxMi40MzcsMTg3LjM0NCcgc3R5bGU9J2ZpbGw6bm9uZTtmaWxsLXJ1bGU6ZXZlbm9kZDtzdHJva2U6IzNCNDE0MztzdHJva2Utd2lkdGg6Mi4wcHg7c3Ryb2tlLWxpbmVjYXA6YnV0dDtzdHJva2UtbGluZWpvaW46bWl0ZXI7c3Ryb2tlLW9wYWNpdHk6MScgLz4KPHBhdGggY2xhc3M9J2JvbmQtOScgZD0nTSAyMTIuNDM3LDE4Ny4zNDQgTCAxNjcuOTA0LDE5OS4yNzYnIHN0eWxlPSdmaWxsOm5vbmU7ZmlsbC1ydWxlOmV2ZW5vZGQ7c3Ryb2tlOiMzQjQxNDM7c3Ryb2tlLXdpZHRoOjIuMHB4O3N0cm9rZS1saW5lY2FwOmJ1dHQ7c3Ryb2tlLWxpbmVqb2luOm1pdGVyO3N0cm9rZS1vcGFjaXR5OjEnIC8+CjxwYXRoIGNsYXNzPSdib25kLTknIGQ9J00gMjAzLjM3MSwxODAuMjI3IEwgMTcyLjE5NywxODguNTgnIHN0eWxlPSdmaWxsOm5vbmU7ZmlsbC1ydWxlOmV2ZW5vZGQ7c3Ryb2tlOiMzQjQxNDM7c3Ryb2tlLXdpZHRoOjIuMHB4O3N0cm9rZS1saW5lY2FwOmJ1dHQ7c3Ryb2tlLWxpbmVqb2luOm1pdGVyO3N0cm9rZS1vcGFjaXR5OjEnIC8+Cjx0ZXh0IHg9JzY0LjU2OTcnIHk9JzExMC42OTUnIGNsYXNzPSdhdG9tLTInIHN0eWxlPSdmb250LXNpemU6MThweDtmb250LXN0eWxlOm5vcm1hbDtmb250LXdlaWdodDpub3JtYWw7ZmlsbC1vcGFjaXR5OjE7c3Ryb2tlOm5vbmU7Zm9udC1mYW1pbHk6c2Fucy1zZXJpZjt0ZXh0LWFuY2hvcjpzdGFydDtmaWxsOiNFODQyMzUnID5PPC90ZXh0Pgo8dGV4dCB4PSc4NS4yMzc2JyB5PScxODcuODI5JyBjbGFzcz0nYXRvbS0zJyBzdHlsZT0nZm9udC1zaXplOjE4cHg7Zm9udC1zdHlsZTpub3JtYWw7Zm9udC13ZWlnaHQ6bm9ybWFsO2ZpbGwtb3BhY2l0eToxO3N0cm9rZTpub25lO2ZvbnQtZmFtaWx5OnNhbnMtc2VyaWY7dGV4dC1hbmNob3I6c3RhcnQ7ZmlsbDojNDI4NEY0JyA+TjwvdGV4dD4KPHRleHQgeD0nODUuMjM3NicgeT0nMjA0LjA1OCcgY2xhc3M9J2F0b20tMycgc3R5bGU9J2ZvbnQtc2l6ZToxOHB4O2ZvbnQtc3R5bGU6bm9ybWFsO2ZvbnQtd2VpZ2h0Om5vcm1hbDtmaWxsLW9wYWNpdHk6MTtzdHJva2U6bm9uZTtmb250LWZhbWlseTpzYW5zLXNlcmlmO3RleHQtYW5jaG9yOnN0YXJ0O2ZpbGw6IzQyODRGNCcgPkg8L3RleHQ+Cjx0ZXh0IHg9JzI2My4zNycgeT0nMTQwLjA5OScgY2xhc3M9J2F0b20tOCcgc3R5bGU9J2ZvbnQtc2l6ZToxOHB4O2ZvbnQtc3R5bGU6bm9ybWFsO2ZvbnQtd2VpZ2h0Om5vcm1hbDtmaWxsLW9wYWNpdHk6MTtzdHJva2U6bm9uZTtmb250LWZhbWlseTpzYW5zLXNlcmlmO3RleHQtYW5jaG9yOnN0YXJ0O2ZpbGw6I0U4NDIzNScgPk88L3RleHQ+Cjx0ZXh0IHg9JzI3Ni4wOTUnIHk9JzE0MC4wOTknIGNsYXNzPSdhdG9tLTgnIHN0eWxlPSdmb250LXNpemU6MThweDtmb250LXN0eWxlOm5vcm1hbDtmb250LXdlaWdodDpub3JtYWw7ZmlsbC1vcGFjaXR5OjE7c3Ryb2tlOm5vbmU7Zm9udC1mYW1pbHk6c2Fucy1zZXJpZjt0ZXh0LWFuY2hvcjpzdGFydDtmaWxsOiNFODQyMzUnID5IPC90ZXh0Pgo8L3N2Zz4K data:image/svg+xml;base64,PD94bWwgdmVyc2lvbj0nMS4wJyBlbmNvZGluZz0naXNvLTg4NTktMSc/Pgo8c3ZnIHZlcnNpb249JzEuMScgYmFzZVByb2ZpbGU9J2Z1bGwnCiAgICAgICAgICAgICAgeG1sbnM9J2h0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnJwogICAgICAgICAgICAgICAgICAgICAgeG1sbnM6cmRraXQ9J2h0dHA6Ly93d3cucmRraXQub3JnL3htbCcKICAgICAgICAgICAgICAgICAgICAgIHhtbG5zOnhsaW5rPSdodHRwOi8vd3d3LnczLm9yZy8xOTk5L3hsaW5rJwogICAgICAgICAgICAgICAgICB4bWw6c3BhY2U9J3ByZXNlcnZlJwp3aWR0aD0nODVweCcgaGVpZ2h0PSc4NXB4JyB2aWV3Qm94PScwIDAgODUgODUnPgo8IS0tIEVORCBPRiBIRUFERVIgLS0+CjxyZWN0IHN0eWxlPSdvcGFjaXR5OjEuMDtmaWxsOiNGRkZGRkY7c3Ryb2tlOm5vbmUnIHdpZHRoPSc4NScgaGVpZ2h0PSc4NScgeD0nMCcgeT0nMCc+IDwvcmVjdD4KPHBhdGggY2xhc3M9J2JvbmQtMCcgZD0nTSAzLjM2MzY0LDQ0LjEyIEwgMTUuOTQ2LDQwLjc0ODUnIHN0eWxlPSdmaWxsOm5vbmU7ZmlsbC1ydWxlOmV2ZW5vZGQ7c3Ryb2tlOiMzQjQxNDM7c3Ryb2tlLXdpZHRoOjEuMHB4O3N0cm9rZS1saW5lY2FwOmJ1dHQ7c3Ryb2tlLWxpbmVqb2luOm1pdGVyO3N0cm9rZS1vcGFjaXR5OjEnIC8+CjxwYXRoIGNsYXNzPSdib25kLTEnIGQ9J00gMTcuMjA0Miw0MS4wODU3IEwgMTguNDQ0NCwzNi40NTczJyBzdHlsZT0nZmlsbDpub25lO2ZpbGwtcnVsZTpldmVub2RkO3N0cm9rZTojM0I0MTQzO3N0cm9rZS13aWR0aDoxLjBweDtzdHJva2UtbGluZWNhcDpidXR0O3N0cm9rZS1saW5lam9pbjptaXRlcjtzdHJva2Utb3BhY2l0eToxJyAvPgo8cGF0aCBjbGFzcz0nYm9uZC0xJyBkPSdNIDE4LjQ0NDQsMzYuNDU3MyBMIDE5LjY4NDYsMzEuODI5JyBzdHlsZT0nZmlsbDpub25lO2ZpbGwtcnVsZTpldmVub2RkO3N0cm9rZTojRTg0MjM1O3N0cm9rZS13aWR0aDoxLjBweDtzdHJva2UtbGluZWNhcDpidXR0O3N0cm9rZS1saW5lam9pbjptaXRlcjtzdHJva2Utb3BhY2l0eToxJyAvPgo8cGF0aCBjbGFzcz0nYm9uZC0xJyBkPSdNIDE0LjY4NzgsNDAuNDExNCBMIDE1LjkyNzksMzUuNzgzJyBzdHlsZT0nZmlsbDpub25lO2ZpbGwtcnVsZTpldmVub2RkO3N0cm9rZTojM0I0MTQzO3N0cm9rZS13aWR0aDoxLjBweDtzdHJva2UtbGluZWNhcDpidXR0O3N0cm9rZS1saW5lam9pbjptaXRlcjtzdHJva2Utb3BhY2l0eToxJyAvPgo8cGF0aCBjbGFzcz0nYm9uZC0xJyBkPSdNIDE1LjkyNzksMzUuNzgzIEwgMTcuMTY4MSwzMS4xNTQ3JyBzdHlsZT0nZmlsbDpub25lO2ZpbGwtcnVsZTpldmVub2RkO3N0cm9rZTojRTg0MjM1O3N0cm9rZS13aWR0aDoxLjBweDtzdHJva2UtbGluZWNhcDpidXR0O3N0cm9rZS1saW5lam9pbjptaXRlcjtzdHJva2Utb3BhY2l0eToxJyAvPgo8cGF0aCBjbGFzcz0nYm9uZC0yJyBkPSdNIDE1Ljk0Niw0MC43NDg1IEwgMTkuNDg4Niw0NC4yOTEyJyBzdHlsZT0nZmlsbDpub25lO2ZpbGwtcnVsZTpldmVub2RkO3N0cm9rZTojM0I0MTQzO3N0cm9rZS13aWR0aDoxLjBweDtzdHJva2UtbGluZWNhcDpidXR0O3N0cm9rZS1saW5lam9pbjptaXRlcjtzdHJva2Utb3BhY2l0eToxJyAvPgo8cGF0aCBjbGFzcz0nYm9uZC0yJyBkPSdNIDE5LjQ4ODYsNDQuMjkxMiBMIDIzLjAzMTMsNDcuODMzOCcgc3R5bGU9J2ZpbGw6bm9uZTtmaWxsLXJ1bGU6ZXZlbm9kZDtzdHJva2U6IzQyODRGNDtzdHJva2Utd2lkdGg6MS4wcHg7c3Ryb2tlLWxpbmVjYXA6YnV0dDtzdHJva2UtbGluZWpvaW46bWl0ZXI7c3Ryb2tlLW9wYWNpdHk6MScgLz4KPHBhdGggY2xhc3M9J2JvbmQtMycgZD0nTSAyNy4yODI2LDQ5LjM4OTkgTCAzMi41MTA5LDQ3Ljk4OScgc3R5bGU9J2ZpbGw6bm9uZTtmaWxsLXJ1bGU6ZXZlbm9kZDtzdHJva2U6IzQyODRGNDtzdHJva2Utd2lkdGg6MS4wcHg7c3Ryb2tlLWxpbmVjYXA6YnV0dDtzdHJva2UtbGluZWpvaW46bWl0ZXI7c3Ryb2tlLW9wYWNpdHk6MScgLz4KPHBhdGggY2xhc3M9J2JvbmQtMycgZD0nTSAzMi41MTA5LDQ3Ljk4OSBMIDM3LjczOTMsNDYuNTg4JyBzdHlsZT0nZmlsbDpub25lO2ZpbGwtcnVsZTpldmVub2RkO3N0cm9rZTojM0I0MTQzO3N0cm9rZS13aWR0aDoxLjBweDtzdHJva2UtbGluZWNhcDpidXR0O3N0cm9rZS1saW5lam9pbjptaXRlcjtzdHJva2Utb3BhY2l0eToxJyAvPgo8cGF0aCBjbGFzcz0nYm9uZC00JyBkPSdNIDM3LjczOTMsNDYuNTg4IEwgNDEuMTEwNywzNC4wMDU2JyBzdHlsZT0nZmlsbDpub25lO2ZpbGwtcnVsZTpldmVub2RkO3N0cm9rZTojM0I0MTQzO3N0cm9rZS13aWR0aDoxLjBweDtzdHJva2UtbGluZWNhcDpidXR0O3N0cm9rZS1saW5lam9pbjptaXRlcjtzdHJva2Utb3BhY2l0eToxJyAvPgo8cGF0aCBjbGFzcz0nYm9uZC00JyBkPSdNIDQwLjc2MTUsNDUuMzc0OSBMIDQzLjEyMTUsMzYuNTY3Mycgc3R5bGU9J2ZpbGw6bm9uZTtmaWxsLXJ1bGU6ZXZlbm9kZDtzdHJva2U6IzNCNDE0MztzdHJva2Utd2lkdGg6MS4wcHg7c3Ryb2tlLWxpbmVjYXA6YnV0dDtzdHJva2UtbGluZWpvaW46bWl0ZXI7c3Ryb2tlLW9wYWNpdHk6MScgLz4KPHBhdGggY2xhc3M9J2JvbmQtMTAnIGQ9J00gMzcuNzM5Myw0Ni41ODggTCA0Ni45NTAyLDU1Ljc5ODknIHN0eWxlPSdmaWxsOm5vbmU7ZmlsbC1ydWxlOmV2ZW5vZGQ7c3Ryb2tlOiMzQjQxNDM7c3Ryb2tlLXdpZHRoOjEuMHB4O3N0cm9rZS1saW5lY2FwOmJ1dHQ7c3Ryb2tlLWxpbmVqb2luOm1pdGVyO3N0cm9rZS1vcGFjaXR5OjEnIC8+CjxwYXRoIGNsYXNzPSdib25kLTUnIGQ9J00gNDEuMTEwNywzNC4wMDU2IEwgNTMuNjkzMSwzMC42MzQyJyBzdHlsZT0nZmlsbDpub25lO2ZpbGwtcnVsZTpldmVub2RkO3N0cm9rZTojM0I0MTQzO3N0cm9rZS13aWR0aDoxLjBweDtzdHJva2UtbGluZWNhcDpidXR0O3N0cm9rZS1saW5lam9pbjptaXRlcjtzdHJva2Utb3BhY2l0eToxJyAvPgo8cGF0aCBjbGFzcz0nYm9uZC02JyBkPSdNIDUzLjY5MzEsMzAuNjM0MiBMIDYyLjkwNCwzOS44NDUxJyBzdHlsZT0nZmlsbDpub25lO2ZpbGwtcnVsZTpldmVub2RkO3N0cm9rZTojM0I0MTQzO3N0cm9rZS13aWR0aDoxLjBweDtzdHJva2UtbGluZWNhcDpidXR0O3N0cm9rZS1saW5lam9pbjptaXRlcjtzdHJva2Utb3BhY2l0eToxJyAvPgo8cGF0aCBjbGFzcz0nYm9uZC02JyBkPSdNIDUzLjIzMjYsMzMuODU4IEwgNTkuNjgwMiw0MC4zMDU3JyBzdHlsZT0nZmlsbDpub25lO2ZpbGwtcnVsZTpldmVub2RkO3N0cm9rZTojM0I0MTQzO3N0cm9rZS13aWR0aDoxLjBweDtzdHJva2UtbGluZWNhcDpidXR0O3N0cm9rZS1saW5lam9pbjptaXRlcjtzdHJva2Utb3BhY2l0eToxJyAvPgo8cGF0aCBjbGFzcz0nYm9uZC03JyBkPSdNIDYyLjkwNCwzOS44NDUxIEwgNjguMTMyNCwzOC40NDQyJyBzdHlsZT0nZmlsbDpub25lO2ZpbGwtcnVsZTpldmVub2RkO3N0cm9rZTojM0I0MTQzO3N0cm9rZS13aWR0aDoxLjBweDtzdHJva2UtbGluZWNhcDpidXR0O3N0cm9rZS1saW5lam9pbjptaXRlcjtzdHJva2Utb3BhY2l0eToxJyAvPgo8cGF0aCBjbGFzcz0nYm9uZC03JyBkPSdNIDY4LjEzMjQsMzguNDQ0MiBMIDczLjM2MDgsMzcuMDQzMycgc3R5bGU9J2ZpbGw6bm9uZTtmaWxsLXJ1bGU6ZXZlbm9kZDtzdHJva2U6I0U4NDIzNTtzdHJva2Utd2lkdGg6MS4wcHg7c3Ryb2tlLWxpbmVjYXA6YnV0dDtzdHJva2UtbGluZWpvaW46bWl0ZXI7c3Ryb2tlLW9wYWNpdHk6MScgLz4KPHBhdGggY2xhc3M9J2JvbmQtOCcgZD0nTSA2Mi45MDQsMzkuODQ1MSBMIDU5LjUzMjYsNTIuNDI3NScgc3R5bGU9J2ZpbGw6bm9uZTtmaWxsLXJ1bGU6ZXZlbm9kZDtzdHJva2U6IzNCNDE0MztzdHJva2Utd2lkdGg6MS4wcHg7c3Ryb2tlLWxpbmVjYXA6YnV0dDtzdHJva2UtbGluZWpvaW46bWl0ZXI7c3Ryb2tlLW9wYWNpdHk6MScgLz4KPHBhdGggY2xhc3M9J2JvbmQtOScgZD0nTSA1OS41MzI2LDUyLjQyNzUgTCA0Ni45NTAyLDU1Ljc5ODknIHN0eWxlPSdmaWxsOm5vbmU7ZmlsbC1ydWxlOmV2ZW5vZGQ7c3Ryb2tlOiMzQjQxNDM7c3Ryb2tlLXdpZHRoOjEuMHB4O3N0cm9rZS1saW5lY2FwOmJ1dHQ7c3Ryb2tlLWxpbmVqb2luOm1pdGVyO3N0cm9rZS1vcGFjaXR5OjEnIC8+CjxwYXRoIGNsYXNzPSdib25kLTknIGQ9J00gNTYuOTcxLDUwLjQxNjggTCA0OC4xNjMzLDUyLjc3NjgnIHN0eWxlPSdmaWxsOm5vbmU7ZmlsbC1ydWxlOmV2ZW5vZGQ7c3Ryb2tlOiMzQjQxNDM7c3Ryb2tlLXdpZHRoOjEuMHB4O3N0cm9rZS1saW5lY2FwOmJ1dHQ7c3Ryb2tlLWxpbmVqb2luOm1pdGVyO3N0cm9rZS1vcGFjaXR5OjEnIC8+Cjx0ZXh0IHg9JzE3LjUxNzQnIHk9JzMxLjE2NjInIGNsYXNzPSdhdG9tLTInIHN0eWxlPSdmb250LXNpemU6NnB4O2ZvbnQtc3R5bGU6bm9ybWFsO2ZvbnQtd2VpZ2h0Om5vcm1hbDtmaWxsLW9wYWNpdHk6MTtzdHJva2U6bm9uZTtmb250LWZhbWlseTpzYW5zLXNlcmlmO3RleHQtYW5jaG9yOnN0YXJ0O2ZpbGw6I0U4NDIzNScgPk88L3RleHQ+Cjx0ZXh0IHg9JzIzLjM1NjknIHk9JzUyLjk1OTUnIGNsYXNzPSdhdG9tLTMnIHN0eWxlPSdmb250LXNpemU6NnB4O2ZvbnQtc3R5bGU6bm9ybWFsO2ZvbnQtd2VpZ2h0Om5vcm1hbDtmaWxsLW9wYWNpdHk6MTtzdHJva2U6bm9uZTtmb250LWZhbWlseTpzYW5zLXNlcmlmO3RleHQtYW5jaG9yOnN0YXJ0O2ZpbGw6IzQyODRGNCcgPk48L3RleHQ+Cjx0ZXh0IHg9JzIzLjM1NjknIHk9JzU4LjIzOTUnIGNsYXNzPSdhdG9tLTMnIHN0eWxlPSdmb250LXNpemU6NnB4O2ZvbnQtc3R5bGU6bm9ybWFsO2ZvbnQtd2VpZ2h0Om5vcm1hbDtmaWxsLW9wYWNpdHk6MTtzdHJva2U6bm9uZTtmb250LWZhbWlseTpzYW5zLXNlcmlmO3RleHQtYW5jaG9yOnN0YXJ0O2ZpbGw6IzQyODRGNCcgPkg8L3RleHQ+Cjx0ZXh0IHg9JzczLjY4NjQnIHk9JzM5LjQ3MzcnIGNsYXNzPSdhdG9tLTgnIHN0eWxlPSdmb250LXNpemU6NnB4O2ZvbnQtc3R5bGU6bm9ybWFsO2ZvbnQtd2VpZ2h0Om5vcm1hbDtmaWxsLW9wYWNpdHk6MTtzdHJva2U6bm9uZTtmb250LWZhbWlseTpzYW5zLXNlcmlmO3RleHQtYW5jaG9yOnN0YXJ0O2ZpbGw6I0U4NDIzNScgPk88L3RleHQ+Cjx0ZXh0IHg9Jzc3LjgyNjQnIHk9JzM5LjQ3MzcnIGNsYXNzPSdhdG9tLTgnIHN0eWxlPSdmb250LXNpemU6NnB4O2ZvbnQtc3R5bGU6bm9ybWFsO2ZvbnQtd2VpZ2h0Om5vcm1hbDtmaWxsLW9wYWNpdHk6MTtzdHJva2U6bm9uZTtmb250LWZhbWlseTpzYW5zLXNlcmlmO3RleHQtYW5jaG9yOnN0YXJ0O2ZpbGw6I0U4NDIzNScgPkg8L3RleHQ+Cjwvc3ZnPgo= CC(=O)NC1=CC=C(O)C=C1 RZVAJINKPMORJFUHFFFAOYSAN 0.000 description 1
 230000036961 partial Effects 0.000 description 1
 239000011433 polymer cement mortar Substances 0.000 description 1
 238000003325 tomography Methods 0.000 description 1
 230000000007 visual effect Effects 0.000 description 1
Classifications

 G—PHYSICS
 G06—COMPUTING; CALCULATING; COUNTING
 G06K—RECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
 G06K9/00—Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
 G06K9/62—Methods or arrangements for recognition using electronic means
 G06K9/6217—Design or setup of recognition systems and techniques; Extraction of features in feature space; Clustering techniques; Blind source separation
 G06K9/6218—Clustering techniques
 G06K9/622—Nonhierarchical partitioning techniques
 G06K9/6221—Nonhierarchical partitioning techniques based on statistics

 G—PHYSICS
 G06—COMPUTING; CALCULATING; COUNTING
 G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
 G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
 G06T17/005—Tree description, e.g. octree, quadtree

 G—PHYSICS
 G06—COMPUTING; CALCULATING; COUNTING
 G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
 G06T7/00—Image analysis
 G06T7/0002—Inspection of images, e.g. flaw detection

 G—PHYSICS
 G06—COMPUTING; CALCULATING; COUNTING
 G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
 G06T7/00—Image analysis
 G06T7/10—Segmentation; Edge detection
 G06T7/136—Segmentation; Edge detection involving thresholding

 G—PHYSICS
 G06—COMPUTING; CALCULATING; COUNTING
 G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
 G06T2207/00—Indexing scheme for image analysis or image enhancement
 G06T2207/10—Image acquisition modality
 G06T2207/10024—Color image

 G—PHYSICS
 G06—COMPUTING; CALCULATING; COUNTING
 G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
 G06T2207/00—Indexing scheme for image analysis or image enhancement
 G06T2207/10—Image acquisition modality
 G06T2207/10028—Range image; Depth image; 3D point clouds

 G—PHYSICS
 G06—COMPUTING; CALCULATING; COUNTING
 G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
 G06T2207/00—Indexing scheme for image analysis or image enhancement
 G06T2207/20—Special algorithmic details
 G06T2207/20212—Image combination
 G06T2207/20221—Image fusion; Image merging
Abstract
This application discloses a kind of plane monitoringnetwork method, equipment and circuit system are calculated, this method comprises: calculating equipment obtains image data to be processed, then image data to be processed is split, obtains N number of subimage data, wherein N is the integer greater than 1.Later, determine the corresponding point cloud information of at least one subimage data in N number of subimage data, according to the corresponding point cloud information of at least one subimage data in N number of subimage data, clustering processing is carried out to corresponding cloud of N number of subimage data, K coarse extraction plane is obtained, processing is optimized to K coarse extraction plane, obtains plane after L optimization, wherein K is the positive integer no more than N, and L is the positive integer no more than K.It can make to calculate the more than one plane that equipment is able to detect that in image data in this way.
Description
This application claims on 03 26th, 2019 submission Patent Office of the People's Republic of China, application No. is 201910234537.7, application
The priority of the Chinese patent application of entitled " a kind of plane monitoringnetwork method and electronic equipment ", entire contents pass through reference knot
It closes in this application.
Technical field
This application involves technical field of computer vision more particularly to a kind of plane monitoringnetwork method, calculate equipment and electricity
Road system.
Background technique
Currently, object dimensional based on mobile phone rebuild and augmented reality (augmented reality, AR) game at
For reality, and by the favor of major cell phone manufacturer and users.The meter with augmented reality game is rebuild in object dimensional
During calculation, threedimensional space plane monitoringnetwork is an important and basic function, because after detecting plane, ability
The anchor point of object is further determined that, to render object at the anchor point determined.Currently, threedimensional planar detection function is
It is injected towards in various small devices, but the computing capability of these equipment is limited, can not handle the higher algorithm of complexity.
Modern AR application program, such as AR game, object, personage's modeling program, interactive process such as based on complex scene,
Mixed reality application etc. proposes very high requirement to the scene understanding ability of rear end program, and one of them are namely based on image biography
The threedimensional structure of sensor or depth transducer comprehension of information scene.From the point of view of current mobile terminal computing capability, current back end journey
Ordered pair scene understands that the processing capacity of algorithm is very limited.Before this, the dominant plane structure in scene can only be all determined,
The position of maximum plane in three dimensions i.e. in scene.Commercial AR solution is also substantially based in scene now
Principal plane (i.e. single plane) carry out the work in later period, including model, render.
As favorable rating of the consumer to artificial intelligence application increasingly increases, demand of the user to function of application
It gradually increases, requirement of the augmented reality application program to rear end scene understanding ability also increases accordingly, and single plane monitoringnetwork is
Through being unable to satisfy the use demand of user.
Summary of the invention
The application provides a kind of plane monitoringnetwork method, calculates equipment and circuit system, use so that calculate equipment can
Detect the more than one plane in image data.
In a first aspect, the embodiment of the present application provides a kind of plane monitoringnetwork method, this method is applied to calculate equipment, the party
Method includes: to calculate equipment to obtain image data to be processed, is then split to image data to be processed, obtains N number of subgraph
Data, wherein N is the integer greater than 1.Later, it calculates equipment and determines at least one subimage data in N number of subimage data
Corresponding point cloud information, then according to the corresponding point cloud information of at least one subimage data in N number of subimage data, to N
Corresponding cloud of a subimage data carries out clustering processing, obtains K coarse extraction plane, optimizes to K coarse extraction plane
Processing obtains plane after L optimization, and wherein K is the positive integer no more than N, and L is the positive integer no more than K.
In the embodiment of the present application, by being split to image data to be processed, N number of subimage data is obtained, later to N
Corresponding cloud of a subimage data carries out clustering processing, compared to the prior art in from center to begin stepping through image to be processed straight
To detecting that a plane just stops detecting, it is only able to detect the scheme of a plane, scheme provided by the present application can be treated
N number of subimage data that processing image segmentation obtains carries out clustering processing, and calculating equipment can be made to be able to detect that figure in this way
As the more than one plane in data.
In a kind of possible design, which is depth image, includes each picture in the depth image
The image coordinate and depth value of vegetarian refreshments.It calculates equipment and gets depth image, depth image is split later, is obtained N number of
Then sub depth image calculates equipment and determines corresponding cloud letter of the sub depth image of at least one of N number of sub depth image
Breath, and according to the corresponding point cloud information of each subimage data in N number of sub depth image, determine at least one sub depth image
The mean square error of plane after the fitting of corresponding cloud.Later, it calculates equipment and determines satisfaction first from N number of sub depth image
The sub depth image of condition forms subgraph image set to be processed, concentrates the sub depth image for including corresponding subgraph to be processed
Point cloud is clustered, and K coarse extraction plane is obtained.Wherein, first condition include corresponding cloud of sub depth image fitting after
The mean square error of plane is less than or equal to first threshold.
In this design, by being split to depth image, N number of sub depth image is obtained, then to N number of sub depth map
Corresponding cloud of the sub depth image for meeting first condition as in carries out clustering processing, and meets the sub depth map of first condition
As corresponding cloud fitting after the mean square error of plane be less than or equal to first threshold, the son for illustrating to meet the first condition is deep
Point in degree corresponding cloud of image is in one plane, that is to say, that may be to corresponding cloud in N number of sub depth image
The sub depth image of plane, which screens, to be clustered, for cloud be not a plane sub depth image without cluster,
So not needing to cluster all sub depth images in N number of sub depth image, to not only can detecte sub depth
More than one plane in image, and the time of plane monitoringnetwork process can be saved.
In a kind of possible design, image data to be processed is depth image.It calculates equipment and obtains depth image, to depth
Degree image is split, and obtains N number of sub depth image.Then the sub depth map of at least one of N number of sub depth image is determined
As corresponding point cloud information, using every corresponding cloud of sub depth image in N number of sub depth image as a node, structure figures
Structure, each node in the graph structure are stored with the corresponding point cloud information of the node.It calculates every in equipment traversal graph structure
A node determines two nodes for meeting second condition in graph structure, then between two nodes for meeting second condition
Construct side；Wherein, second condition includes that the depth value of corresponding cloud of any node in two nodes is continuous and two nodes pair
The angle that should be put between the normal vector of cloud is less than angle threshold value.Later, equipment is calculated to determine to tie in N number of sub depth image with figure
With the corresponding sub depth image of node at least one side in structure, subgraph image set to be processed is formed, to subgraph collection to be processed
In include corresponding cloud of sub depth image clustered, obtain K coarse extraction plane.
In this design, by being split to depth image, N number of sub depth image is obtained, then according to N number of sub depth
The corresponding point cloud information of image constructs graph structure, constructs side between two nodes for meeting second condition in graph structure later,
That is, constructing a side between two nodes that may be fitted to plane, carried out later for the node for having side subsequent
Clustering processing, so there is no need to carry out subsequent processing for sub depth image lower a possibility that being fitted to plane, from
And the time of entire detection planarization process can be saved.
In a kind of possible design, the image data to be processed include by binocular camera shooting the first RGB image and
Second RGB image.After calculating equipment gets the first RGB image and the second RGB image, first RGB image is carried out
Image presegmentation obtains N number of first face block, and carries out image presegmentation to second RGB image, obtains N number of second face block,
Wherein the first face block and the second face block have position corresponding relationship.For each first face block in the block of N number of first face, equipment is calculated
Execute: determined from the block of N number of second face with the first face block have position corresponding relationship the second face block, according to the first face block,
And there is the second face block of position corresponding relationship with the first face block, it determines disparity map, sub depth is determined according to disparity map
Image.It may thereby determine that from N number of sub depth image.Then, calculating equipment can be according to the N number of sub depth image group determined
At subgraph image set to be processed, the corresponding point cloud information of the sub depth image of at least one of N number of sub depth image is determined, it
Afterwards, equipment is calculated according to the corresponding point cloud information of at least one subimage data in N number of subimage data, to subgraph to be processed
The N number of corresponding cloud of sub depth image for including in image set is clustered, and K coarse extraction plane is obtained.
In this design, it is split respectively by the first RGB image and the second RGB image that are shot to binocular camera,
It obtains to determine N number of disparity map for the N number of first face block and N number of second face block of plane, then can determine N number of possibility
For the sub depth image of plane.Then it can may be clustered to N number of for corresponding cloud of sub depth image of plane, without
It needs to carry out clustering processing to the whole image in the first RGB image and the second RGB image, so as to reduce treating capacity, such as
This, can not only detect the more than one plane in image data, and can accelerate to detect the speed of plane.
In a kind of possible design, a kind of sub depth that may be implemented to include to subgraph to be processed concentration presented below
Corresponding cloud of image is clustered, and the mode of K coarse extraction plane is obtained, in this approach, calculate equipment can according to
Processing subgraph concentrates the corresponding point cloud information of the sub depth image of every for including, and establishes minimum heap data structure, wherein minimum
The son that heap data structure is used to concentrate subgraph to be processed according to the mean square error of every sub corresponding cloud of depth image is deep
Degree image is ranked up, and the mean square error for being located at the corresponding cloud of sub depth image on heap top is minimum.For minimum heap data
Structure executes predetermined registration operation, until plane after the fitting of the corresponding cloud of any two node in minimum heap data structure
Mean square error is greater than first threshold, obtains K coarse extraction plane；Wherein, predetermined registration operation includes: the heap from minimum heap data structure
Sub depth image is taken out in top, the son for meeting third condition is deep if determining from the sub depth image adjacent with sub depth image
Image is spent, then is merged sub depth image with the sub depth image for meeting third condition, sub depth image after being merged；
Third condition include mean square error after pointcloud fitting plane corresponding with the sub depth image be less than first threshold and it is described
Square error is minimum；Depth image after fusion is added into minimum heap data structure.
In this design, it may be implemented by establishing minimum heap data structure to N number of corresponding cloud of sub depth image
Mean square error is ranked up, and the mean square error of the corresponding cloud of sub depth image at the heap bottom of most rickle to heap top is smaller and smaller,
Mean square error positioned at the corresponding cloud of sub depth image on heap top is minimum, so as to take out from the heap top of most rickle every time
The smallest corresponding cloud of sub depth image of mean square error, preferentially son the smallest to the mean square error in minimum heap data structure is deep
Degree corresponding cloud of image is clustered, i.e., preferentially carries out clustering processing to the most possible node for forming a plane, in this way
Can as fast as possible finding may be the node of a plane, and merged.For each from minimum heap data structure
Corresponding cloud of the sub depth image taken out, if another corresponding cloud of sub depth image that can be merged can be found,
So fused cloud is continued to be pressed into most rickle and be ranked up, until each sub depth image pair remaining in most rickle
It cannot continue to merge between the point cloud answered, end of clustering.So as to accelerate to determine the speed of coarse extraction plane.
In a kind of possible design, which is the point cloud for including in threedimensional space.To described wait locate
Reason image data is split, and is obtained N number of subimage data, be may include: using the threedimensional space as octree structure
The node of first level；Using threedimensional space as the node of the first level of octree structure, octree structure is constructed；For eight
Each child node that the first level and the ith level in fork tree construction include, calculating equipment can perform the following operations: if should
Child node meets fourth condition, then carries out eight equal portions segmentations to the child node, obtain eight child nodes of i+1 level；Until
All child nodes for including in last level meet fifth condition, building obtain include the child node of M level Octree
Structure；Wherein fourth condition includes that the mean square error of corresponding cloud of child node is greater than first threshold；I is the integer greater than 1, the
I level includes 8i child node, and fifth condition includes the mean square error of corresponding cloud of child node no more than first threshold, or
Person, corresponding cloud of child node include that point quantity is less than amount threshold.Then, it calculates equipment and determines N number of undivided child node
At least one of the undivided corresponding point cloud information of child node.Later, equipment is calculated according in N number of undivided child node
The corresponding point cloud information of at least one undivided child node, it is corresponding to undivided child node N number of in octree structure
Point cloud carries out clustering processing, obtains K coarse extraction plane.
In this design, using the point cloud in threedimensional space as the firstlevel nodes of octree structure, and to the first layer
Node is split as eight equal portions, later, for each child node in each level, the mean square error of corresponding cloud of child node
Greater than first threshold, illustrate point that corresponding cloud of the child node includes not in one plane, so, to the mean square error of cloud
The child node that difference is greater than first threshold is split, until the point that corresponding cloud of each child node in octree structure includes
In one plane, alternatively, corresponding cloud of child node point for including is few enough, that is, the quantity put is less than amount threshold, no longer
Continue to divide, include two classes in the N number of child node for not continuing segmentation in octree structure finally obtained in this way: one kind is point
The mean square error of cloud is less than or equal to the child node of first threshold, and the another kind of point quantity for including for point cloud is less than amount threshold
Child node.Later, clustering processing, available more than one plane are carried out to the N number of child node for not continuing segmentation.
In a kind of possible design, according to the undivided child node pair of at least one of N number of undivided child node
The point cloud information answered carries out clustering processing to corresponding cloud of undivided child node N number of in octree structure, obtains K slightly
Plane is extracted, may include: to calculate equipment to be corresponded to according to the undivided child node of at least one of N number of undivided child node
Point cloud information, determine the normal vector of each corresponding cloud of undivided child node in N number of undivided child node, and by method
Vector passes through Hough transformation at the point in parameter space, then, it is determined that the method for N number of corresponding cloud of undivided child node out
The K point set that vector is formed in parameter space, each point set have an aggregation center；For each point set, determination is grown
Point in the pericentral preset range of aggregation of the point set will fall the corresponding undivided son section of point within a preset range
Corresponding cloud of point permeates coarse extraction plane.It should be noted that undivided child node herein refers to leaf
Node, the i.e. undivided child node do not have child node.
In this design, the normal vector of N number of corresponding cloud of undivided child node is turned by Hough (Hough) transformation
As soon as the point in parameter space is changed into, then the point that the normal vector for belonging to conplane cloud is formed in parameter space
It can flock together, these points will tend to a central point, so determining the point set formed in parameter space, join
The corresponding coarse extraction plane of each point set is formed in number space, can be quickly obtained N number of undivided child node tool in this way
There is the child node of coplanar relation, the child node with coplanar relation is then merged into a coarse extraction plane, may be implemented quickly
Obtain more than one coarse extraction plane.
In a kind of possible design, one kind presented below may be implemented to optimize processing to K coarse extraction plane, obtain
The mode of plane after to L optimization: it determines the normal vector of each coarse extraction plane in K coarse extraction plane, traverses K coarse extraction
Any coarse extraction plane in plane, meets the coarse extraction plane of Article 6 part if it exists, then by coarse extraction plane and meets the
The coarse extraction plane of six conditions permeates plane, obtain L optimize after plane.Wherein, Article 6 part include: normal vector with
The normal vector of coarse extraction plane is parallel and is less than variance threshold values with the variance after coarse extraction plane fitting plane.
By the design, the multiple coarse extraction planes in K coarse extraction plane being actually a plane can be carried out
Fusion, obtained from more accurately L optimize after plane.
Second aspect, the embodiment of the present application provide a kind of plane monitoringnetwork method, and this method is applied to calculate equipment, the party
Method includes: to obtain image data to be processed；Semantic segmentation is carried out to image data to be processed, is obtained N number of with markup information
Subimage data, N are the integer greater than 1；The markup information is used to mark the target object in subimage data；According to every height
The markup information of image data determines the Q subgraph numbers with plane from N number of subimage data with markup information
According to；Q is the integer greater than 0 and less than or equal to N；Determine that each there is plane in the Q subimage datas with plane
The corresponding point cloud information of subimage data；According to the subgraph number each in the Q subimage datas with plane with plane
According to corresponding point cloud information, K coarse extraction plane is determined from the Q subimage datas with plane；K be more than or equal to
The integer of Q；Processing is optimized to K coarse extraction plane, obtains plane after L optimization；L is the positive integer no more than K.
Based on the program, semantic segmentation, available N number of subgraph with markup information are carried out to data image to be processed
As data, the Q subimage datas with plane are then determined from N number of subimage data with markup information, thus
It can only need to carry out plane monitoringnetwork to the Q subimage datas with plane, not need to the subgraph number for not having plane
According to plane monitoringnetwork is carried out, so as to reduce treating capacity, and the Q subimage datas with plane are handled, it can be with
Detect more than one plane.
In a kind of possible design, the image data to be processed include by binocular camera shooting the first RGB image and
Second RGB image.Semantic segmentation is carried out to image data to be processed, obtains N number of subimage data with markup information, it can be with
Include: that semantic segmentation is carried out to the first RGB image, obtains N number of the first subgraph with markup information, and scheme to the 2nd RGB
As carrying out semantic segmentation, N number of the second subgraph with markup information is obtained, wherein each the first son with markup information
Image and the second subgraph with the first subgraph with position corresponding relationship with markup information, which form one, has mark
Infuse the subimage data of information.It determines in the Q subimage datas with plane each with the subimage data pair of plane
The point cloud information answered may include: for the subgraph number of the Q subimage datas with plane with plane each of
According to calculating equipment can be performed following operation: according to first with markup information for including in the subimage data with plane
Subgraph and the second subgraph with markup information with the first subgraph with position corresponding relationship, determine parallax
Figure, determines sub depth image according to disparity map, according to sub depth image, determines to have the subimage data of plane corresponding
Point cloud information.According to the subimage data corresponding point cloud information each in Q subimage datas with plane with plane,
K coarse extraction plane is determined from the Q subimage datas with plane, may include: according to the subgraph with plane
The corresponding point cloud information of data determines K coarse extraction plane from Q sub depth images.
In this design, in the first RGB image and the 2nd RGB figure that image data to be processed includes by binocular camera shooting
When picture, semantic segmentation, then N number of first subgraph obtained from segmentation can be carried out to the first RGB image and the second RGB image respectively
Respectively obtained in picture and N number of second subgraph first subgraph of the Q with plane and with first subgraph with plane
Q with position corresponding relationship the second subgraphs with plane, further according to Q the first subgraphs and Q with plane
The second subgraph with plane obtains Q disparity map, then obtains Q sub depth images according to Q disparity map, so as to
Only need to carry out plane monitoringnetworks with the sub depth images of plane to Q, do not need to the subimage data for not having plane into
Row plane monitoringnetwork, so as to reduce treating capacity.
In a kind of possible design, according to the corresponding point cloud information of subimage data with plane, from Q sub depth
K coarse extraction plane is determined in image, may include: to make corresponding cloud of every sub depth image in Q sub depth images
For a node, graph structure is constructed, wherein each node in graph structure is stored with the corresponding point cloud information of node.Time then,
Each node in graph structure is gone through, two nodes for meeting second condition in graph structure is determined, is then meeting second condition
Two nodes between construct side, wherein second condition include corresponding cloud of any node in two nodes depth value connect
Angle between the continuous and normal vector of two node corresponding points clouds is less than angle threshold value；Determine in Q sub depth images with figure
With the corresponding sub depth image of node at least one side in structure, subgraph image set to be processed is formed；To subgraph to be processed
Corresponding cloud of sub depth image that concentration includes carries out clustering processing, obtains K coarse extraction plane.
In this design, graph structure is constructed according to the corresponding point cloud information of the sub depth image of Q, later to full in graph structure
Side is constructed between two nodes of sufficient second condition, that is to say, that constructs one between two nodes that may be fitted to plane
A side carries out subsequent clustering processing for the node for having side later, that is to say, that the sub depth map that may be fitted to plane
As carrying out clustering processing, so there is no need to carry out subsequent place for sub depth image lower a possibility that being fitted to plane
Reason, so as to save the time of entire detection planarization process.
In a kind of possible design, the corresponding cloud of sub depth image for including is concentrated to gather subgraph to be processed
Class obtains K coarse extraction plane, may include: to be concentrated every sub depth image for including corresponding according to subgraph to be processed
Point cloud information establishes minimum heap data structure, wherein minimum heap data structure is used for according to the corresponding point of every sub depth image
The mean square error of cloud is ranked up the sub depth image that subgraph to be processed is concentrated, and the sub depth image for being located at heap top is corresponding
Point cloud mean square error it is minimum.Then, for minimum heap data structure, predetermined registration operation is executed, until minimum heap data structure
In corresponding cloud of any two node fitting after plane mean square error be greater than first threshold, obtain K coarse extraction put down
Face；Wherein, predetermined registration operation include: from taking out sub depth image in heap top in minimum heap data structure, if from sub depth image
The sub depth image for meeting third condition is determined in adjacent sub depth image, then sub depth image and will meet third condition
Sub depth image merged, sub depth image after being merged；Third condition includes point corresponding with the sub depth image
Mean square error after cloud fit Plane is less than first threshold and the mean square error is minimum；Sub depth image is added after merging
Into minimum heap data structure.
In this design, it may be implemented to concentrate the Q for including to subgraph to be processed by establishing minimum heap data structure
The mean square error of sub corresponding cloud of depth image is ranked up, the corresponding point of sub depth image on heap bottom to the heap top of most rickle
The mean square error of cloud is smaller and smaller, and the mean square error positioned at the corresponding cloud of sub depth image on heap top is minimum, so as to every
The smallest corresponding cloud of sub depth image of mean square error is taken out in the secondary heap top from most rickle, preferentially to minimum heap data structure
In the smallest corresponding cloud of sub depth image of mean square error clustered, i.e., preferentially form plane to most possible
Node carries out clustering processing, in this way can as fast as possible finding may be the node of a plane, and merged.For every
A corresponding cloud of the sub depth image taken out from minimum heap data structure, if another son that can be merged can be found
Corresponding cloud of depth image is ranked up then fused cloud is continued to be pressed into most rickle, until remaining in most rickle
It cannot continue to merge between the corresponding cloud of sub depth image of remaining each, end of clustering.So as to accelerate to determine slightly to mention
It makes even the speed in face.
In a kind of possible design, processing is optimized to K coarse extraction plane, obtains plane after L optimization, it can be with
Comprise determining that the normal vector of each coarse extraction plane in K coarse extraction plane, traverse in K coarse extraction plane any slightly mentions
It makes even face, meets the coarse extraction plane of Article 6 part if it exists, then put down coarse extraction plane with the coarse extraction for meeting Article 6 part
Face permeates plane, obtain L optimize after plane.Wherein, Article 6 part includes: the normal direction of normal vector Yu coarse extraction plane
Amount is parallel and is less than variance threshold values with the variance after coarse extraction plane fitting plane.
By the design, the multiple coarse extraction planes in K coarse extraction plane being actually a plane can be carried out
Fusion, obtained from more accurately L optimize after plane.
The third aspect, the embodiment of the present application provide a kind of calculating equipment, including at least one processor.This at least one
Processor is configured as performing the following operations: obtaining image data to be processed；Image data to be processed is split, N is obtained
A subimage data, N are the integer greater than 1；Determine corresponding cloud of at least one subimage data in N number of subimage data
Information；According to the corresponding point cloud information of at least one subimage data in N number of subimage data, to N number of subimage data pair
The point cloud answered carries out clustering processing, obtains K coarse extraction plane；K is the positive integer no more than N；K coarse extraction plane is carried out
Optimization processing obtains plane after L optimization；L is the positive integer no more than K.
In a kind of possible design, which is depth image；It include each pixel in depth image
The image coordinate and depth value of point；Image data to be processed is split, obtains N number of subimage data, comprising: to depth map
As being split, N number of sub depth image is obtained；Determine the corresponding point of at least one subimage data in N number of subimage data
Cloud information, comprising: determine the corresponding point cloud information of the sub depth image of at least one of N number of sub depth image；According to N number of son
The corresponding point cloud information of at least one subimage data in image data gathers corresponding cloud of N number of subimage data
Class processing, obtains K coarse extraction plane, comprising: according to corresponding cloud letter of each subimage data in N number of sub depth image
Breath determines the mean square error of plane after the fitting of every sub corresponding cloud of depth image；It is determined from N number of sub depth image
The sub depth image for meeting first condition forms subgraph image set to be processed；First condition is corresponding cloud of sub depth image
The mean square error of plane is less than or equal to first threshold after fitting；Concentrate the sub depth image for including corresponding subgraph to be processed
Point cloud clustered, obtain K coarse extraction plane.
In a kind of possible design, image data to be processed is depth image.Image data to be processed is split,
N number of subimage data is obtained, may include: to be split to depth image, obtains N number of sub depth map.Determine N number of subgraph number
The corresponding point cloud information of at least one subimage data in may include: at least one determined in N number of sub depth image
The corresponding point cloud information of a sub depth image.According to corresponding cloud of at least one subimage data in N number of subimage data
Information carries out clustering processing to corresponding cloud of N number of subimage data, obtains K coarse extraction plane, may include: by N number of son
Every corresponding cloud of sub depth image constructs graph structure as a node in depth image；Each node in graph structure
It is stored with the corresponding point cloud information of node；Each node in graph structure is traversed, determines to meet second condition in graph structure
Two nodes, then construct side between two nodes for meeting second condition；Second condition includes any section in two nodes
Angle of the depth value of corresponding cloud of point continuously and between the normal vector of two node corresponding points clouds is less than angle threshold value；Really
Sub depth image corresponding with having the node at least one side in graph structure in N number of sub depth image is made, is formed to be processed
Subgraph image set；It concentrates the corresponding cloud of sub depth image for including to cluster subgraph to be processed, it is flat to obtain K coarse extraction
Face.
In a kind of possible design, image data to be processed includes by the first RGB image and the of binocular camera shooting
Two RGB images；Image data to be processed is split, N number of subimage data is obtained, may include: to the first RGB image into
Row image presegmentation obtains N number of first face block；And image presegmentation is carried out to the second RGB image, obtain N number of second face block；The
Block and the second face block have position corresponding relationship on one side；For each first face block in the block of N number of first face, calculating equipment can be held
The following operation of row: the second face block that there is position corresponding relationship with the first face block is determined from the block of N number of second face；According to first
Face block and the second face block with the first face block with position corresponding relationship, determine disparity map；Son is determined according to disparity map
Depth image；Subgraph image set to be processed is formed according to the N number of sub depth image determined.It determines in N number of subimage data extremely
Few corresponding point cloud information of a subimage data, may include: to determine the sub depth of at least one of N number of sub depth image
The corresponding point cloud information of image；According to the corresponding point cloud information of at least one subimage data in N number of subimage data, to N
Corresponding cloud of a subimage data carries out clustering processing, obtains K coarse extraction plane, may include: according to N number of subgraph number
The corresponding point cloud information of at least one subimage data in concentrates the N number of sub depth image for including to subgraph to be processed
Corresponding cloud is clustered, and K coarse extraction plane is obtained.
In a kind of possible design, the corresponding cloud of sub depth image for including is concentrated to gather subgraph to be processed
Class obtains K coarse extraction plane, may include: to be concentrated every sub depth image for including corresponding according to subgraph to be processed
Point cloud information establishes minimum heap data structure；Minimum heap data structure is used for according to corresponding cloud of every sub depth image
Mean square error is ranked up the sub depth image that subgraph to be processed is concentrated, and is located at the corresponding point of sub depth image on heap top
The mean square error of cloud is minimum.For minimum heap data structure, predetermined registration operation is executed, until any two in minimum heap data structure
The mean square error of plane is greater than first threshold after the fitting of a corresponding cloud of node, obtains K coarse extraction plane；Wherein, in advance
If operation includes: from taking out sub depth image in heap top in minimum heap data structure, if deep from the son adjacent with sub depth image
The sub depth image for meeting third condition is determined in degree image, then by sub depth image and the sub depth map for meeting third condition
As being merged, sub depth image after being merged；Third condition includes pointcloud fitting plane corresponding with the sub depth image
Mean square error afterwards is less than first threshold and the mean square error is minimum；Depth image after fusion is added to most rickle number
According in structure.
In a kind of possible design, image data to be processed is the point cloud for including in threedimensional space.To image to be processed
Data are split, and obtain N number of subimage data, may include: using threedimensional space as the first level of octree structure
Node；For the first level and the ith level each child node for including in octree structure, calculate equipment can execute with
Lower operation: if child node meets fourth condition, carrying out eight equal portions segmentations to child node, obtains eight son sections of i+1 level
Point；Wherein fourth condition includes that the mean square error of corresponding cloud of child node is greater than first threshold；I is integer greater than 1, ith
Level includes 8i child node；Until all child nodes for including in last level meet fifth condition, building obtains including M
The octree structure of the child node of a level；Wherein, fifth condition includes that the mean square error of corresponding cloud of child node is not more than
First threshold, alternatively, corresponding cloud of child node includes that point quantity is less than amount threshold；Determine in octree structure it is N number of not
The child node of segmentation.Determine the corresponding point cloud information of at least one subimage data in N number of subimage data；Determine N number of son
The corresponding point cloud information of at least one subimage data in image data may include: in determining N number of undivided child node
The corresponding point cloud information of at least one undivided child node.According at least one subgraph number in N number of subimage data
According to corresponding point cloud information, clustering processing is carried out to corresponding cloud of N number of subimage data, obtains K coarse extraction plane, it can be with
It include: the child node corresponding point cloud information undivided according at least one of N number of undivided child node, to Octree knot
Corresponding cloud of N number of undivided child node carries out clustering processing in structure, obtains K coarse extraction plane.
In a kind of possible design, according to the undivided child node pair of at least one of N number of undivided child node
The point cloud information answered carries out clustering processing to corresponding cloud of undivided child node N number of in octree structure, obtains K slightly
Plane is extracted, may include: child node corresponding cloud letter undivided according at least one of N number of undivided child node
Breath, determines the normal vector of each corresponding cloud of undivided child node in N number of undivided child node, and normal vector is passed through
Hough transformation is at the point in parameter space；Determine the normal vector of N number of corresponding cloud of undivided child node in parameter sky
Between middle formation K point set, each point set have an aggregation center；For each point set, the aggregation for falling in point set is determined
Point in pericentral preset range；The corresponding point Yun Rong of the corresponding undivided child node of point within a preset range will be fallen
It is combined into a coarse extraction plane.
In a kind of possible design, processing is optimized to K coarse extraction plane, obtains plane after L optimization, it can be with
Comprise determining that the normal vector of each coarse extraction plane in K coarse extraction plane；Any slightly mentioning in K coarse extraction plane of traversal
It makes even face, meets the coarse extraction plane of Article 6 part if it exists, then put down coarse extraction plane with the coarse extraction for meeting Article 6 part
Face permeates plane, obtain L optimize after plane；Wherein, Article 6 part includes: the normal direction of normal vector Yu coarse extraction plane
Amount is parallel and is less than variance threshold values with the variance after coarse extraction plane fitting plane.
Fourth aspect, the embodiment of the present application provide a kind of calculating equipment, including at least one processor.This at least one
Processor is configured as performing the following operations: obtaining image data to be processed；Semantic segmentation is carried out to image data to be processed,
N number of subimage data with markup information is obtained, N is the integer greater than 1；Markup information is for marking in subimage data
Target object；According to the markup information of each subimage data, Q is determined from N number of subimage data with markup information
A subimage data with plane；Q is the integer greater than 0 and less than or equal to N；Determine the Q subgraphs with plane
Each with the corresponding point cloud information of subimage data of plane in data；According to every in the Q subimage datas with plane
The corresponding point cloud information of a subimage data with plane determines that K slightly mention from the Q subimage datas with plane
It makes even face；K is the integer more than or equal to Q；Processing is optimized to K coarse extraction plane, obtains plane after L optimization；L is
Positive integer no more than K.
In a kind of possible design, image data to be processed includes by the first RGB image and the of binocular camera shooting
Two RGB images；Semantic segmentation is carried out to image data to be processed, obtains N number of subimage data with markup information, N is big
In 1 integer, it may include: that semantic segmentation is carried out to the first RGB image, obtain N number of the first subgraph with markup information；
And semantic segmentation is carried out to the second RGB image, obtain N number of the second subgraph with markup information；Wherein, each that there is mark
First subgraph of information and the second subgraph group with markup information with the first subgraph with position corresponding relationship
At a subimage data with markup information；Determine that each there is plane in the Q subimage datas with plane
The corresponding point cloud information of subimage data may include: to have plane for each of Q subimage datas with plane
Subimage data, calculate equipment can execute following operation: according to include in the subimage data with plane have mark
Infuse the first subgraph of information and the second subgraph with markup information with the first subgraph with position corresponding relationship
Picture determines disparity map；Sub depth image is determined according to disparity map；According to sub depth image, the son with plane is determined
The corresponding point cloud information of image data；According to the subimage data each in the Q subimage datas with plane with plane
Corresponding point cloud information determines K coarse extraction plane from the Q subimage datas with plane, may include: according to tool
There is the corresponding point cloud information of the subimage data of plane, determines K coarse extraction plane from Q sub depth images.
In a kind of possible design, according to the corresponding point cloud information of subimage data with plane, from Q sub depth
K coarse extraction plane is determined in image, may include: to make corresponding cloud of every sub depth image in Q sub depth images
For a node, graph structure is constructed；Each node in graph structure is stored with the corresponding point cloud information of node；It traverses in graph structure
Each node, determine two nodes for meeting second condition in graph structure, then in two nodes for meeting second condition
Between construct side, wherein second condition includes that the depth value of corresponding cloud of any node in two nodes is continuous and two sections
Angle between the normal vector of point corresponding points cloud is less than angle threshold value；Determine in Q sub depth images with have in graph structure
The corresponding sub depth image of the node at least one side, forms subgraph image set to be processed；Include to subgraph to be processed concentration
Corresponding cloud of sub depth image carries out clustering processing, obtains K coarse extraction plane.
In a kind of possible design, the corresponding cloud of sub depth image for including is concentrated to gather subgraph to be processed
Class obtains K coarse extraction plane, may include: to be concentrated every sub depth image for including corresponding according to subgraph to be processed
Point cloud information establishes minimum heap data structure；Minimum heap data structure is used for according to corresponding cloud of every sub depth image
Mean square error is ranked up the sub depth image that subgraph to be processed is concentrated, and is located at the corresponding point of sub depth image on heap top
The mean square error of cloud is minimum；For minimum heap data structure, predetermined registration operation is executed, until any two in minimum heap data structure
The mean square error of plane is greater than first threshold after the fitting of a corresponding cloud of node, obtains K coarse extraction plane；Wherein, in advance
If operation includes: from taking out sub depth image in heap top in minimum heap data structure, if deep from the son adjacent with sub depth image
The sub depth image for meeting third condition is determined in degree image, then by sub depth image and the sub depth map for meeting third condition
As being merged, sub depth image after being merged；Third condition includes pointcloud fitting plane corresponding with the sub depth image
Mean square error afterwards is less than first threshold and mean square error is minimum；Depth image after fusion is added to minimum heap data knot
In structure.
In a kind of possible design, processing is optimized to K coarse extraction plane, obtains plane after L optimization, it can be with
Comprise determining that the normal vector of each coarse extraction plane in K coarse extraction plane；Any slightly mentioning in K coarse extraction plane of traversal
It makes even face, meets the coarse extraction plane of Article 6 part if it exists, then put down coarse extraction plane with the coarse extraction for meeting Article 6 part
Face permeates plane；Wherein, Article 6 part includes: that normal vector is parallel with the normal vector of coarse extraction plane and and coarse extraction
Variance after plane fitting plane is less than variance threshold values.
5th aspect, the embodiment of the present application also provide a kind of calculating equipment, which includes executing any of the abovedescribed side
Module/unit of the method for the possible design of any one of face.These module/units can be by hardware realization, can also be with
Corresponding software realization is executed by hardware.
6th aspect, also provides a kind of computer readable storage medium in the embodiment of the present application, described computerreadable to deposit
Storage media includes computer program, when computer program is run on an electronic device, so that electronic equipment execution is abovementioned
The method of the possible design of any one of either side.
7th aspect, the embodiment of the present application also provides a kind of program product, including instruction, when described program product is in electronics
When being run in equipment, so that the method for calculating equipment and executing any one possible design of any of the abovedescribed aspect.
In addition, third aspect technical effect brought by any possible design method into the 7th aspect can be found in first
Technical effect brought by different designs mode in aspect or second aspect, details are not described herein again.
Detailed description of the invention
Fig. 1 is the structural schematic diagram of a kind of electronic equipment provided by the present application；
Fig. 2 is a kind of plane monitoringnetwork method flow schematic diagram provided by the present application；
Fig. 3 a is a kind of schematic diagram of a scenario provided by the present application；
Fig. 3 b is a kind of depth image schematic diagram provided by the present application；
Fig. 3 c~Fig. 3 h is plane monitoringnetwork process schematic provided by the present application；
Fig. 4 a is a kind of subdivision schematic diagram of threedimensional space provided by the present application；
Fig. 4 b is a kind of octree structure schematic diagram provided by the present application；
Fig. 5 is another plane monitoringnetwork process schematic provided by the present application；
Fig. 6 is a kind of image schematic diagram including mark provided by the present application；
Fig. 7 is another image schematic diagram including mark provided by the present application；
Fig. 8 a is a kind of RGB image schematic diagram provided by the present application；
Fig. 8 b is the image schematic diagram of presegmentation provided by the present application；
Fig. 9 is another plane monitoringnetwork method flow schematic diagram provided by the present application；
Figure 10 is a kind of structural schematic diagram for calculating equipment provided by the present application.
Specific embodiment
This application provides a kind of plane monitoringnetwork method, equipment and circuit system are calculated, equipment is calculated and is based on getting
Image data to be processed be split, and plane monitoringnetwork processing is carried out according to the obtained subimage data of segmentation respectively, thus
It can detecte multiple planes in scene, and then can be improved and calculate equipment to the understandability of scene.
In order to keep the purposes, technical schemes and advantages of the application clearer, below in conjunction with attached drawing to the application make into
One step it is described in detail.
The scheme of plane monitoringnetwork provided by the embodiments of the present application can be applied to various calculating equipment, which can be
Electronic equipment is also possible to server.Wherein electronic equipment can include but is not limited to personal computer, server computer,
Handheld or laptop devices, mobile device (such as mobile phone, mobile phone, tablet computer, personal digital assistant, media play
Device etc.), consumer electronic devices, minicomputer, mainframe computer, mobile robot, unmanned plane etc..
When electronic equipment is needed to the plane in image data to be processed is detected, in a kind of possible implementation, electricity
Sub equipment can use plane monitoringnetwork method provided by the embodiments of the present application, realize that detecting multiple planes obtains testing result.Separately
In a kind of possible implementation, image data to be processed can be sent to realizing plane monitoringnetwork process by electronic equipment
Other equipment of processing capacity, such as server or terminal device, then the electronic equipment receives the inspection from other equipment
Survey result.
In following embodiment, for calculating equipment and be electronic equipment, to the plane monitoringnetwork provided in the embodiment of the present application
Method is introduced.
A kind of method for plane monitoringnetwork that present application example provides, suitable for electronic equipment as shown in Figure 1, below first letter
Singly introduce the specific structure of electronic equipment.
Refering to what is shown in Fig. 1, the hardware structural diagram of the electronic equipment for the embodiment of the present application application.As shown in Figure 1, electric
Sub equipment 100 may include display equipment 110, processor 120 and memory 130.Wherein, memory 130 can be used for storing
Software program and data, processor 120 can be stored in the software program and data of memory 130 by running, thus
Execute the various function application and data processing of electronic equipment 100.
Memory 130 can mainly include storing program area and storage data area, wherein storing program area can store operation system
Application program (such as image collecting function etc.) needed for system, at least one function etc.；Storage data area can be stored according to electronics
Equipment 100 uses created data (such as audio data, text information, image data etc.) etc..In addition, memory 130
May include highspeed random access memory, can also include nonvolatile memory, a for example, at least disk memory,
Flush memory device or other volatile solidstate parts.
Processor 120 is the control centre of electronic equipment 100, utilizes various interfaces and the entire electronic equipment of connection
Various pieces execute electronic equipment 100 by running or executing the software program and/or data that are stored in memory 130
Various functions and processing data, to carry out integral monitoring to electronic equipment.Processor 120 may include one or more processing
Unit, such as: processor 120 may include application processor (application processor, AP), modulation /demodulation processing
Device, graphics processor (graphics processing unit, GPU), imagesignal processor (image signal
Processor, ISP), controller, memory, Video Codec, digital signal processor (digital signal
Processor, DSP), baseband processor and/or neural network processor (Neuralnetwork Processing Unit,
NPU) etc..Wherein, different processing units can be independent device, also can integrate in one or more processors.
Wherein, NPU is neural network (neuralnetwork, NN) computation processor, by using for reference biological neural network
Structure, such as transfer mode between human brain neuron is used for reference, it, can also continuous self study to input information fast processing.It is logical
Crossing NPU may be implemented the application such as intelligent cognition of electronic equipment 100, such as: image recognition, recognition of face, speech recognition, text
Understand etc..
In some embodiments, processor 120 may include one or more interfaces.Interface may include integrated circuit
(interintegrated circuit, I2C) interface, integrated circuit builtin audio (interintegrated circuit
Sound, I2S) interface, pulse code modulation (pulse code modulation, PCM) interface, universal asynchronous receivingtransmitting transmitter
(universal asynchronous receiver/transmitter, UART) interface, mobile industry processor interface
(mobile industry processor interface, MIPI), universal input export (generalpurpose
Input/output, GPIO) interface, Subscriber Identity Module (subscriber identity module, SIM) interface, and/or
Universal serial bus (universal serial bus, USB) interface etc..
I2C interface is a kind of bidirectional synchronization universal serial bus, including serial data line (serial data line,
SDA) He Yigen serial time clock line (derail clock line, SCL).In some embodiments, processor 120 may include
Multiple groups I2C bus.Processor 120 can distinguish coupled with touch sensors, charger, flash of light by different I2C bus interface
Lamp, camera 160 etc..Such as: processor 120 can be made processor 120 and be touched by I2C interface coupled with touch sensors
Sensor is communicated by I2C bus interface, realizes the touch function of electronic equipment 100.
I2S interface can be used for voice communication.In some embodiments, processor 120 may include multiple groups I2S bus.
Processor 120 can be coupled by I2S bus with audiofrequency module, realize the communication between processor 120 and audiofrequency module.One
In a little embodiments, audiofrequency module can transmit audio signal to WiFi module 190 by I2S interface, and realization passes through bluetooth headset
The function of receiving calls.
Pcm interface can be used for voice communication, by analog signal sampling, quantization and coding.In some embodiments, sound
Frequency module can be coupled with WiFi module 190 by pcm bus interface.In some embodiments, audiofrequency module can also pass through
Pcm interface transmits audio signal to WiFi module 190, realizes the function of receiving calls by bluetooth headset.The I2S interface and
The pcm interface may be used to voice communication.
UART interface is a kind of Universal Serial Bus, is used for asynchronous communication.The bus can be bidirectional communications bus.
The data that it will be transmitted are converted between serial communication and parallel communications.In some embodiments, UART interface usually by with
In connection processor 120 and WiFi module 190.Such as: processor 120 passes through the bluetooth in UART interface and WiFi module 190
Module communication, realizes Bluetooth function.In some embodiments, audiofrequency module can be passed by UART interface to WiFi module 190
Audio signal is passed, realizes the function of playing music by bluetooth headset.
MIPI interface can be used to connect the peripheral components such as processor 120 and display equipment 110, camera 160.MIPI
Interface includes 160 serial line interface of camera (camera serial interface, CSI), display screen serial line interface (display
Serial interface, DSI) etc..In some embodiments, processor 120 and camera 160 are communicated by CSI interface, real
The shooting function of existing electronic equipment 100.Processor 120 and display screen realize the aobvious of electronic equipment 100 by DSI interface communication
Show function.
GPIO interface can pass through software configuration.GPIO interface can be configured as control signal, may be alternatively configured as counting
It is believed that number.In some embodiments, GPIO interface can be used for connecting processor 120 and camera 160, show equipment 110,
WiFi module 190, audiofrequency module, sensor module etc..GPIO interface can be additionally configured to I2C interface, I2S interface, UART
Interface, MIPI interface etc..
USB interface is the interface for meeting USB standard specification, specifically can be Mini USB interface, Micro USB interface,
USB Type C interface etc..USB interface can be used for connecting charger for the charging of electronic equipment 100, can be used for electronics and sets
Data are transmitted between standby 100 and peripheral equipment.It can be used for connection earphone, audio played by earphone.The interface can be with
For connecting other electronic equipments, such as AR equipment etc..
It is understood that the interface connection relationship of each intermodule of signal of the embodiment of the present invention, only schematically illustrates,
The structure qualification to electronic equipment 100 is not constituted.In other embodiments of the application, electronic equipment 100 can also be used
The combination of different interface connection type or multiple interfaces connection type in abovedescribed embodiment.
It further include the camera 160 for shooting image or video in electronic equipment 100.Camera 160 can be commonly
Camera is also possible to focusing camera head.
Electronic equipment 100 can also include input equipment 140, digital information for receiving input, character information or connect
Touch touch operation/noncontact gesture, and generate letter related with the user setting of electronic equipment 100 and function control
Number input etc..
Show equipment 110, including display panel 111, for showing information input by user or being supplied to user's
Information and the various menu interfaces of electronic equipment 100 etc. are mainly used for showing in electronic equipment 100 in the embodiment of the present application
Image to be detected that camera or sensor obtain.Optionally, display panel can use liquid crystal display (liquid
Crystal display, LCD) or the forms such as Organic Light Emitting Diode (organic lightemitting diode, OLED)
To configure display panel 111.
Electronic equipment 100 can also include one or more sensors 170, such as imaging sensor, infrared sensor, swash
Optical sensor, pressure sensor, gyro sensor, baroceptor, Magnetic Sensor, acceleration transducer, range sensor,
Close to optical sensor, ambient light sensor, fingerprint sensor, touch sensor, temperature sensor, bone conduction sensor etc.,
Middle imaging sensor can be flight time (time of flight, TOF) sensor, structured light sensor etc..
In addition to this, electronic equipment 100 can also include for the power supply 150 to other module for power supply.Electronic equipment 100
It can also include less radiofrequency (radio frequency, RF) circuit 180, for carrying out network communication with Wireless Communication Equipment,
It can also include WiFi module 190, for carrying out WiFi communication with other equipment, for example, for obtaining other equipment transmission
Image or data etc..
Although and it is not shown in FIG. 1, electronic equipment 100 can also include flash lamp, bluetooth module, external interface, press
Other possible functional modules such as key, motor, details are not described herein.
Based on abovementioned introduction, the application provides the method and calculating equipment of a kind of plane monitoringnetwork, and wherein method can make
It calculates equipment and is able to detect that the more than one plane in image data.In the embodiment of the present application, method and calculating equipment are bases
The reality of apparatus and method for is calculated since the principle that method and calculating equipment solve the problems, such as is similar in same inventive concept
Applying can be with crossreference, and overlaps will not be repeated.
In the embodiment of the present application, it is described for calculating equipment and being electronic equipment 100, but is not intended to limit this hair
Bright embodiment is applied in other kinds of calculating equipment.As shown in fig.2, the detailed process of the plane monitoringnetwork method can wrap
It includes:
Step 201: electronic equipment 100 obtains image data to be processed.
Herein, image data to be processed can be two dimensional image, than as the following examples one in depth image, embodiment
The first RGB image and the second RGB image in three, can also be with three dimensional point cloud, the point cloud in two as the following examples, herein
It is not specifically limited.
It should be understood that image data to be processed can be electronic equipment shooting, it is also possible to from electronic equipment for depositing
It is obtained in the picture library of storage image, is also possible to what other equipment were sent.
Step 202: electronic equipment 100 is split image data to be processed, obtains N number of subimage data, N be greater than
1 integer.
Step 203: electronic equipment 100 determines corresponding cloud of at least one subimage data in N number of subimage data
Information.
Step 204: electronic equipment 100 is according to corresponding cloud of at least one subimage data in N number of subimage data
Information carries out clustering processing to corresponding cloud of N number of subimage data, obtains K coarse extraction plane；K is just whole no more than N
Number.
Step 205: electronic equipment 100 optimizes processing to K coarse extraction plane, obtains plane after L optimization；L is
Positive integer no more than K.
In the embodiment of the present application, electronic equipment is split image data to be processed, and to N number of subgraph that segmentation obtains
As corresponding cloud progress clustering processing of data, the more than one plane in image data to be processed can detecte.
It describes in detail combined with specific embodiments below to abovementioned plane monitoringnetwork method shown in Fig. 2.
Embodiment one
In the embodiment of the present application, the image data to be processed in abovementioned steps 201 is depth image, and depth image can be with
Regard a secondary gray level image as, wherein the gray value of each pixel in gray level image can characterize certain point in scene and exist
Projection on the optical axis of camera.It is a kind of schematic diagram of the RGB image of scene provided by the embodiments of the present application referring to Fig. 3 a.
In the extraction for carrying out depth information for the scene in the corresponding threedimensional space of RGB image shown in Fig. 3 a, obtain such as Fig. 3 b institute
The depth image shown.
In the embodiment of the present application, depth information can be obtained using modes such as ToF camera, structure light, laser scannings, from
And obtain depth image.It should be understood that in the embodiment of the present application can also using it is other it is any can obtain depth image by the way of
(or camera) realizes acquisition depth image.Hereinafter only it is illustrated for obtaining depth image using ToF camera, but this
The limitation to depth image mode is obtained is not caused, it is subsequent not repeat.
It should be noted that, although point cloud is threedimensional concept, the pixel in depth image is twodimensional concept, but known
In the case where the depth value that some in two dimensional image is put, the image coordinate of the point can be converted into the seat of the world in threedimensional space
Mark, so, the point cloud in threedimensional space can be recovered according to depth image.For example, can be completed using multiple view geometry algorithm
Image coordinate is converted into world coordinates, specific conversion regime and process are not construed as limiting.
Electronic equipment can carry out plane monitoringnetwork processing to depth image, specifically after obtaining depth image using ToF
It may include following several possible processes.
Process one, is split depth image, and multiple nonoverlapping regions is divided into (each region to be known as son below
Image, alternatively referred to as sub depth image).
If regarding plane monitoringnetwork method provided in an embodiment of the present invention as a plane detection algorithm process, process
One is considered as the initialization procedure of this plane detection algorithm, and what is inputted in this plane detection algorithm is a frame depth map
Picture has image coordinate (u, v) and depth value (gray scale) on depth image.This process is split for depth image, is obtained
Then subgraph after multiple segmentations can carry out subsequent processing to each subgraph respectively.Due to can be with according to depth information
Correspondence recovers a cloud, then carrying out processing correspondence for each subgraph obtained after segmentation can reflect in three dimensions
Corresponding cloud of each subgraph is handled, that is to say, that can be used inside algorithm cloud world coordinates (x, y,
Z) it is calculated, for example is fitted the calculating of plane, on the one hand can detect multiple planes, another party as far as possible
Face may also speed up the processing speed to depth image.
For the depth image shown in Fig. 3 b, the above process one is described in detail below with reference to specific example.
It is split for depth image shown in Fig. 3 b, the signal of the subgraph after obtaining segmentation as shown in Figure 3c
Scheme, it is merely exemplary in Fig. 3 c to show the not overlapping region (hereinafter referred to subgraph) of 42 regular sizes after segmentation, it can
Choosing, 42 regions after segmentation are also possible to different size of region.It should be understood that can basis during concrete implementation
Actual needs is split, and the quantity of specific subgraph is not specifically limited.
In one possible implementation, multiple nonoverlapping regions are split into depth image, it can be by depth
The region in image for a regular size is spent, each region may include multiple points, and each point can be mapped to threedimensional space
Between in a number of point composition point cloud, the first moment of the corresponding cloud in these regions can be calculated during initialization
And second moment, wherein first moment is the center of gravity of the corresponding cloud in this region, and second moment is the side of the corresponding cloud in this region
Difference.
After obtaining 42 subgraphs as shown in Figure 3c, the subsequent processing for 42 subgraphs is (such as at cluster
Reason) it can be there are many implementation.In a kind of possible implementation, for corresponding cloud of each subgraph, carry out respectively
Such as the clustering processing in following processes two.It is needed in this mode at all subgraphs obtained after Range Image Segmentation
Reason, compared to the prior art in from center begin stepping through image to be processed until detecting that a plane just stops detecting, can only
Detect that the scheme of a plane, scheme provided by the present application are split to obtain subgraph to entire image to be processed, and right
All subgraphs carry out clustering processing, so as to detect multiple planes.
It, can be first from 42 sons in alternatively possible implementation in order to further speed up the speed of plane monitoringnetwork
It determines that corresponding cloud may be fitted to the subgraph of a plane in image, is then fitted to one for the possibility determined
The subgraph of a plane carries out the clustering processing such as following processes two, so there is no need to for a possibility that being fitted to plane compared with
Low subgraph carries out subsequent processing, so as to save the time of entire detection planarization process.
In one example, for subgraph shown in Fig. 3 c, graph structure can be constructed, graph structure is a kind of computer number
According to structure, using the information of corresponding cloud of a subgraph as a node of graph structure, for example, corresponding with subgraph B3
For the information of point cloud is as a node in graph structure, subgraph B3 corresponding node in graph structure includes the subgraph
As the first moment, second moment (also referred to as mean square error) and the calculated normal vector of plane fitting of corresponding cloud of B3.Then
It will likely be fitted to one side of construction between two nodes of plane, carry out subsequent processing for the node for having side.
In the embodiment of the present application, it is equal with the quadratic sum of the error of initial data corresponding points that mean square error is fitting data
Value, that is to say, that for being fitted to obtain a plane A according to all the points for including in cloud, for some point in cloud
For, the mean square error for putting cloud is the mean value of the quadratic sum of distance between all the points and plane A in point cloud.About the square of cloud
Error is not repeating hereinafter.
In one possible implementation, two nodes of construction can be chosen whether according to the relationship between two nodes
Between side.Specifically, corresponding cloud of each node carries out that corresponding normal vector can be obtained after plane fitting, such as
Angle is greater than angle threshold value between the normal vector of the corresponding fit Plane of two nodes of fruit, just not structure between the two nodes
Side is made, if angle is less than or equal to angle threshold value between the normal vector of two nodes, just constructs side between the two nodes.So
Afterwards, the clustering processing that process two is carried out for the node for having side there is not the node on side without cluster.
Referring to Fig. 3 d, schematic diagram is initialized for graph structure provided by the embodiments of the present application.
As shown in Figure 3d, a classification is done for the corresponding all node of graph of depth map, wherein black circles expression does not have
The node of depth value, × it is the discontinuous node of depth value, two node edges are discontinuous, and solid black point indicates subgraph pair
The smaller node of the mean square error (mean square error, MSE) that the point cloud plane fitting answered comes out.It constructs
While be exactly in fact graph structure while, it connects two adjacent nodes, indicates there is relationship between the two nodes, does not connect and is exactly
It is not related.Assuming that connected two nodes are A node and B node, then connection side needs to meet following condition: condition 1,
The depth value of any of A node and B node corresponding cloud of node is continuous, i.e., A node and B node are all not without depth value
Node, nor the discontinuous this node of depth value；Normal vector folder between condition 2, A node and B node the two nodes
Angle is less than angle threshold value.
Process two carries out clustering processing for the subgraph in process one.
Only for node (i.e. darker regions in Fig. 3 d) the progress clustering processing for having side shown in Fig. 3 d in process two
For be illustrated.
It, can be (hereinafter referred to as minimum by establishing minimum heap data structure in order to further speed up the speed of clustering processing
Heap) to realize, preferentially the node progress clustering processing to most possible one plane of formation, most rickle can be regarded as section
The process that the processing sequence of point is ranked up, can prioritize processing the smallest node of second moment by most rickle, most from second moment
Small node is begun stepping through.
In one example, a most rickle is initially set up, all nodes all respectively become one kind in figure at this time.To each
A node carries out plane fitting, and each node fitting result corresponds to a fitting mean square error, and big according to fitting mean square error
Each node is pressed into most rickle by small sequence from big to small, and the heap top (object popped up every time) of most rickle is lowest mean square
The corresponding node of error.It being iterated later, iteration popup each time is all the current node with minimum error of fitting, this
A node is merged with some node of surrounding, the fused mean square error of available two nodes, after selection fusion
Two nodes before the square the smallest fused node replacement of error, are added graph structure, and rejoin most rickle.After fusion
The mean square error of node if it is less than or be equal to first threshold, then by the fusion posterior nodal point be added most rickle in, if the fusion
The mean square error of posterior nodal point is greater than first threshold, then abandons the fusion posterior nodal point.The stop condition of iteration is all sections in figure
The plane fitting mean square error of point between any two is greater than first threshold.
Process two is described in detail below.
S1 carries out plane fitting according to corresponding cloud of each node, for example can use principal component analysis
(principal component analysis, PCA) algorithm carries out plane fitting, cardinal principle to cloud are as follows: assuming that section
Point all in corresponding cloud is put in one plane, then the characteristic value of Calculation Plane, if having one in three characteristic values
A characteristic value is less than threshold value, it may be said that the point for including in bright cloud in one plane, should be corresponding less than the characteristic value of threshold value special
Sign vector is exactly the normal vector of plane.Then a mean square error can be calculated according to all points for being fitted to plane, this is
Square error is the mean value of the quadratic sum of distance between all the points and the plane of fitting in corresponding cloud of the node.One node pair
A mean square error is answered, and node is pressed into according to the sequence from big to small of mean square error by most rickle according to calculated result.
S2 takes out the corresponding node of least meansquare error from most rickle, by the corresponding node of the least meansquare error with
The adjacent node of the node is merged, and the mean square error of fusion posterior nodal point is calculated.
For 9 nodes marked below with reference to dotted line frame shown in Fig. 3 c, node fusion process illustrate
It is bright.
As shown in 3b, 9 nodes that dotted line frame is marked are respectively node B3, node B4, node B5, node C3, node
C4, node C5, node D3, node D4, node D5, it is assumed that this 9 nodes are all the nodes for having side, corresponding to this 9 nodes
Point cloud carries out plane fitting respectively and obtains mean square error, it is assumed that the corresponding mean square error of node C4 is minimum, has 8 around node C4
A node B3, node B4, node B5, node C3, node C5, node D3, node D4, node D5, by node C4 respectively with surrounding
8 nodes in each node merged, available fused 8 new nodes, respectively node C4B3, node
C4B4, node C4B5, node C4C3, node C4C5, node C4D3, node C4D4, node C4D5, it is fused for this 8
Corresponding cloud of the fused new node of each of new node carries out plane fitting and obtains mean square error, it is assumed that fused new
The corresponding mean square error of node interior joint C4B4 is minimum in the corresponding mean square error of abovementioned 8 fused new nodes and should
The corresponding mean square error of node C4B4 is less than or equal to first threshold, then selection node C4B4 replaces node C4 and node B4 to add
Enter graph structure, and node C4B4 is added to most rickle.That is, dotted line frame is marked in Fig. 3 c after fusion process
9 nodes of note, become such as 8 nodes that dotted line frame is marked in Fig. 3 e, respectively node B3, node B5, node C3, node
C4B4, node C5, node D3, node D4, node D5.It then proceedes to 8 nodes in Fig. 3 e according to such as abovementioned for node
The fusion process of C4 and surroundings nodes is clustered, until meeting the stop condition of iteration.The stop condition of iteration is institute
The mean square error that the corresponding cloud of new node for having in node the fusion between node twobytwo obtain is obtained by plane fitting is all
Greater than first threshold.
Herein, there can be at least two possible modes to determine whether aggregators C4 and node B4.
In one of possible mode, egress C4 8 new sections that 8 nodes merge with around respectively are being determined
After the corresponding mean square error of point, first obtain 8 mean square errors are made comparisons with first threshold respectively, if this 8 new sections
There are the nodes that mean square error is less than or equal to first threshold in point, then being less than or equal to the section of first threshold from mean square error
Determine that the corresponding node of least meansquare error is added to most rickle in point.If mean square error is not present in this 8 new nodes
Less than or equal to the new node of first threshold, then illustrating that node C4 cannot be merged with surroundings nodes, that is to say, that pass through this
After fusion process, 9 nodes that as a result still dotted line frame as shown in Figure 3c is marked.
In alternatively possible mode, egress C4 8 new sections that 8 nodes merge with around respectively are being determined
After the corresponding mean square error of point, it is ranked up, is determined in this 8 new nodes against the corresponding mean square error of 8 new nodes
The corresponding new node of least meansquare error makes comparisons the least meansquare error with first threshold, if the least meansquare error
Less than or equal to first threshold, then being added the corresponding new node of the least meansquare error to most rickle.If the minimum is equal
Square error is greater than first threshold, then illustrating that node C4 cannot be merged with surroundings nodes, that is to say, that passes through this fusion process
Later, 9 nodes that as a result still dotted line frame as shown in Figure 3c is marked.
It should be noted that by being illustrated for handling and having the node on side in this present embodiment, so be directed to one
It is a have while node C4 around if it is none of it is other have while node, there is no need to carry out to this node C4 for that
Fusion process is stated, if there are other nodes for having side around node C4, can centainly be found around node C4 fused
New node meets the node that mean square error is less than or equal to first threshold.Certainly, the embodiment of the present application can also be for such as Fig. 3 c
Shown in divide after subgraph directly carry out clustering processing, then being possible to node C4 occur cannot to save with around any one
The case where point is merged.
In abovementioned cluster process, abovementioned clustering processing successively is carried out from be hit by a bullet out the node of least meansquare error of most rickle,
If the node of the least meansquare error has the node that can be merged, just fused node is added into most rickle,
If the node of the least meansquare error is added the node that the node of the least meansquare error can not merge again
Into most rickle, until each node in some moment most rickle can not find the node that can be merged, with regard to stopping clustering
Journey.At this point, corresponding cloud of each node in most rickle is on different surfaces, i.e., any two node can not be fused into one
Plane, therefore, each node in most rickle in remaining each node can correspond to a plane.
It in abovedescribed embodiment, is illustrated in the clustering processing mode of single thread as an example, that is to say, that be directed to depth
Corresponding 42 nodes of image are added to most rickle, pop up a node every time, find from the node around the node
With the node that can merge the node, after obtaining fusion posterior nodal point, fusion posterior nodal point is put into most rickle, is then proceeded to
Foregoing fusion process is carried out from the most rickle the smallest node of mean square error of being hit by a bullet out, until cluster process terminates.In this mode
In, it is begun stepping through from a node, finally obtains multiple planes.
It should be understood that the embodiment of the present application can also detect multiple planes using the clustering processing mode of multithreading, such as with
Using the clustering processing mode of three threads, at this moment once pop up 3 the smallest nodes of mean square error from most rickle, popup it is every
A node corresponds to a thread, and the fusion process of per thread can refer to the fusion process of abovementioned three thread in these three threads.
It is all added into most rickle in the fusion posterior nodal point of three threads, then proceedes to 3 mean square error minimums of being hit by a bullet out from most rickle
Node, foregoing fusion process is carried out with surroundings nodes, until cluster process terminates.This mode may be implemented simultaneously from multiple
Node is begun stepping through, and finally obtains multiple planes, compared to the abovementioned mode from a node traverses, the processing speed of this mode
It spends faster, so as to faster obtain the plane monitoringnetwork result of entire image to be processed.
For have side shown in Fig. 3 d node carry out process two in clustering processing after, it is available as illustrated in figure 3f
Cluster result, i.e., if the figure of grey in Fig. 3 f to join together is coarse extraction plane.
It there may be two coarse extraction planes is actually conplane in multiple coarse extraction planes that process two obtains
Situation, it is also possible to there is the point for being not belonging to the coarse extraction plane in coarse extraction plane, it is also possible to have other around coarse extraction plane
Point belong to the coarse extraction plane, that is to say, that the coarse extraction plane may miss some points for belonging to the coarse extraction plane.
More accurate plane monitoringnetwork is as a result, optimize processing to coarse extraction plane in order to obtain, specifically may refer to following processes three,
Process four and process five.
For the plane further refined, avoid in coarse extraction plane exist be not belonging to the coarse extraction plane
Point can carry out erosion algorithm operation for the coarse extraction plane, and specific implementation is referring to process three.
Process three, the coarse extraction plane obtained to clustering processing in the above process two carry out marginal operation, can also claim
For erosion algorithm operation.
The edge of coarse extraction plane obtained in process two is zigzag fashion, and according to the precision of graph structure, this is thick
The edge for extracting plane will cover different regions, and may intervene the region of another plane.Etching operation, which can reduce, works as
The domination region of each node in preceding figure, and leave the region in each plane bosom.
Based on Fig. 3 f, coarse extraction plane therein is marked, obtain such as Fig. 3 g thus to mark each coarse extraction flat
The image in face, it should be appreciated that coarse extraction plane be marked this movement be not detect planarization process in one it is necessary
Movement is merely for convenience of illustrating the coarse extraction plane obtained by process two.
As Fig. 3 g illustrates the coarse extraction plane A that dotted line frame marked, coarse extraction plane B, coarse extraction plane C, thick
Extract plane D, coarse extraction plane E, coarse extraction plane F.
By taking the coarse extraction plane A that dotted line frame in Fig. 3 g is marked as an example, etching operation is carried out to coarse extraction plane A and is removed slightly
Coarse extraction plane A can be avoided so that obtained plane is more accurate as far as possible by extracting the node that plane A jagged edges one enclose
In include being not belonging to the point of the coarse extraction plane.
Coarse extraction Planar Contraction can be made to arrive for the etching operation of the coarse extraction plane that process two obtains, the above process three
The circle of edge one is removed in center, for example the coarse extraction plane A in comparative diagram 3a and Fig. 3 g, Fig. 3 g is the desktop in Fig. 3 a, In
The depth value tomography at table angle in Fig. 3 g, if the erosion algorithm without process three operates, that is to say, that in the above process
The region growings operation of process four is directly carried out after two, it is easy to cause coarse extraction plane A and following coarse extraction plane C,
Coarse extraction plane D etc. carries out region growing together, but actually coarse extraction plane A and following coarse extraction plane C, coarse extraction
Plane D is not approximately the same plane, if together carry out region growing will obtain error result, so to coarse extraction plane into
It, can be to avoid such case after row etching operation.
For the plane further refined, avoiding each coarse extraction plane that from may missing, some to belong to this thick
The point for extracting plane can carry out algorithm of region growing operation for the coarse extraction plane A, specifically by taking coarse extraction plane A as an example
Implementation is referring to process four.
Process four is further operated using algorithm of region growing for the coarse extraction plane by etching operation, to avoid
Omit the point for belonging to the coarse extraction plane.
It, can be simultaneously using multiple coarse extraction planes as center region, to outgrowth in a kind of possible implementation.
Below with reference to specific example, to how realizing that region growing is described in detail.
To be for growth outward is realized in center region, due to being grasped through excessive erosion by the coarse extraction plane A of etching operation
The coarse extraction plane A of work is removed obtained from marginal operation for coarse extraction plane A, so by the thick of etching operation
Extracting plane A is not coarse extraction plane A, below for the coarse extraction plane A of etching operation is known as coarse extraction plane A'
It is illustrated.
Using coarse extraction plane A' as center region to outgrowth during, need to every around coarse extraction plane A'
A pixel (these pixels are not in coarse extraction plane A') carries out judging whether to be divided to coarse extraction plane A', is
Convenient for explanation, it will judge whether that this pixel that can be divided to coarse extraction plane A' is known as operating point.
In the embodiment of the present application, there are many may determine that whether operating point can be divided to the realization of coarse extraction plane A'
Mode.
In a kind of possible implementation, if there are 50% or more neighbours in all vicinity points around operating point
Nearly pixel belongs to some coarse extraction plane, such as coarse extraction plane A', puts down then the operating point is also divided to the coarse extraction
Face A'.
By taking operating point is pixel K as an example, if there is 8 vicinity points around pixel K, if this 8 neighbouring pictures
There are more than 4 pixels in vegetarian refreshments and belong to coarse extraction plane A', for example there are 5 vicinity points to belong to slightly around pixel K
Plane A' is extracted, then the corresponding point cloud data of pixel K is just substituting to the corresponding plane fitting side coarse extraction plane A'
Cheng Zhong, to verify whether pixel K belongs to coarse extraction plane A', if verification result is to belong to the coarse extraction plane
A', then pixel K is just divided to coarse extraction plane A'；If verification result is to be not belonging to coarse extraction plane A',
It so abandons for pixel K being divided to coarse extraction plane A'.
In alternatively possible implementation, if there are 50% neighbours in all vicinity points around the operating point
Nearly pixel belongs to coarse extraction plane A', and 50% vicinity points belong to coarse extraction plane C', at this time can be by by the behaviour
Make point and carry out plane fitting with the vicinity points for belonging to two coarse extraction planes respectively, pixel K is divided to quasi by selection
Close coarse extraction plane belonging to the corresponding smaller vicinity points of mean square error of result.
By taking operating point is pixel K as an example, if there is 8 vicinity points around pixel K, if 4 pixel categories
In coarse extraction plane A', then pixel K is substituted into coarse extraction plane A''s close to coarse extraction plane C' by 4 pixels respectively
The plane fitting equation of plane fitting equation and coarse extraction plane C' compares the substitution of two plane fitting equations as a result, later
Pixel K is divided in the better coarse extraction plane of result in the substitution result of two coarse extraction planes (A' and C').
By the above process four, it is around each coarse extraction plane that process three can be obtained and with some coarse extraction
The conplane neighbor point that belongs to of plane is divided to the coarse extraction plane, so that the plane that detected is more complete
It is whole.
The above process two, process three, multiple planes that any one process detects in process four, are all based on to two
Obtained from the understanding for tieing up image, there may be visual error, for example there may be two planes are actually same
A plane, but what is reflected on 2d may not be a plane.In order to avoid there is such case, can pass through
Following processes five, which are realized, eliminates multiple planes collimation error that may be present that the several processes in front detect.
Process five merges plane operations to multiple planes, is referred to as merging Plat algorithm.
Process five can be adapted for carrying out for the result that any process obtains in the above process two, process three and process four
Merge plane operations, is illustrated so that the result obtained for process four merges plane operations as an example below.
It can be by process four using primary bottomup hierarchical clustering algorithm for multiple planes that process four obtains
Obtained in all planes once traversed, merge any two spatially plane similar in space length.
In one possible implementation, plane twobytwo in the multiple planes that can be obtained using PCA algorithmic procedure four
Plane fitting is carried out, for example multiple planes that process four obtains are plane A', B', C', D', E', F', twobytwo after plane fitting
The plane DE's that plane BC and plane D' that available plane B', plane C' are fitted, plane E' are fitted is square
Error is less than threshold value, then plane B' and plane C' are in one plane, plane D' and plane E' in one plane, then may be used
Continuing plane BC, plane DE together with remaining plane A', F', continue to be fitted between plane twobytwo, it is known that any
The mean square error for the plane being fitted between two planes is both greater than threshold value, and fitting terminates, and all planes remaining at this time are not one
In a plane.
In one implementation, can first judge whether any two plane is parallel, then judge that parallel two are flat
Whether the space length of face spatially is close.
In some embodiments, normal vector that two planes can be calculated, by determining whether normal vector determines in parallel
Whether two planes are parallel, if the normal vector of two planes is parallel, two planes are also parallel, illustratively, A pairs of plane
The normal vector answered is n1=(A1, B1, C1), and the corresponding normal vector of plane A is n2=(A2, B2, C2), if A1/A2=B1/
B2 and B1/B2=C1/C2 and A1/A2, B1/B2, C1/C2 are same constant, then plane A is parallel with plane B, herein,
"/" is indicated divided by A1/A2 indicates A1 divided by A2.
In an implementation, if it is determined that it is not parallel to go out two planes, then the two planes will not be judged as same put down
Face, the two planes will not be judged as the similar plane on space length in other words；If it is determined that two planes are parallel,
So the direction of the common vertical line of the two planes can be determined by the way that there are vertical ranges between the two determining planes
(the namely direction of normal vector) illustrates variance with regard to big if the distance on common vertical line direction of 2 planes is very big；If
Two planes on common vertical line direction apart from very little, illustrate variance with regard to small, it is possible to by PCA algorithm to two planes into
Row fitting, calculates mean square error, then decentralization obtains variance, then determines two planes by determining variance size
Whether distance is close, for example variance is less than or equal to variance threshold values, illustrates the closely located of two planes, the two are flat in other words
Face is in one plane.
In one example, the merging plane operations in such as process five are carried out based on each plane in Fig. 3 g, can obtained
Plane after to merging as illustrated in figure 3h.Wherein by Fig. 3 g coarse extraction plane D and coarse extraction plane E merge into it is same flat
Face, coarse extraction plane B and coarse extraction plane C merge into approximately the same plane, and wherein coarse extraction plane A is a plane, coarse extraction
Plane F is a plane.
By abovementioned five processes, may be implemented quickly to being traversed in entire depth image, to quickly detect
To the plane of multiple precisions.
It should be noted that abovementioned five processes can be used alone, also can choose one or more processes and combine makes
With.For example, process three is omitted only with Four processes such as process one, process two, process four, processes five.It can certainly be by
Other methods for combining to realize abovementioned plane monitoringnetwork.Illustratively, the above process three can also process four or process five it
After carry out.
In the examples below, plane monitoringnetwork method may include multiple processes, and between any two processes there may be
The case where nesting executes illustratively can nested implementation procedure A in process one.
Embodiment two
In the embodiment of the present application, the image data to be processed in abovementioned steps 201 can be three dimensional point cloud.This Shen
Please embodiment can be adapted for orderly putting cloud, be readily applicable to unordered cloud.Orderly all the points in point cloud are arranged in order,
It can be easily found the consecutive points information of each point, the point arrangement in unordered cloud is unordered, general nothing without rule
Method finds the consecutive points information of each point.
For the scene shown in Fig. 3 a, by establishing Octree knot to point cloud corresponding in scene as shown in Figure 3a
Structure carries out plane fitting to the point cloud in 8 nodes in one layer in conjunction with plane fitting mode, then according to each node respectively
In all the points whether in approximately the same plane determine whether to continue to divide the point cloud in the node layer, final detection obtains multiple
Refine plane.
In the extraction for carrying out depth information for scene shown in Fig. 3 a, can be swept using ToF camera, structure light, laser
The modes such as retouch to obtain depth information, to obtain depth image, the related content of depth image be may refer in embodiment one
Associated description, details are not described herein again.
Process one, the octree index of building point cloud, is split threedimensional point cloud with realizing.
In one possible implementation, the point cloud in a scene is subjected to decentralization, and according to different points
Resolution is divided by slightly to the set of essence.The plane for the point cloud that resolution ratio herein includes by the corresponding voxel of octree nodes is intended
It closes result to determine, wherein each octree nodes characterizes the set at a voxel midpoint.During constructing Octree,
The first order and second order moments that this node corresponds to voxel midpoint cloud are calculated while constructing each node.
Below with reference to Fig. 4 a and Fig. 4 b, describe in detail to how to construct octree structure.
It is illustrated first against octree structure, octree structure is by a kind of data model, and octree structure passes through
Volume elements subdivision is carried out to the geometry entity of threedimensional space, each volume elements Time & Space Complexity having the same passes through circulation
Recursive division methods carry out subdivision to the geometric object for the threedimensional space that size is (2n*2n*2n) (2n*2n*2n), thus structure
At a directional diagram with root node.If divided volume elements attribute having the same in octree structure, such as phase
Same attribute can be for the point in volume elements in the same plane, then the volume elements constitutes a leaf node, otherwise continues to the body
Member is split into 8 subcubes, successively passs subdivision, for the spatial object of (2n*2n*2n) (2n*2n*2n) size, at most cuts open
Divide nn times.
A referring to fig. 4 is the subdivision schematic diagram provided by the embodiments of the present application to threedimensional space.As shown in fig. 4 a, to threedimensional
The geometric object in space carries out subdivision, and subdivision result can reflect in the octree structure shown in Fig. 4 b.For example, in Fig. 4 a
In, eight an equal amount of subcube B1, B2, B3 ... B8 are divided into a cube A in Fig. 4 a, reflection is in fig. 4b
For eight child node b1, b2, b3 ... b8 of a root node a connection.Continue to be split into eight for subcube B3 in Fig. 4 a
An equal amount of subcube C1, C2, C3 ... C8, reflection in fig. 4b for eight child node c1, c2 of child node b3 connection,
c3……c8.Continue to be split into eight an equal amount of subcube D1, D2, D3 ... D8 for subcube B8 in Fig. 4 a,
Reflection is eight child node d1, d2, d3 ... d8 of child node b3 connection in fig. 4b.
Each child node can characterize the set at a voxel midpoint in abovementioned Octree, may include one in a voxel
A or multiple points.
It is by carrying out corresponding cloud of depth image by slightly to essence for the octree structure that the above process one constructs
Segmentation obtains the Octree child node of each level from top to bottom, and as shown in Figure 4 b, the first level includes child node a, the second layer
Grade includes child node b1, b2, b3 ... b8.Third level includes child node c1, c2, c3 ... c8, d1, d2, d3 ... d8.Needle
To each child node in each layer, if whether the point for continuing to divide be by including in corresponding cloud of the child node is same
In a plane, such as in Fig. 4 b, corresponding cloud of child node b1 in the same plane, so do not need to child node b1 after
Continuous subdivision, and the point that the corresponding voxel of child node b3 includes is not in the same plane, so needing to continue to child node b3
It carries out continuing to divide, obtains child node c1, c2, c3 ... c8 in next level.To the last every height in a level
The point for including in corresponding cloud of node is all in one plane, alternatively, the quantity in point cloud including point is less than amount threshold,
Do not continuing to divide the child node in the level.
Whether need to continue subdivision for each subcube in each level, can determine whether the node pair in each level
The point cloud answered whether in one plane, if the point for including in subcube not in one plane, needs to continue to divide
The subcube；If the point for including in subcube is in one plane, not continuing to divide the subcube can be with
It is realized by following procedure A.It should be noted that when the quantity for the point for including in subcube is less than amount threshold, i.e.,
The point for including in subcube is set in one plane, also not continue to divide the subcube.
Process A, the normal vector of each voxel of rapidly extracting judge whether voxel needs to continue to divide.
The normal vector that the corresponding voxel of each child node is calculated since the thicker level in octree structure, is then pressed
According to by slightly to the normal direction for the corresponding voxel of eight octree nodes of the sequence levelone levelone of essence calculated downwards in each level
Amount.
In a kind of example, for the corresponding voxel of each octree nodes, voxel can be calculated using PCA algorithm
Normal vector, such as the corresponding voxel of node b1, node b1 correspond to a voxel, the threedimensional point cloud for including in the voxel corresponding 3
A characteristic value, for example be r1, r2, r3, the corresponding feature vector of each characteristic value, that is to say, that include three in the voxel
Corresponding 3 feature vectors of dimension point cloud, then determine to be worth the smallest characteristic value from three characteristic values r1, r2, r3, for example r2 is most
It is small, then the corresponding feature vector of characteristic value r2 is the normal vector for the threedimensional point cloud for including in the voxel.
After the normal vector (the as corresponding feature vector of minimal eigenvalue) for calculating the corresponding voxel of node b1,
The quotient between the sum of this feature value r2 and three characteristic values (i.e. the sum of r1, r2, r3 three) is calculated, by the quotient and third
Threshold value is compared, if the quotient is less than or equal to the third threshold value, include in the corresponding voxel of node b1 three
Point cloud is tieed up in the same plane, it is also assumed that the threedimensional point cloud for including in the corresponding voxel of node b1 is a plane.
If the quotient is greater than the third threshold value, the threedimensional point cloud for including in the corresponding voxel of node b1 is not in approximately the same plane
On, it is also assumed that the threedimensional point cloud for including in the corresponding voxel of node b1 is not a plane, it may be possible to constitute multiple
Plane needs node b1 to continue eight child nodes for being divided into next stage in this case, eight child nodes of next stage after
It is continuous to determine the need for continuing to segment in the manner described above.For each child node, if meeting the condition of convergence, so that it may no longer
Subdivision, the condition of convergence can be any one of the following conditions: condition one, and the quantity for the point for including in the corresponding voxel of node is small
In amount threshold；Condition two, the quotient between the sum of the corresponding characteristic value of normal vector of the corresponding voxel of node and other feature value
Value is greater than third threshold value.
Since minimal eigenvalue (namely mean square error) can actually reflect the degree of fluctuation put on normal vector direction,
If this minimal eigenvalue is less than or equal to second threshold, illustrate that the degree of fluctuation put on normal vector direction is small, also
Illustrate that node b1 is a plane.
A by the above process can count eight child nodes in a level simultaneously for a level
Algorithm vector, determines whether the point cloud for including in the corresponding voxel of each byte point needs to continue to divide, so as to quickly complete
At the cutting procedure of eight child nodes of a level, for a child node, pass through abovementioned rapidly extracting child node pair
The normal vector answered may be implemented quickly to determine whether the point cloud for including in the corresponding voxel of a node is a plane, from
And it can quickly determine the need for continuing to segment the point cloud for including in the corresponding voxel of the node.
Octree structure is being obtained by process A, the point that the corresponding voxel of the octree nodes in each level includes
In one plane, that is to say, that the corresponding normal vector of the corresponding voxel of each octree nodes, it later can be by from each
The voxel with coplanar relation is determined in the corresponding voxel of a node, the voxel with coplanar relation is merged, and is determined
The concrete mode of coplanar relation may refer to following processes three.
Process two corresponds between each voxel whether have coplanar relation for node layer each in octree structure.
In a kind of possible implementation, using the corresponding body of node each in each layer of Hough (Hough) transformation calculations
Whether element has coplanar relation, and the duality relation of the sample space and parameter space that specifically can use Hough transform is realized.
Wherein, sample space is the point cloud of the corresponding voxel packet of each octree nodes as obtained in aforementioned process A
What normal vector was constituted.One normal vector corresponding one is crossed the plane of origin, then the normal vector then corresponds to one in parameter space
A point.Illustratively, the corresponding plane equation of a voxel in sample space is ax+by+cz=1, is considered as sampled point
It is the relationship of antithesis between (x, y, z) and normal vector (a, b, c).
If the normal vector of the corresponding plane equation of a voxel is converted by Hough (Hough) transformation in parameter space
In a point, then the point that the normal vector for belonging to conplane voxel is formed in parameter space may build up together,
These points together will tend to a central point in parameter space, that is to say, that other are in this central point
Surrounding fluctuates.With the increase of sampled point, the normal vector of many voxels is also just acquired, the result of parameter space will have centainly
Statistical property, that is to say, that with the increase of sampled point, these sampled points, which can be presented, levels off to the characteristic at some center, i.e.,
Closer to the center, sampled point is more, statistical property possessed by the result here it is parameter space.Due to coplanar relation
The corresponding normal vector of each voxel be parallel to each other, so the statistical property being had according to the result of parameter space can obtain
The set of coplanar voxel object.
In some embodiments, for the normal direction of the corresponding voxel of the child node for belonging to same level in octree structure
Amount can determine in the result of parameter space there is the coplanar corresponding set of voxel, specific mistake by Kmeans algorithm
Journey is as follows: the aggregation central point in parameter space is got first, than if any 3 aggregation central points, then in each aggregation
Centered on heart point, a ball is drawn by the radius of a ball of preset value, the point in ball is the set with the voxel of coplanar relation,
The specific value of the preset value is not construed as limiting, and can be configured according to actual needs.
It obtains in octree structure in the voxel of same level child node, has for by Hough (Hough) mapping mode
There is the set of the voxel of coplanar relation, so that it may obtain the corresponding plane equation of set of these voxels with coplanar relation.
By the available multiple coarse extraction planes of abovementioned Hough (Hough) conversion process, each coarse extraction plane is corresponding
One plane equation.
It, can be by clustering processing mode to level of child nodes pair each in octree structure in alternatively possible implementation
The voxel answered is clustered, child node c1, c2, c3 ... the c8 and d1, d2, d3 ... for including with third level in Fig. 4 b
For d8, the corresponding voxel of 16 child nodes for including in the third level is clustered, specific cluster process can refer to abovementioned
Clustering processing mode in embodiment one.
It, can be with for being clustered below with reference to example to 16 child nodes that the third level in octree structure includes
From by taking a child node c1 as an example, this child node c1 is melted with other 15 child nodes respectively in 16 child nodes
It closes, finds out a child node from other 15 child nodes in addition to child node c1 and melt with what child node c1 was merged
The corresponding mean square error of child node is minimum after conjunction, for example child node is corresponding after the fusion merged with child node c1 found out
The smallest child node of mean square error be c3, and the smallest mean square error is less than or equal to first threshold, then can will be sub
The pointcloud fitting that the point cloud and the corresponding voxel of child node c3 that the corresponding voxel of node c1 includes include is at a plane.This melts
Conjunction process iteration is multiple, until can not find the child node that can be merged twobytwo.Fusion results are obtained obtaining third level
Later, fusion results are merged with child node each in the second level (child node b1, b2, b4 ... b7).Again will later
The fusion results of each child node of second level are merged with the node of the first level, it is last it is available between any two not
The node that can be merged, by abovementioned cluster process, available multiple coarse extraction planes.
The coarse extraction plane obtained in process two is the fitting result of a maximum probability, is inaccurate, if it is eight
The fitting result that calculates in the highlevel volume elements of tree construction is pitched, then this is the result is that very inaccurate.For example, process two obtains
Coarse extraction plane may have an outlier, that is, partial dot is in this coarse extraction plane, but not in Hough transform
It in obtained coarse extraction plane, needs to reject these outliers at this time, for the plane further refined, can be directed to
The coarse extraction plane carries out RANSAC operation, and specific implementation is referring to process four.
Process three, based on random sampling consistency (random sample consensus, RANSAC) algorithm to process two
The obtained corresponding plane equation of coarse extraction plane is refined.
In one possible implementation, it is based on the thick fitting result of Hough transform (i.e. process two), is become with Hough
For the coarse extraction plane B got in return, randomly selects the point for including in one group of point and coarse extraction plane B and be fitted one again
Plane, for example be plane X1, wherein be the point in plane that Hough transform mode is fitted, if plane X1 is corresponding square
Error is greater than the corresponding mean square error of coarse extraction plane B, then this group point is not the interior point of coarse extraction plane B, then again
One group of point and coarse extraction plane B is selected to continue to be fitted a plane；If the corresponding mean square error of plane X1 is less than or equal to
The corresponding mean square error of coarse extraction plane B, then intraoffice (inlier) point that this group point is coarse extraction plane B, later, with this
Subject to plane X1, continue that another group of point is added into plane X1, then is fitted a plane, such as plane X2 again；If plane
The corresponding mean square error of X2 is less than or equal to the corresponding mean square error of plane X1, then being subject to plane X2, continues to plane
Other points are added in X2, then are fitted a plane again, the plane after optimization is finally obtained with this iterative processing.
In alternatively possible implementation, it can be based on the thick fitting result of Hough transform (i.e. process two), with thick
For extracting plane C, corresponding equation is plane equation, takes outside coarse extraction plane C and substitutes into plane equation progress
Verify whether the point is intraoffice point, wherein intraoffice point refers to the point shoulding be in coarse extraction plane C；If the point intraoffice point,
Then the point is added into coarse extraction plane C, if the point is not intraoffice point, abandons the point, is continued flat in the coarse extraction
Whether be intraoffice point, such iteration is excellent to finally obtaining if other points being taken to be substituting to the plane equation to verify other points outside the C of face
Plane after change.
In multiple planes that process three obtains, may have two planes is a plane in fact, it is possible to mistake
Multiple planes that journey three obtains carry out implementing process referring to the phase of process five in abovedescribed embodiment one as merged plane operations
Hold inside the Pass.
Embodiment three
In the embodiment of the present application, the image data to be processed in abovementioned steps 201 may be RGB image.
In one example, can shoot Same Scene or so two images, this two images by binocular camera is
RGB image, wherein can be according to left figure (hereinafter referred to as A figure) and right figure (hereinafter referred to as B figure), it can be with vertical for A figure and B figure
Body matching algorithm (such as binocular solid matching (Stereo Matching)) obtains disparity map, and then available depth image.
Then plane fitting can be carried out to cloud according to depth image, to realize the purpose of detection plane.
Multiple planes are quickly detected in order to realize, to the two width figure of left and right of binocular camera shooting in the embodiment of the present application
As being split processing respectively, segmentation result is obtained.For example the corresponding face block of target object is obtained using semantic segmentation mode, then
For example, handling to obtain multiple face blocks with similar quality using image presegmentation.
For carrying out semantic segmentation processing for A figure, the segmentation result that progress semantic segmentation obtains is schemed for A are as follows: tool
There is image after the segmentation of at least one target object, for example the semanteme being arranged is people, then can be obtained by the mark for being labelled with people
The people that note frame is outlined, the corresponding region of this callout box includes target object: people, then by this corresponding region of annotation frame
Then referred to as subgraph carries out subsequent processing at least one subgraph that semantic segmentation is handled.For example, by A figure point
The multiple subgraphs obtained after cutting, then handle to obtain parallax with the multiple subgraphs obtained after B figure segmentation are carried out Stereo matching
Figure, then determines depth map further according to disparity map.
For carrying out image presegmentation processing for A figure, the segmentation result that is split for A figure are as follows: including
It is multiple to be alternatively referred to as subgraph for image after the segmentation in the region (also referred to as face block) of plane, each face block, then be directed to
Each subgraph carries out subsequent processing.For example, will obtained multiple subgraphs after A figure segmentation, such as A1, A2 and A3, then with by B
The multiple subgraphs obtained after figure segmentation, such as B1, B2 and B3 carry out Stereo Matching Algorithm and handle to obtain disparity map, then again
Depth map is determined according to disparity map.Wherein, neutron image A1 and subgraph B1 is in different points of view in Same Scene
Same place band of position C is shot, so can finally obtain the position area according to subgraph A1 and subgraph B1
The corresponding depth map of domain C, the corresponding depth map of this band of position C are known as sub depth image.
After obtaining multiple sub depth images, every sub depth image can be reverted into the point cloud in threedimensional space,
Then the sub depth image that corresponding cloud can be fitted to a plane can be filtered out from multiple sub depth images, then
Clustering processing is carried out for the sub depth image filtered out, finally obtains multiple planes.
Below with reference to specific example, the plane monitoringnetwork method in embodiment three is introduced.
Referring to Fig. 5, for another plane monitoringnetwork method process schematic provided by the embodiments of the present application.As shown in figure 5, should
Plane monitoringnetwork method includes the following steps:
Step S1 obtains binocular camera and shoots to obtain left (A figure), right (B figure) two width RGB images.
Step S2 carries out semantic segmentation to A figure and B figure respectively or image presegmentation is handled, obtains left and right two width RGB
The corresponding segmentation result of image.
Wherein, semantic segmentation can be there are many possible implementation, and a kind of possible partitioning scheme is to use full convolution
Network implementations coarse segmentation.
It before carrying out coarse segmentation using full convolutional network, needs to be trained full convolutional network, can just obtain can be with
Full convolutional network model for being split to RGB image.Wherein, in training process end to end, input can be passed through
Data set is the image for being labelled with the binocular camera shooting of plane, and when mark needs to carry out the mark of pixel scale, multiple planes
Content is indicated using multiple labels (label).Following S21~the S23 of specific training process:
S21, by the RGB image of the RGB image (referred to as A figure) of the left camera shooting of binocular camera and the shooting of right camera
The mark of (referred to as B figure) progress pixel scale.Mark schematic diagram as shown in FIG. 6, what wherein pixel scale marked out is to ride certainly
The people of driving.
S22, the A figure after mark and the B figure after mark are separately input into full convolutional network.
S23 obtains the available parameter of one group of full convolutional network using the mode training of the full convolutional network of general training,
Available parameter is the model structure parameter of full convolutional network to get to trained full convolutional network model.
It is carried out before electronic equipment factory it should be noted that abovementioned training process S21~S23 can be, also
Say that trained full convolutional network model has been preconfigured in the electronic device before factory, then needing to make in user
When being detected with the plane monitoringnetwork method to plane, the processor in electronic equipment can call directly the full volume of preconfiguration
Product network model, is split processing to RGB image to be processed.Abovementioned full convolutional network model is also possible in electronic equipment
After factory, it is trained by abovementioned training process S21~S23.
It, will when needing to be split using full convolutional network model RGB image to be processed collected to binocular camera
RGB image to be processed is input in full convolutional network model, output result be include pixel scale mark semantic segmentation figure
Picture.
In alternatively possible partitioning scheme, using convolutional neural networks (regionconvolutional neural
Networks, RCNN) perhaps YOLO (you only look once) or SSD (single shot detector) etc.
A variety of quick semantic segmentation modes, are split processing to RGB image to be processed.
Stating several RCNN, YOLO, SSD etc. in use, quickly semantic segmentation mode carries out slightly RGB image to be processed
Before segmentation, needs to be trained RCNN model, YOLO model or SSD model, can just obtain can be used for RGB image
RCNN model, YOLO model or the SSD model being split.
In training process end to end, the data set of RCNN model, YOLO model or SSD model is input to as mark
The RGB image of plane, when mark, are marked by the way of bounding box, and the result obtained in this way is relative to pixel scale
It is a thicker result for mark.Later, using general VGG16 AlexNet mode training network.Specific instruction
Practice the following S24~S25 of process:
S24, by the RGB image of the RGB image (referred to as A figure) of the left camera shooting of binocular camera and the shooting of right camera
The mark of (referred to as B figure) progress pixel scale.Mark schematic diagram as shown in Figure 7, wherein the mode of bounding box marks
Out be imageregion that rectangle frame is outlined, as Fig. 7 rectangle frame in the object that is outlined be a vehicle.
Data set (B after the A after multiple groups mark schemes and marks schemes) is input to appointing in RCNN, YOLO, SSD by S25
One neural network, training obtains corresponding network model, for example data set input RCNN training is obtained corresponding RCNN mould
Type, YOLO correspond to YOLO model, SSD corresponds to SSD model.
It is carried out before electronic equipment factory it should be noted that training process S24~S25 can be, that is,
Say that trained full convolutional network model has been preconfigured in the electronic device before factory, then user need using
When the plane monitoringnetwork method detects plane, the processor in electronic equipment can call directly the full convolution of preconfiguration
Network model is split processing to RGB image to be processed.Abovementioned full convolutional network model is also possible to go out in electronic equipment
After factory, it is trained by abovementioned training process S21~S23.
It, will when needing to be split using full convolutional network model RGB image to be processed collected to binocular camera
RGB image to be processed is input in full convolutional network model, output result be include pixel scale mark semantic segmentation figure
Picture.
The corresponding region of callout box that above two semantic segmentation mode obtains is carried out by granularity of a target object
Segmentation, and a target object may include multiple planes, such as vehicle, in three dimensions corresponding cloud and more than one
A plane, the result obtained in this way are simultaneously inaccurate.So each callout box obtained for above two semantic segmentation is corresponding
Region, continue to divide, segmentation granularity can be configured according to actual needs, herein with no restrictions.
The partitioning scheme that being split for the corresponding region of each callout box may refer in abovedescribed embodiment one carries out,
Subsequent processing mode (such as clustering processing, etching operation, region growing operation) can also be with reference to related in abovedescribed embodiment one
Content, details are not described herein again.
In another partitioning scheme, image presegmentation processing is to be schemed using image presegmentation algorithm to RGB to be processed
Segmentation as carrying out similar face block.Wherein image presegmentation algorithm is the algorithm clustered based on similarity between pixel, is obtained
To segmentation result, RGB image as shown in Figure 8 a is schemed after obtaining segmentation as shown in Figure 8 b after image presegmentation algorithm
Picture.Partitioning scheme compared to CNN, the segmentation result based on image presegmentation algorithm are more fine.
A figure and B figure are obtained a disparity map, then the disparity map that will be obtained by binocular parallax matching algorithm by step S3
Depth image is converted to, includes multiple subgraphs in the depth image.
In one example, the segmentation result based on abovementioned steps S2, can scheme A and B figure is matched using binocular parallax
Algorithm is handled, to realize the position of the face block obtained after reduction segmentation in three dimensions.Binocular parallax matching algorithm
Basic principle is the position for searching for the current pixel of a wherein mesh camera in another mesh camera, passes through image coordinate, combines
The position of the inside and outside ginseng also original pixel of binocular camera in space, therefore when carrying out binocular camera disparity computation, we can be with
Some pixel is first extracted in a divisional plane block, polar curve search is carried out, when the points of search are more and more, so that it may
Obtain many threedimensional space points.Binocular parallax matching algorithm may include currently used BM, SGBM.
It should be understood that due to the subgraph after having been divided in abovementioned steps S2 binocular vision can also be being carried out
When difference matching, directly it may be the calculating of the subgraph progress binocular parallax of plane based on presegmentation result, and be reduced directly
The threedimensional space position of these subgraphs.Calculation amount can be reduced in this way, accelerate the speed of plane monitoringnetwork.
When semantic segmentation obtains subgraph, it is flat can to determine whether the target object split has according to mark
Face only can determine disparity map to the subgraph with face block then when determining disparity map according to step S3, again will later
Obtained disparity map is converted to depth image.Can reduce the treating capacity of subsequent cluster in this way, save detection planarization process when
Between.
Step S4 determines the son of corresponding cloud in the same plane from each subgraph that depth image includes
Image.
Herein, plane fitting can be carried out to each subgraph by PCA algorithm, calculates the mean square error of fit Plane
Difference, if the mean square error of fit Plane is less than first threshold, then corresponding cloud of the subgraph is in the same plane.
Step S5 carries out clustering processing, obtains to the subgraph of each corresponding cloud determined in the same plane
To multiple coarse extraction planes.
In step s 5, clustering processing process can refer to the phase of the clustering processing process in abovedescribed embodiment one inside the Pass
Hold, details are not described herein again.
Step S6 carries out process of refinement, plane after being optimized to each coarse extraction plane.
In step s 6, one of erosion algorithm, algorithm of region growing and merging plane operations or more can be used
To coarse extraction plane process of refinement obtained in step S5, correlated process can refer to the phase of abovedescribed embodiment one inside the Pass for kind operation
Hold.
It should be understood that after abovementioned steps S3 obtains depth image, abovedescribed embodiment can also be used in the embodiment of the present application
Two processing mode, which is realized, detects multiple planes, and related content can refer to the related content of embodiment two, and details are not described herein again.
By three above embodiment, on the one hand may be implemented to detection while multiple planes in scene, and image
Sensor is no longer limited to depth camera, and algorithm is also no longer limited to traditional clustering algorithm.On the other hand, by being carried out to image
Presegmentation, then handling respectively the image after presegmentation can guarantee that the high speed of plane detection algorithm executes, thus
The multiple planes being quickly detected in scene.
A kind of plane monitoringnetwork method is also provided in some other embodiment, in the embodiment of the present application, is set applied to calculating
It is standby, referring to Fig. 9, method includes the following steps:
Step 901, image data to be processed is obtained.
Herein, image data to be processed can be two dimensional image, than as the following examples one in depth image, embodiment
The first RGB image and the second RGB image in three, can also be with three dimensional point cloud, the point cloud in two as the following examples, herein
It is not specifically limited.
It should be understood that image data to be processed can be electronic equipment shooting, it is also possible to from electronic equipment for depositing
It is obtained in the picture library of storage image, is also possible to what other equipment were sent.
Step 902, semantic segmentation is carried out to image data to be processed, obtains N number of subimage data with markup information,
N is the integer greater than 1；Markup information is used to mark the target object in subimage data.
Step 903, according to the markup information of each subimage data, from N number of subimage data with markup information
Determine the Q subimage datas with plane；Q is the integer greater than 0 and less than or equal to N.
Step 904, determine that each the subimage data with plane is corresponding in the Q subimage datas with plane
Point cloud information.
Step 905, according to the subimage data corresponding point each in the Q subimage datas with plane with plane
Cloud information determines K coarse extraction plane from the Q subimage datas with plane；Wherein, K is whole more than or equal to Q
Number.
Step 906, processing is optimized to K coarse extraction plane, obtains plane after L optimization；L is just no more than K
Integer.
Based on the program, semantic segmentation, available N number of subgraph with markup information are carried out to data image to be processed
As data, the Q subimage datas with plane are then determined from N number of subimage data with markup information, thus
It can only need to carry out plane monitoringnetwork to the Q subimage datas with plane, not need to the subgraph number for not having plane
According to plane monitoringnetwork is carried out, so as to reduce treating capacity, and the Q subimage datas with plane are handled, it can be with
Detect more than one plane.
In one possible implementation, the image data to be processed in abovementioned steps 901 includes being clapped by binocular camera
The first RGB image and the second RGB image taken the photograph.
Abovementioned steps 902 may be accomplished by: carrying out semantic segmentation to first RGB image, obtains N number of tool
There is the first subgraph of markup information；And semantic segmentation is carried out to second RGB image, it obtains N number of with markup information
Second subgraph；Wherein, first subgraph with markup information and with first subgraph with position
The second subgraph with markup information of corresponding relationship forms the subimage data with markup information.
Illustratively, the first RGB image can be the A figure in embodiment three, and the second RGB image can be in embodiment three
B figure.
Abovementioned steps 904 may be accomplished by: for the Q each of the subimage datas with plane
Subimage data with plane executes: having markup information according to include in the subimage data with plane
First subgraph and the second subgraph with markup information with first subgraph with position corresponding relationship, really
Make disparity map；Sub depth image is determined according to the disparity map；According to the sub depth image, determine described with flat
The corresponding point cloud information of the subimage data in face.
Abovementioned steps 905 may be accomplished by: according to corresponding cloud of the subimage data with plane
Information determines K coarse extraction plane from Q sub depth images.
It should be noted that determining K coarse extraction plane and clustering processing and being optimized to coarse extraction plane
Etc. processes specific implementation may refer to the related embodiment in embodiment one description, details are not described herein again.
Abovementioned plane monitoringnetwork method can be also used for intelligent robot navigation in addition to being applied to augmented reality field, from
It is dynamic the fields such as to drive, so as to realize barrier avoiding function that computer judges automatically.
In other embodiments of the application, the embodiment of the present application also provides a kind of calculating equipment, as shown in Figure 10, should
Calculating equipment may include: processor 1001；Memory 1002；And one or more computer programs 1003, abovementioned each device
Part can be connected by one or more communication bus 1004.
Wherein the one or more computer program 1003 is stored in abovementioned memory 1002 and is configured as being located
It manages device 1001 to execute, which includes instruction, and exemplary, abovemetioned instruction can be used for executing such as
Each step in Fig. 2 and Fig. 5 in corresponding embodiment.Specifically, processor 1001 can be used for executing the step in Fig. 2
201 step 205, processor 1001 are used to execute the step S1 step S6 in Fig. 5.
Through the above description of the embodiments, it is apparent to those skilled in the art that, for description
It is convenienct and succinct, only the example of the division of the above functional modules, in practical application, can according to need and will be upper
It states function distribution to be completed by different functional modules, i.e., the internal structure of device is divided into different functional modules, to complete
All or part of function described above.The specific work process of the system, apparatus, and unit of foregoing description, before can referring to
The corresponding process in embodiment of the method is stated, details are not described herein.
Each functional unit in each embodiment of the embodiment of the present application can integrate in one processing unit, can also be with
It is that each unit physically exists alone, can also be integrated in one unit with two or more units.Abovementioned integrated list
Member both can take the form of hardware realization, can also realize in the form of software functional units.
If the integrated unit is realized in the form of SFU software functional unit and sells or use as independent product
When, it can store in a computer readable storage medium.Based on this understanding, the technical solution of the embodiment of the present application
Substantially all or part of the part that contributes to existing technology or the technical solution can be with software product in other words
Form embody, which is stored in a storage medium, including some instructions use so that one
It is each that computer equipment (can be personal computer, the first electronic equipment or the network equipment etc.) or processor execute the application
The all or part of the steps of a embodiment the method.And storage medium abovementioned include: flash memory, mobile hard disk, only
Read the various media that can store program code such as memory, random access memory, magnetic or disk.
The above, the only specific embodiment of the embodiment of the present application, but the protection scope of the embodiment of the present application is not
It is confined to this, anyone skilled in the art can think easily in the technical scope that the embodiment of the present application discloses
The change or replacement arrived should all be covered within the protection scope of the embodiment of the present application, therefore the protection model of the embodiment of the present application
Enclosing should be subject to the protection scope in claims.
Claims (28)
1. a kind of plane monitoringnetwork method is applied to calculate equipment, which is characterized in that the described method includes:
Obtain image data to be processed；
The image data to be processed is split, N number of subimage data is obtained, the N is the integer greater than 1；
Determine the corresponding point cloud information of at least one subimage data in N number of subimage data；
The corresponding point cloud information of at least one subimage data according in N number of subimage data, to N number of son
Corresponding cloud of image data carries out clustering processing, obtains K coarse extraction plane；The K is the positive integer no more than the N；
Processing is optimized to the K coarse extraction plane, obtains plane after L optimization；The L is just no more than the K
Integer.
2. the method as described in claim 1, which is characterized in that the image data to be processed is depth image；The depth
It include the image coordinate and depth value of each pixel in image；
It is described that the image data to be processed is split, obtain N number of subimage data, comprising:
The depth image is split, N number of sub depth image is obtained；
The corresponding point cloud information of at least one subimage data in determination N number of subimage data, comprising:
Determine at least one of N number of sub depth image corresponding point cloud information of sub depth image；
The corresponding point cloud information of at least one subimage data according in N number of subimage data, to N number of son
Corresponding cloud of image data carries out clustering processing, obtains K coarse extraction plane, comprising:
According to the corresponding point cloud information of every sub depth image in N number of sub depth image, every sub depth image is determined
The mean square error of plane after the fitting of corresponding cloud；
The sub depth image for meeting first condition is determined from N number of sub depth image, forms subgraph image set to be processed；Institute
The mean square error for stating plane after the fitting that first condition includes the corresponding cloud of sub depth image is less than or equal to the first threshold
Value；
It concentrates the corresponding cloud of sub depth image for including to cluster the subgraph to be processed, it is flat to obtain K coarse extraction
Face.
3. the method as described in claim 1, which is characterized in that the image data to be processed is depth image；
It is described that the image data to be processed is split, obtain N number of subimage data, comprising:
The depth image is split, N number of sub depth image is obtained；
The corresponding point cloud information of at least one subimage data in determination N number of subimage data, comprising:
Determine at least one of N number of sub depth image corresponding point cloud information of sub depth image；
The corresponding point cloud information of at least one subimage data according in N number of subimage data, to N number of son
Corresponding cloud of image data carries out clustering processing, obtains K coarse extraction plane, comprising:
Every corresponding cloud of sub depth image in N number of sub depth image is constructed into graph structure as a node；It is described
Each node in graph structure is stored with the corresponding point cloud information of the node；
Each node in the graph structure is traversed, determines two nodes for meeting second condition in the graph structure, then
Side is constructed between two nodes for meeting the second condition；The second condition is corresponding including any node in two nodes
Point cloud depth value is continuous and the normal vector of two node corresponding points clouds between angle be less than angle threshold value；
Determine sub depth map corresponding with having the node at least one side in the graph structure in N number of sub depth image
Picture forms subgraph image set to be processed；
It concentrates the corresponding cloud of sub depth image for including to cluster the subgraph to be processed, it is flat to obtain K coarse extraction
Face.
4. the method as described in claim 1, which is characterized in that the image data to be processed includes being shot by binocular camera
First RGB image and the second RGB image；
It is described that the image data to be processed is split, obtain N number of subimage data, comprising:
Image presegmentation is carried out to first RGB image, obtains N number of first face block；And figure is carried out to second RGB image
As presegmentation, N number of second face block is obtained；First face block and second face block have position corresponding relationship；
For each first face block in N number of first face block, execute:
Second face block that there is position corresponding relationship with first face block is determined from N number of second face block；
There is second face block of position corresponding relationship according to first face block and with first face block, determine
Disparity map；Sub depth image is determined according to the disparity map；
Subgraph image set to be processed is formed according to the N number of sub depth image determined；
The corresponding point cloud information of at least one subimage data in determination N number of subimage data, comprising:
Determine at least one of N number of sub depth image corresponding point cloud information of sub depth image；
The corresponding point cloud information of at least one subimage data according in N number of subimage data, to the N
Corresponding cloud of a subimage data carries out clustering processing, obtains K coarse extraction plane, comprising:
The corresponding point cloud information of at least one subimage data according in N number of subimage data, to described wait locate
Reason subgraph concentrates the N number of corresponding cloud of sub depth image for including to be clustered, and obtains K coarse extraction plane.
5. such as the described in any item methods of claim 24, which is characterized in that described to include to the subgraph concentration to be processed
Corresponding cloud of sub depth image clustered, obtain K coarse extraction plane, comprising:
The corresponding point cloud information of the sub depth image of every for including is concentrated according to the subgraph to be processed, establishes minimum heap data
Structure；The minimum heap data structure is used for the mean square error according to every sub corresponding cloud of depth image to described to be processed
The sub depth image that subgraph is concentrated is ranked up, and most positioned at the mean square error of the corresponding cloud of sub depth image on heap top
It is small；
For the minimum heap data structure, predetermined registration operation is executed, until any two section in the minimum heap data structure
The mean square error of plane is greater than the first threshold after the fitting of corresponding cloud of point, obtains the K coarse extraction plane；
Wherein, the predetermined registration operation include: from taking out sub depth image in heap top in the minimum heap data structure, if from institute
It states in the adjacent sub depth image of sub depth image and determines the sub depth image for meeting third condition, then by the sub depth map
As being merged with the sub depth image for meeting third condition, sub depth image after being merged, the third condition packet
Mean square error after including pointcloud fitting plane corresponding with the sub depth image is less than first threshold and the mean square error most
It is small；Sub depth image after the fusion is added into the minimum heap data structure.
6. the method as described in claim 1, which is characterized in that the image data to be processed is the point for including in threedimensional space
Cloud；
It is described that the image data to be processed is split, obtain N number of subimage data, comprising:
Using the threedimensional space as the node of the first level of octree structure；
For the first level and the ith level each child node for including in the octree structure, execute: if the sub section
Point meets fourth condition, then carries out eight equal portions segmentations to the child node, obtain eight child nodes of i+1 level；Wherein institute
It states the mean square error that fourth condition includes corresponding cloud of child node and is greater than the first threshold；The i is the integer greater than 1,
Ith level includes 8i child node；
Until all child nodes for including in last described level meet fifth condition, building obtain include M level son
The octree structure of node；The fifth condition includes the mean square error of corresponding cloud of child node no more than first threshold
Value, alternatively, corresponding cloud of child node includes that point quantity is less than amount threshold；
Determine N number of undivided child node in the octree structure；
The corresponding point cloud information of at least one subimage data in determination N number of subimage data, comprising:
Determine the undivided corresponding point cloud information of child node of at least one of described N number of undivided child node；
The corresponding point cloud information of at least one subimage data according in N number of subimage data is described right
Corresponding cloud of N number of subimage data carries out clustering processing, obtains K coarse extraction plane, comprising:
According to the undivided corresponding point cloud information of child node of at least one of N number of undivided child node, to described
Corresponding cloud of N number of undivided child node described in octree structure carries out clustering processing, obtains K coarse extraction plane.
7. method as claimed in claim 6, which is characterized in that it is described according in N number of undivided child node at least
The corresponding point cloud information of one undivided child node, it is corresponding to N number of undivided child node described in the octree structure
Point cloud carry out clustering processing, obtain K coarse extraction plane, comprising:
According to the undivided corresponding point cloud information of child node of at least one of N number of undivided child node, institute is determined
The normal vector of each corresponding cloud of undivided child node in N number of undivided child node is stated, and the normal vector is passed through
Hough transformation is at the point in parameter space；
Determine the K point set that the normal vector of N number of corresponding cloud of undivided child node is formed in parameter space, often
A point set has an aggregation center；
For each point set, the point fallen in the pericentral preset range of aggregation of the point set is determined；
The permeate coarse extraction of the corresponding corresponding cloud of undivided child node of the point fallen within a preset range is put down
Face.
8. the method according to claim 1 to 7, which is characterized in that described excellent to K coarse extraction plane progress
Change processing obtains plane after L optimization, comprising:
Determine the normal vector of each coarse extraction plane in the K coarse extraction plane；
Any coarse extraction plane in the K coarse extraction plane is traversed, meets the coarse extraction plane of Article 6 part if it exists, then
The coarse extraction plane is permeated a plane with the coarse extraction plane for meeting the Article 6 part, is obtained flat after L optimization
Face；
Wherein, the Article 6 part includes: that normal vector is parallel with the normal vector of the coarse extraction plane and flat with the coarse extraction
Variance after the fit Plane of face is less than variance threshold values.
9. a kind of plane monitoringnetwork method is applied to calculate equipment, which is characterized in that the described method includes:
Obtain image data to be processed；
Semantic segmentation is carried out to the image data to be processed, obtains N number of subimage data with markup information, the N is
Integer greater than 1；The markup information is used to mark the target object in the subimage data；
According to the markup information of each subimage data, Q are determined from N number of subimage data with markup information
Subimage data with plane；The Q is the integer greater than 0 and less than or equal to the N；
Determine that each corresponding cloud of the subimage data with plane is believed in the Q subimage datas with plane
Breath；
According to the subimage data corresponding point cloud information each in the Q subimage datas with plane with plane,
K coarse extraction plane is determined from the Q subimage datas with plane；The K is whole more than or equal to the Q
Number；
Processing is optimized to the K coarse extraction plane, obtains plane after L optimization；The L is just no more than the K
Integer.
10. method as claimed in claim 9, which is characterized in that the image data to be processed includes being shot by binocular camera
The first RGB image and the second RGB image；
It is described that semantic segmentation is carried out to the image data to be processed, N number of subimage data with markup information is obtained, is wrapped
It includes:
Semantic segmentation is carried out to first RGB image, obtains N number of the first subgraph with markup information；And to described
Two RGB images carry out semantic segmentation, obtain N number of the second subgraph with markup information；Wherein, each described with mark letter
The first subgraph ceased and the second subgraph with markup information with first subgraph with position corresponding relationship
One subimage data with markup information of composition；
It is described to determine in the Q subimage datas with plane each with corresponding cloud of subimage data of plane
Information, comprising:
For subimage data of each of the Q subimage datas with plane with plane, execute:
According to including the first subgraph with markup information and with described in the subimage data with plane
Second subgraph with markup information of one subgraph with position corresponding relationship, determines disparity map；
Sub depth image is determined according to the disparity map；
According to the sub depth image, the corresponding point cloud information of subimage data with plane is determined；
It is described according in a subimage datas with plane of the Q each to there is corresponding cloud of subimage data of plane to believe
Breath determines K coarse extraction plane from the Q subimage datas with plane, comprising:
According to the corresponding point cloud information of the subimage data with plane, K coarse extraction is determined from Q sub depth images
Plane.
11. the method as described in claim 9 or 10, which is characterized in that the subimage data according to plane
Corresponding point cloud information determines K coarse extraction plane from Q sub depth images, comprising:
Every corresponding cloud of sub depth image in the Q sub depth images is constructed into graph structure as a node；It is described
Each node in graph structure is stored with the corresponding point cloud information of the node；
Each node in the graph structure is traversed, determines two nodes for meeting second condition in the graph structure, then
Side is constructed between two nodes for meeting the second condition；The second condition is corresponding including any node in two nodes
Point cloud depth value is continuous and the normal vector of two node corresponding points clouds between angle be less than angle threshold value；
Determine sub depth map corresponding with having the node at least one side in the graph structure in the Q sub depth images
Picture forms subgraph image set to be processed；
It concentrates the corresponding cloud of sub depth image for including to carry out clustering processing the subgraph to be processed, obtains K and slightly mention
It makes even face.
12. method as claimed in claim 11, which is characterized in that described to concentrate the son for including deep the subgraph to be processed
Degree corresponding cloud of image is clustered, and K coarse extraction plane is obtained, comprising:
The corresponding point cloud information of the sub depth image of every for including is concentrated according to the subgraph to be processed, establishes minimum heap data
Structure；The minimum heap data structure is used for the mean square error according to every sub corresponding cloud of depth image to described to be processed
The sub depth image that subgraph is concentrated is ranked up, and most positioned at the mean square error of the corresponding cloud of sub depth image on heap top
It is small；
For the minimum heap data structure, predetermined registration operation is executed, until any two section in the minimum heap data structure
The mean square error of plane is greater than the first threshold after the fitting of corresponding cloud of point, obtains the K coarse extraction plane；
Wherein, the predetermined registration operation include: from taking out sub depth image in heap top in the minimum heap data structure, if from institute
It states in the adjacent sub depth image of sub depth image and determines the sub depth image for meeting third condition, then by the sub depth map
As being merged with the sub depth image for meeting third condition, sub depth image after being merged；The third condition packet
Mean square error after including pointcloud fitting plane corresponding with the sub depth image is less than first threshold and the mean square error most
It is small；Sub depth image after the fusion is added into the minimum heap data structure.
13. such as the described in any item methods of claim 912, which is characterized in that described to be carried out to the K coarse extraction plane
Optimization processing obtains plane after L optimization, comprising:
Determine the normal vector of each coarse extraction plane in the K coarse extraction plane；
Any coarse extraction plane in the K coarse extraction plane is traversed, meets the coarse extraction plane of Article 6 part if it exists, then
The coarse extraction plane is permeated a plane with the coarse extraction plane for meeting the Article 6 part；
Wherein, the Article 6 part includes: that normal vector is parallel with the normal vector of the coarse extraction plane and flat with the coarse extraction
Variance after the fit Plane of face is less than variance threshold values.
14. a kind of calculating equipment, which is characterized in that including at least one processor；
At least one described processor, is configured as performing the following operations:
Obtain image data to be processed；
The image data to be processed is split, N number of subimage data is obtained, the N is the integer greater than 1；
Determine the corresponding point cloud information of at least one subimage data in N number of subimage data；
The corresponding point cloud information of at least one subimage data according in N number of subimage data, to N number of son
Corresponding cloud of image data carries out clustering processing, obtains K coarse extraction plane；The K is the positive integer no more than the N；
Processing is optimized to the K coarse extraction plane, obtains plane after L optimization；The L is just no more than the K
Integer.
15. calculating equipment as claimed in claim 14, which is characterized in that the image data to be processed is depth image；Institute
State the image coordinate and depth value in depth image including each pixel；
It is described that the image data to be processed is split, obtain N number of subimage data, comprising:
The depth image is split, N number of sub depth image is obtained；
The corresponding point cloud information of at least one subimage data in determination N number of subimage data, comprising:
Determine at least one of N number of sub depth image corresponding point cloud information of sub depth image；
The corresponding point cloud information of at least one subimage data according in N number of subimage data, to N number of son
Corresponding cloud of image data carries out clustering processing, obtains K coarse extraction plane, comprising:
According to the corresponding point cloud information of every sub depth image in N number of sub depth image, every sub depth image is determined
The mean square error of plane after the fitting of corresponding cloud；
The sub depth image for meeting first condition is determined from N number of sub depth image, forms subgraph image set to be processed；Institute
The mean square error for stating plane after the fitting that first condition includes the corresponding cloud of sub depth image is less than or equal to the first threshold
Value；
It concentrates the corresponding cloud of sub depth image for including to cluster the subgraph to be processed, it is flat to obtain K coarse extraction
Face.
16. calculating equipment as claimed in claim 14, which is characterized in that the image data to be processed is depth image；
It is described that the image data to be processed is split, obtain N number of subimage data, comprising:
The depth image is split, N number of sub depth image is obtained；
The corresponding point cloud information of at least one subimage data in determination N number of subimage data, comprising:
Determine at least one of N number of sub depth image corresponding point cloud information of sub depth image；
The corresponding point cloud information of at least one subimage data according in N number of subimage data, to N number of son
Corresponding cloud of image data carries out clustering processing, obtains K coarse extraction plane, comprising:
Every corresponding cloud of sub depth image in N number of sub depth image is constructed into graph structure as a node；It is described
Each node in graph structure is stored with the corresponding point cloud information of the node；
Each node in the graph structure is traversed, determines two nodes for meeting second condition in the graph structure, then
Side is constructed between two nodes for meeting the second condition；The second condition is corresponding including any node in two nodes
Point cloud depth value is continuous and the normal vector of two node corresponding points clouds between angle be less than angle threshold value；
Determine sub depth map corresponding with having the node at least one side in the graph structure in N number of sub depth image
Picture forms subgraph image set to be processed；
It concentrates the corresponding cloud of sub depth image for including to cluster the subgraph to be processed, it is flat to obtain K coarse extraction
Face.
17. calculating equipment as claimed in claim 14, which is characterized in that the image data to be processed includes by binocular camera
The first RGB image and the second RGB image of shooting；
It is described that the image data to be processed is split, obtain N number of subimage data, comprising:
Image presegmentation is carried out to first RGB image, obtains N number of first face block；And figure is carried out to second RGB image
As presegmentation, N number of second face block is obtained；First face block and second face block have position corresponding relationship；
For each first face block in N number of first face block, execute:
Second face block that there is position corresponding relationship with first face block is determined from N number of second face block；
There is second face block of position corresponding relationship according to first face block and with first face block, determine
Disparity map；Sub depth image is determined according to the disparity map；
Subgraph image set to be processed is formed according to the N number of sub depth image determined；
The corresponding point cloud information of at least one subimage data in determination N number of subimage data, comprising:
Determine at least one of N number of sub depth image corresponding point cloud information of sub depth image；
The corresponding point cloud information of at least one subimage data according in N number of subimage data, to the N
Corresponding cloud of a subimage data carries out clustering processing, obtains K coarse extraction plane, comprising:
The corresponding point cloud information of at least one subimage data according in N number of subimage data, to described wait locate
Reason subgraph concentrates the N number of corresponding cloud of sub depth image for including to be clustered, and obtains K coarse extraction plane.
18. such as the described in any item calculating equipment of claim 1517, which is characterized in that described to the subgraph to be processed
Corresponding cloud of sub depth image that concentration includes is clustered, and K coarse extraction plane is obtained, comprising:
The corresponding point cloud information of the sub depth image of every for including is concentrated according to the subgraph to be processed, establishes minimum heap data
Structure；The minimum heap data structure is used for the mean square error according to every sub corresponding cloud of depth image to described to be processed
The sub depth image that subgraph is concentrated is ranked up, and most positioned at the mean square error of the corresponding cloud of sub depth image on heap top
It is small；
For the minimum heap data structure, predetermined registration operation is executed, until any two section in the minimum heap data structure
The mean square error of plane is greater than the first threshold after the fitting of corresponding cloud of point, obtains the K coarse extraction plane；
Wherein, the predetermined registration operation include: from taking out sub depth image in heap top in the minimum heap data structure, if from institute
It states in the adjacent sub depth image of sub depth image and determines the sub depth image for meeting third condition, then by the sub depth map
As being merged with the sub depth image for meeting third condition, sub depth image after being merged；The third condition packet
Mean square error after including pointcloud fitting plane corresponding with the sub depth image is less than first threshold and the mean square error most
It is small；Sub depth image after the fusion is added into the minimum heap data structure.
19. calculating equipment as claimed in claim 14, which is characterized in that the image data to be processed is to wrap in threedimensional space
The point cloud included；
It is described that the image data to be processed is split, obtain N number of subimage data, comprising:
Using the threedimensional space as the node of the first level of octree structure；
For the first level and the ith level each child node for including in the octree structure, execute: if the sub section
Point meets fourth condition, then carries out eight equal portions segmentations to the child node, obtain eight child nodes of i+1 level；Wherein institute
It states the mean square error that fourth condition includes corresponding cloud of child node and is greater than the first threshold；The i is the integer greater than 1,
Ith level includes 8i child node；
Until all child nodes for including in last described level meet fifth condition, building obtain include M level son
The octree structure of node；The fifth condition includes the mean square error of corresponding cloud of child node no more than first threshold
Value, alternatively, corresponding cloud of child node includes that point quantity is less than amount threshold；
Determine N number of undivided child node in the octree structure；
The corresponding point cloud information of at least one subimage data in determination N number of subimage data；
The corresponding point cloud information of at least one subimage data in determination N number of subimage data, comprising:
Determine the undivided corresponding point cloud information of child node of at least one of described N number of undivided child node；
The corresponding point cloud information of at least one subimage data according in N number of subimage data is described right
Corresponding cloud of N number of subimage data carries out clustering processing, obtains K coarse extraction plane, comprising:
According to the undivided corresponding point cloud information of child node of at least one of N number of undivided child node, to described
Corresponding cloud of N number of undivided child node described in octree structure carries out clustering processing, obtains K coarse extraction plane.
20. calculating equipment as claimed in claim 19, which is characterized in that described according in N number of undivided child node
The corresponding point cloud information of at least one undivided child node, N number of undivided son described in the octree structure is saved
Corresponding cloud of point carries out clustering processing, obtains K coarse extraction plane, comprising:
According to the undivided corresponding point cloud information of child node of at least one of N number of undivided child node, institute is determined
The normal vector of each corresponding cloud of undivided child node in N number of undivided child node is stated, and the normal vector is passed through
Hough transformation is at the point in parameter space；
Determine the K point set that the normal vector of N number of corresponding cloud of undivided child node is formed in parameter space, often
A point set has an aggregation center；
For each point set, the point fallen in the pericentral preset range of aggregation of the point set is determined；
The permeate coarse extraction of the corresponding corresponding cloud of undivided child node of the point fallen within a preset range is put down
Face.
21. such as the described in any item calculating equipment of claim 1420, which is characterized in that described to the K coarse extraction plane
Processing is optimized, plane after L optimization is obtained, comprising:
Determine the normal vector of each coarse extraction plane in the K coarse extraction plane；
Any coarse extraction plane in the K coarse extraction plane is traversed, meets the coarse extraction plane of Article 6 part if it exists, then
The coarse extraction plane is permeated a plane with the coarse extraction plane for meeting the Article 6 part, is obtained flat after L optimization
Face；
Wherein, the Article 6 part includes: that normal vector is parallel with the normal vector of the coarse extraction plane and flat with the coarse extraction
Variance after the fit Plane of face is less than variance threshold values.
22. a kind of calculating equipment, which is characterized in that including at least one processor；
At least one described processor, is configured as performing the following operations:
Obtain image data to be processed；
Semantic segmentation is carried out to the image data to be processed, obtains N number of subimage data with markup information, the N is
Integer greater than 1；The markup information is used to mark the target object in the subimage data；
According to the markup information of each subimage data, Q are determined from N number of subimage data with markup information
Subimage data with plane；The Q is the integer greater than 0 and less than or equal to the N；
Determine that each corresponding cloud of the subimage data with plane is believed in the Q subimage datas with plane
Breath；
According to the subimage data corresponding point cloud information each in the Q subimage datas with plane with plane,
K coarse extraction plane is determined from the Q subimage datas with plane；The K is whole more than or equal to the Q
Number；
Processing is optimized to the K coarse extraction plane, obtains plane after L optimization；The L is just no more than the K
Integer.
23. calculating equipment as claimed in claim 22, which is characterized in that the image data to be processed includes by binocular camera
The first RGB image and the second RGB image of shooting；
It is described that semantic segmentation is carried out to the image data to be processed, N number of subimage data with markup information is obtained, it is described
N is the integer greater than 1, comprising:
Semantic segmentation is carried out to first RGB image, obtains N number of the first subgraph with markup information；And to described
Two RGB images carry out semantic segmentation, obtain N number of the second subgraph with markup information；Wherein, each described with mark letter
The first subgraph ceased and the second subgraph with markup information with first subgraph with position corresponding relationship
One subimage data with markup information of composition；
It is described to determine in the Q subimage datas with plane each with corresponding cloud of subimage data of plane
Information, comprising:
For subimage data of each of the Q subimage datas with plane with plane, execute:
According to including the first subgraph with markup information and with described in the subimage data with plane
Second subgraph with markup information of one subgraph with position corresponding relationship, determines disparity map；
Sub depth image is determined according to the disparity map；
According to the sub depth image, the corresponding point cloud information of subimage data with plane is determined；
It is described according in a subimage datas with plane of the Q each to there is corresponding cloud of subimage data of plane to believe
Breath determines K coarse extraction plane from the Q subimage datas with plane, comprising:
According to the corresponding point cloud information of the subimage data with plane, K coarse extraction is determined from Q sub depth images
Plane.
24. the calculating equipment as described in claim 22 or 23, which is characterized in that the subgraph according to plane
The corresponding point cloud information of data determines K coarse extraction plane from Q sub depth images, comprising:
Every corresponding cloud of sub depth image in the Q sub depth images is constructed into graph structure as a node；It is described
Each node in graph structure is stored with the corresponding point cloud information of the node；
Each node in the graph structure is traversed, determines two nodes for meeting second condition in the graph structure, then
Side is constructed between two nodes for meeting the second condition；The second condition is corresponding including any node in two nodes
Point cloud depth value is continuous and the normal vector of two node corresponding points clouds between angle be less than angle threshold value；
Determine sub depth map corresponding with having the node at least one side in the graph structure in the Q sub depth images
Picture forms subgraph image set to be processed；
It concentrates the corresponding cloud of sub depth image for including to carry out clustering processing the subgraph to be processed, obtains K and slightly mention
It makes even face.
25. calculating equipment as claimed in claim 24, which is characterized in that described to include to the subgraph concentration to be processed
Corresponding cloud of sub depth image is clustered, and K coarse extraction plane is obtained, comprising:
The corresponding point cloud information of the sub depth image of every for including is concentrated according to the subgraph to be processed, establishes minimum heap data
Structure；The minimum heap data structure is used for the mean square error according to every sub corresponding cloud of depth image to described to be processed
The sub depth image that subgraph is concentrated is ranked up, and most positioned at the mean square error of the corresponding cloud of sub depth image on heap top
It is small；
For the minimum heap data structure, predetermined registration operation is executed, until any two section in the minimum heap data structure
The mean square error of plane is greater than the first threshold after the fitting of corresponding cloud of point, obtains the K coarse extraction plane；
Wherein, the predetermined registration operation include: from taking out sub depth image in heap top in the minimum heap data structure, if from institute
It states in the adjacent sub depth image of sub depth image and determines the sub depth image for meeting third condition, then by the sub depth map
As being merged with the sub depth image for meeting third condition, sub depth image after being merged；The third condition packet
Mean square error after including pointcloud fitting plane corresponding with the sub depth image is less than first threshold and the mean square error most
It is small；Sub depth image after the fusion is added into the minimum heap data structure.
26. such as the described in any item calculating equipment of claim 2225, which is characterized in that described to the K coarse extraction plane
Processing is optimized, plane after L optimization is obtained, comprising:
Determine the normal vector of each coarse extraction plane in the K coarse extraction plane；
Any coarse extraction plane in the K coarse extraction plane is traversed, meets the coarse extraction plane of Article 6 part if it exists, then
The coarse extraction plane is permeated a plane with the coarse extraction plane for meeting the Article 6 part, is obtained flat after L optimization
Face；
Wherein, the Article 6 part includes: that normal vector is parallel with the normal vector of the coarse extraction plane and flat with the coarse extraction
Variance after the fit Plane of face is less than variance threshold values.
27. a kind of circuit system, which is characterized in that including at least one processing circuit, be configured as executing as claim 1 to
8 any methods, alternatively, executing the method as described in claim 9 to 13 is any.
28. a kind of computer storage medium, which is characterized in that the computer storage medium includes computer program, works as calculating
When machine program is run on the computing device, so that the calculating equipment executes method as described in any of the claims 1 to 8, or
Person executes the method as described in claim 9 to 13 is any.
Priority Applications (2)
Application Number  Priority Date  Filing Date  Title 

CN201910234537  20190326  
CN2019102345377  20190326 
Publications (1)
Publication Number  Publication Date 

CN110458805A true CN110458805A (en)  20191115 
Family
ID=68482304
Family Applications (1)
Application Number  Title  Priority Date  Filing Date 

CN201910605510.4A Pending CN110458805A (en)  20190326  20190705  A kind of plane monitoringnetwork method calculates equipment and circuit system 
Country Status (1)
Country  Link 

CN (1)  CN110458805A (en) 
Cited By (4)
Publication number  Priority date  Publication date  Assignee  Title 

CN111275724A (en) *  20200226  20200612  武汉大学  Airborne point cloud roof plane segmentation method based on octree and boundary optimization 
WO2021103875A1 (en) *  20191125  20210603  歌尔股份有限公司  Laser sensorbased plane detection method and device 
WO2021147113A1 (en) *  20200123  20210729  华为技术有限公司  Plane semantic category identification method and image data processing apparatus 
CN113313701A (en) *  20210610  20210827  兰州智悦信息科技有限公司  Electric vehicle charging port twostage visual detection positioning method based on shape prior 
Citations (8)
Publication number  Priority date  Publication date  Assignee  Title 

KR20130085199A (en) *  20120119  20130729  삼성전자주식회사  Apparatus and method for plane detection 
WO2014020529A1 (en) *  20120802  20140206  Earthmine, Inc.  Threedimensional plane panorama creation through houghbased line detection 
CN106204705A (en) *  20160705  20161207  长安大学  A kind of 3D point cloud segmentation method based on multiline laser radar 
CN107341804A (en) *  20160429  20171110  成都理想境界科技有限公司  Determination method and device, image superimposing method and the equipment of cloud data midplane 
CN107358609A (en) *  20160429  20171117  成都理想境界科技有限公司  A kind of image superimposing method and device for augmented reality 
CN108665472A (en) *  20170401  20181016  华为技术有限公司  The method and apparatus of point cloud segmentation 
US20180364717A1 (en) *  20170614  20181220  Zoox, Inc.  Voxel Based Ground Plane Estimation and Object Segmentation 
CN109359614A (en) *  20181030  20190219  百度在线网络技术（北京）有限公司  A kind of plane recognition methods, device, equipment and the medium of laser point cloud 

2019
 20190705 CN CN201910605510.4A patent/CN110458805A/en active Pending
Patent Citations (8)
Publication number  Priority date  Publication date  Assignee  Title 

KR20130085199A (en) *  20120119  20130729  삼성전자주식회사  Apparatus and method for plane detection 
WO2014020529A1 (en) *  20120802  20140206  Earthmine, Inc.  Threedimensional plane panorama creation through houghbased line detection 
CN107341804A (en) *  20160429  20171110  成都理想境界科技有限公司  Determination method and device, image superimposing method and the equipment of cloud data midplane 
CN107358609A (en) *  20160429  20171117  成都理想境界科技有限公司  A kind of image superimposing method and device for augmented reality 
CN106204705A (en) *  20160705  20161207  长安大学  A kind of 3D point cloud segmentation method based on multiline laser radar 
CN108665472A (en) *  20170401  20181016  华为技术有限公司  The method and apparatus of point cloud segmentation 
US20180364717A1 (en) *  20170614  20181220  Zoox, Inc.  Voxel Based Ground Plane Estimation and Object Segmentation 
CN109359614A (en) *  20181030  20190219  百度在线网络技术（北京）有限公司  A kind of plane recognition methods, device, equipment and the medium of laser point cloud 
NonPatent Citations (2)
Title 

YIGONG ZHANG: "Split and Merge for Accurate Plane Segmentation in RGBD Images", 《2017 4TH IAPR ASIAN CONFERENCE ON PATTERN RECOGNITION》 * 
缪君: "基于稀疏点云的多平面场景稠密重建", 《自动化学报》 * 
Cited By (4)
Publication number  Priority date  Publication date  Assignee  Title 

WO2021103875A1 (en) *  20191125  20210603  歌尔股份有限公司  Laser sensorbased plane detection method and device 
WO2021147113A1 (en) *  20200123  20210729  华为技术有限公司  Plane semantic category identification method and image data processing apparatus 
CN111275724A (en) *  20200226  20200612  武汉大学  Airborne point cloud roof plane segmentation method based on octree and boundary optimization 
CN113313701A (en) *  20210610  20210827  兰州智悦信息科技有限公司  Electric vehicle charging port twostage visual detection positioning method based on shape prior 
Similar Documents
Publication  Publication Date  Title 

CN110458805A (en)  A kind of plane monitoringnetwork method calculates equipment and circuit system  
US10719759B2 (en)  System for building a map and subsequent localization  
US10121076B2 (en)  Recognizing entity interactions in visual media  
CN110168477A (en)  Deep learning system for cuboid detection  
Civera et al.  Towards semantic SLAM using a monocular camera  
CN108230337B (en)  Semantic SLAM system implementation method based on mobile terminal  
CN205721777U (en)  Visual search equipment and system  
Mei et al.  Closing loops without places  
CN106534616B (en)  A kind of video image stabilization method and system based on characteristic matching and motion compensation  
CN107871106A (en)  Method for detecting human face and device  
CN109816769A (en)  Scene based on depth camera ground drawing generating method, device and equipment  
CN107369183A (en)  Towards the MAR Tracing Registration method and system based on figure optimization SLAM  
CN107924452A (en)  Combined shaped for face's alignment in image returns  
CN106934351A (en)  Gesture identification method, device and electronic equipment  
CN109215131A (en)  The driving method and device of conjecture face  
CN110399809A (en)  The face critical point detection method and device of multiple features fusion  
Li et al.  Object Detection in the Context of Mobile Augmented Reality  
CN109242961A (en)  A kind of face modeling method, apparatus, electronic equipment and computerreadable medium  
CN111541943B (en)  Video processing method, video operation method, device, storage medium and equipment  
CN110517319A (en)  A kind of method and relevant apparatus that camera posture information is determining  
CN109670517A (en)  Object detection method, device, electronic equipment and target detection model  
CN110533773A (en)  A kind of threedimensional facial reconstruction method, device and relevant device  
Rashmi et al.  Skeleton based Human Action Recognition for Smart City Application using Deep Learning  
Xie et al.  Toward 3D Object Reconstruction from Stereo Images  
CN112862874B (en)  Point cloud data matching method and device, electronic equipment and computer storage medium 
Legal Events
Date  Code  Title  Description 

PB01  Publication  
PB01  Publication  
SE01  Entry into force of request for substantive examination  
SE01  Entry into force of request for substantive examination 