WO2023066143A1 - 全景图像的图像分割方法、装置、计算机设备和存储介质 - Google Patents

全景图像的图像分割方法、装置、计算机设备和存储介质 Download PDF

Info

Publication number
WO2023066143A1
WO2023066143A1 PCT/CN2022/125243 CN2022125243W WO2023066143A1 WO 2023066143 A1 WO2023066143 A1 WO 2023066143A1 CN 2022125243 W CN2022125243 W CN 2022125243W WO 2023066143 A1 WO2023066143 A1 WO 2023066143A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
panoramic image
panoramic
field
boundary
Prior art date
Application number
PCT/CN2022/125243
Other languages
English (en)
French (fr)
Inventor
林晓帆
姜文杰
Original Assignee
影石创新科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 影石创新科技股份有限公司 filed Critical 影石创新科技股份有限公司
Publication of WO2023066143A1 publication Critical patent/WO2023066143A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/60—Rotation of whole images or parts thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • the present application relates to the field of computers, in particular to an image segmentation method, device, computer equipment and storage medium for a panoramic image.
  • Instance segmentation is a kind of computer vision. Instance segmentation is to detect all the pixels where the target is located on the basis of target detection. That is, it requires outputting the region where the target (that is, the instance) is located and all the pixels belonging to the target in the region at the same time.
  • a panorama image is a special image with an aspect ratio of 2:1 and is composed of multiple images. It follows the latitude and longitude expansion method, the width of the image is the latitude 0-2 ⁇ , and the height of the image is the longitude 0- ⁇ . Therefore, it can record all information of 360 degrees horizontally and 180 degrees pitched.
  • the instance segmentation is generally performed on the plane expansion image of the panoramic image, so some objects in the panoramic image will be distorted, and the rectangular frame of the detection result cannot reasonably frame the deformed and extended objects. target, resulting in deviations in the detection results.
  • the current instance segmentation algorithm generally uses the pre-selected rectangular box (Bounding-Box) of the target to perform instance segmentation.
  • a convolutional neural network CNN
  • this detection method is more suitable for ordinary planar images without distortion.
  • panoramic images due to the existence of panoramic distortion, the above detection method cannot achieve accurate instance segmentation in panoramic images.
  • An image segmentation method for a panoramic image comprising:
  • a local image segmentation result corresponding to the local image features is obtained based on a result of the pooling processing
  • the panoramic image segmentation result corresponding to the panoramic image is acquired.
  • the image features include non-boundary area image features and boundary area image features
  • Said extracting the image features of said panorama image comprises:
  • the image features of the boundary area of the panoramic image are extracted through a preset target deformation adaptation operator.
  • the detection target includes a non-boundary position target
  • a field angle frame corresponding to the non-boundary position object in the panoramic image is identified.
  • the detection target includes a boundary position target
  • the initial field of view frame corresponding to the first detection target and the second detection target is obtained, and the viewing angle
  • the field corner frame includes the area outside the boundary range of the panoramic image
  • the field of view frame corresponding to the boundary position object in the panoramic image is acquired.
  • obtaining the field of view frame corresponding to the boundary position target in the panoramic image includes:
  • the initial field of view frame of any boundary in the panoramic image is mapped to the corresponding boundary of the boundary where the initial field of view frame is located;
  • the initial field of view frame in the corresponding boundary is filtered by a non-maximum value filtering algorithm, and the field of view frame corresponding to the boundary position object in the panoramic image is obtained.
  • the local image segmentation result corresponding to the local image feature after obtaining the panoramic image segmentation result corresponding to the panoramic image based on the result of pooling processing, it further includes:
  • An image segmentation device for a panoramic image comprising:
  • a data acquisition module configured to acquire a panoramic image and extract image features of the panoramic image
  • the field of view processing module is used to identify the field of view frame corresponding to the detection target in the panoramic image based on the image features;
  • a branch detection module configured to acquire local image features corresponding to the field of view frame based on the image features
  • An area processing module configured to perform pooling processing on the local image features through a spherical projection-based pooling operator, and obtain a local image segmentation result corresponding to the local image features based on the result of the pooling processing;
  • the result obtaining module is configured to obtain the panoramic image segmentation result corresponding to the panoramic image according to the local image segmentation result corresponding to the local image feature.
  • the image features include non-boundary area image features and boundary area image features
  • the data acquisition module is specifically configured to: extract the non-boundary area image of the panoramic image through a preset conventional convolution operator Features: extracting the image features of the boundary area of the panoramic image through a preset target deformation adaptation operator.
  • a computer device comprising a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:
  • a local image segmentation result corresponding to the local image features is obtained based on a result of the pooling processing
  • the panoramic image segmentation result corresponding to the panoramic image is acquired.
  • a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:
  • a local image segmentation result corresponding to the local image features is obtained based on a result of the pooling processing
  • the panoramic image segmentation result corresponding to the panoramic image is acquired.
  • the image segmentation method, device, computer equipment and storage medium of the above-mentioned panoramic image extract the image features of the panoramic image by acquiring the panoramic image; The local image features corresponding to the field of view frame; after the local image features are pooled by the spherical projection-based pooling operator, the local image segmentation results corresponding to the local image features are obtained based on the results of the pooling processing; according to the local image The partial image segmentation result corresponding to the feature, and the panoramic image segmentation result corresponding to the panoramic image is obtained.
  • the present application extracts local image features through the field of view frame, and processes the panoramic image based on the field of view frame defined on the spherical surface, which can obtain better feature extraction effects.
  • the pooling processing operator of the spherical projection After the pooling processing operator of the spherical projection performs pooling processing on the local image features, it can effectively perform pooling processing on the local image features in the form of the field of view frame to obtain the local image segmentation results corresponding to the local image features, and then obtain the final The result of instance segmentation ensures the accuracy of instance segmentation.
  • Fig. 1 is the application environment diagram of the image segmentation method of panoramic image in an embodiment
  • Fig. 2 is a schematic flow chart of an image segmentation method for a panoramic image in an embodiment
  • Fig. 3 is a schematic diagram of a rectangular border in panoramic image target detection in an embodiment
  • Fig. 4 is a schematic diagram of the field of view frame in panoramic image target detection in one embodiment
  • Fig. 5 is a schematic subflow diagram of step 203 in Fig. 2 in one embodiment
  • Fig. 6 is a schematic subflow diagram of step 308 in Fig. 3 in one embodiment
  • Fig. 7 is a schematic flow chart of the step of updating the results of panoramic image segmentation in one embodiment
  • Fig. 8 is a structural block diagram of an image segmentation device for a panoramic image in an embodiment
  • Figure 9 is an internal block diagram of a computer device in one embodiment.
  • Panoramic distortion refers to that during the scanning and imaging process of the panoramic image, since the image distance remains unchanged, the object distance increases with the increase of the scanning angle, thus Causes the scale on the image to gradually shrink from the center to both sides.
  • Most of the existing instance segmentation algorithms for panoramic images use the Bounding-Box (BBox) of the target.
  • BBox Bounding-Box
  • the detected rectangular frame cannot reasonably frame the deformed and extended detection target in the image segmentation method using BBox, resulting in poor instance segmentation effect.
  • the image segmentation method for a panoramic image provided in this application can be applied to the application environment shown in FIG. 1 .
  • the terminal 102 communicates with the server 104 through the network.
  • the panoramic image can be sent to the server 104, and the server 104 performs instance segmentation on the panoramic image submitted by the terminal 102.
  • the server 104 obtains the panoramic image, extracts the image features of the panoramic image; based on the image features, recognizes the field of view frame corresponding to the detection target in the panoramic image; based on the image features, obtains the local image features corresponding to the field of view frame; After the pooling processing operator performs pooling processing on the local image features, the local image segmentation results corresponding to the local image features are obtained based on the results of the pooling processing; according to the local image segmentation results corresponding to the local image features, the panoramic images corresponding to the panoramic image are obtained Split results.
  • the terminal 102 can be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices, and the server 104 can be realized by an independent server or a server cluster composed of multiple servers.
  • a method for image segmentation of a panoramic image is provided.
  • the method is applied to the server 104 in FIG. 1 as an example for illustration, including the following steps:
  • Step 201 acquire a panoramic image, and extract image features of the panoramic image.
  • the panorama image is a special image, generally with an aspect ratio of 2:1, and is formed by splicing multiple images. It follows the latitude and longitude expansion method, the width of the image is the latitude 0-2 ⁇ , and the height of the image is the longitude 0- ⁇ . Therefore, it can record all information of 360 degrees horizontally and 180 degrees pitched.
  • the instance segmentation of the panoramic image is performed on the panoramic image, since some objects in the panoramic image will be segmented to the left and right sides of the image horizontally, it cannot be detected as the same object.
  • the detection method for instance segmentation cannot effectively frame the detection target, thus affecting the accuracy of instance segmentation of panoramic images.
  • Image features mainly include image color features, texture features, shape features and spatial relationship features.
  • the color feature is a global feature, which describes the surface properties of the scene corresponding to the image or image region
  • the texture feature is also a global feature, which also describes the surface properties of the scene corresponding to the image or image region
  • shape features are Two types of representation methods, one is the contour feature, the other is the regional feature, the contour feature of the image is mainly aimed at the outer boundary of the object, and the regional feature of the image is related to the entire shape area
  • the spatial relationship feature refers to the segmentation in the image
  • the mutual spatial position or relative direction relationship between the multiple targets, these relationships can also be divided into connection/adjacency relationship, overlapping/overlapping relationship and containment/containment relationship, etc.
  • the present application may use a pre-built convolutional neural network to extract image features of the panoramic image, thereby realizing instance segmentation of the panoramic image.
  • the convolutional neural network specifically includes a backbone network, a detection branch, and a segmentation branch.
  • the input panoramic image first extracts features through the backbone network, and then performs instance segmentation through a detection branch and a segmentation branch respectively.
  • the convolutional neural network of this embodiment can specifically be realized by transforming the convolutional networks of plane graph instances such as Mask R-CNN and CascadeMask R-CNN.
  • the panoramic image can be submitted to the server 104 through the terminal 102, so that the instance segmentation corresponding to the panoramic image can be performed through the server 104, and the type of the detection target in the panoramic image and the detection target can be determined.
  • the server 104 receives the panoramic image. That is, the panoramic image can be convolutionally processed through the preset convolutional neural network. First, the image features of the panoramic image can be extracted through the backbone network in the convolutional neural network. Subsequent processing is then performed based on the extracted image features.
  • Step 203 based on the image features, identify the field of view frame corresponding to the detection target in the panoramic image.
  • Step 205 based on the image features, obtain local image features corresponding to the field of view frame.
  • the frame of the field of view is BFoV (Bounding Field-of-View).
  • BFoV regards the panoramic image as a spherical surface, uses the latitude and longitude coordinates of the target to represent its center point, and uses its two viewing angles in the horizontal and vertical directions
  • Field-of-Views indicates the space it occupies.
  • BFoV is specifically defined as ( ⁇ , ⁇ , h, w).
  • ⁇ and ⁇ are the latitude and longitude coordinates of the target on the spherical surface, respectively;
  • h and w represent the two field angles of the target in the horizontal and vertical directions, similar to height and width.
  • the local image feature refers to the image feature corresponding to the frame part of the field of view segmented from the panoramic image.
  • the detection branch is used to extract the rectangular frame corresponding to the detection target in the panoramic image, but in the solution of the present application, the detection branch can be modified to extract the field angle frame of the target.
  • the framed area includes not only the detected distorted target, but also the panoramic image background content around the irregular distorted target. Therefore, when the input is a panoramic image, the rectangular border introduces more background noise information, which affects the effect of the subsequent segmentation branch.
  • the target detection method based on the rectangular frame frames the detection targets on the left and right sides of the panoramic image in the detected rectangular frame (with green planted wall), however, in the left frame, in addition to the detection target, due to the panoramic distortion of the detection target, the background part of the sky is also included.
  • the distortion of the field of view frame to the upper and lower regions can be extended to include the detection target, thereby reducing the framed background content and improving the accuracy of instance segmentation.
  • the panoramic image can be segmented based on the image features to obtain the local image features corresponding to the field of view frame. Specifically, after obtaining the field of view frame corresponding to the detection target in the panoramic image, the image range corresponding to the field of view frame can be further determined, and the image features within the image range can be used as local image features corresponding to the field of view frame.
  • Step 207 After performing pooling processing on the local image features by using a pooling processing operator based on spherical projection, a local image segmentation result corresponding to the local image features is obtained based on the result of the pooling processing.
  • Step 209 according to the partial image segmentation result corresponding to the partial image feature, obtain the panoramic image segmentation result corresponding to the panoramic image.
  • Pooling which is a process of abstracting information in convolutional neural network processing, and is mainly used to reduce the complexity of calculation.
  • the pooling operator designed based on spherical projection can be used instead of the original convolution
  • the RoI Align/RoI Pooling operator used for rectangular frame detection in the neural network enables the convolutional neural network to process the detection target within the frame of the field of view.
  • the segmentation results of the local image corresponding to each local image feature can be processed through the segmentation branch of the convolutional neural network, so as to obtain each field of view angle
  • the instance segmentation result corresponding to the border the instance segmentation result specifically includes the position of the field angle border and the classification result of the detection target within the field angle border. Then, by synthesizing the instance segmentation results corresponding to each field of view frame, a complete instance segmentation result corresponding to the panoramic image can be obtained.
  • the image segmentation method of the above-mentioned panoramic image by obtaining the panoramic image, extracts the image features of the panoramic image; based on the image features, identifies the field of view frame corresponding to the detection target in the panoramic image; based on the image features, obtains the local image corresponding to the field of view frame Features; After pooling the local image features through the pooling operator based on spherical projection, the local image segmentation results corresponding to the local image features are obtained; according to the local image segmentation results corresponding to the local image features, the panorama corresponding to the panoramic image is obtained Image segmentation results.
  • the present application When detecting panoramic images, the present application extracts local image features through the field of view frame, and processes the panoramic image based on the field of view frame defined on the spherical surface, which can obtain better feature extraction effects.
  • the pooling processing operator of the spherical projection After the pooling processing operator of the spherical projection performs pooling processing on the local image features, it can effectively perform pooling processing on the local image features in the form of the field of view frame to obtain the local image segmentation results corresponding to the local image features, and then obtain the final The result of instance segmentation ensures the accuracy of instance segmentation.
  • the image features include non-boundary area image features and boundary area image features;
  • Step 201 includes: extracting the non-boundary area image features of the panoramic image through a preset conventional convolution operator; The image features of the boundary area of the panoramic image are sub-extracted.
  • the operator is the basic unit of neural network calculation, and the convolution operation is an operation often used in image processing. It has the function of enhancing the original signal characteristics and reducing noise.
  • the preset target deformation adaptive convolution operator means that the application modifies the existing convolutional neural network model for instance segmentation, and replaces some conventional convolution operators with convolution operators that can adapt to the target deformation. , such as deformable convolution, equirectangular projection convolution, and spherical convolution and other types of convolution operators, these preset target deformation adaptation convolution operators are obtained by using panoramic images for training.
  • the panoramic image can be submitted to the server 104 through the terminal 102, so that the instance segmentation corresponding to the panoramic image can be performed through the server 104, and the type of the detection target in the panoramic image and the detection target can be determined.
  • the server 104 receives the panoramic image.
  • the panoramic image can be convoluted through a convolutional neural network including a preset target deformation adaptive convolution operator.
  • this application constructs a convolution model more suitable for panoramic images by replacing some traditional convolution operators with preset target deformation adaptive convolution operators.
  • the target deformation adaptation convolution operator By presetting the target deformation adaptation convolution operator, it has better adaptability to the deformation of the detection target at the boundary of the panoramic image.
  • the boundary part of the panoramic image is convoluted through the preset target deformation adaptive convolution operator to obtain the corresponding image features of the boundary area.
  • targets at non-boundary positions they can be detected by other conventional target detection convolution operators of convolutional neural networks.
  • the image features of the panoramic image are extracted by preset target deformation adaptation operators, which can effectively ensure the accuracy of feature extraction.
  • the detected target includes a non-boundary position target
  • step 203 includes: based on the non-boundary image feature, identifying a field of view frame corresponding to the non-boundary position target in the panoramic image.
  • the non-boundary position target refers to the initial detection target that is not segmented to both ends of the panoramic image, and the non-boundary target is a complete target, generally located in the middle of the panoramic image.
  • the position of the detection target in the panoramic image can be determined based on the image features, so as to determine which initial detection targets in the panoramic image belong to non-boundary position targets.
  • the non-boundary image features in the panoramic image can be determined first, and then, based on these non-boundary image features, it can be determined which detection target data in the panoramic image are non-boundary position objects, and then the field of view borders corresponding to these non-boundary position objects can be identified.
  • the process of detecting the border of the field of view can be realized based on the heat map.
  • the heat map of the detection target in the panoramic image, the offset data of the detection target, and the field of view of the detection target are extracted through the convolutional neural network. Among them, the confidence degree of the target at each position is marked in the heat map. First, the detection target with low confidence is filtered out through the heat map, and the position of the detection target is determined according to the offset data of the detection target.
  • Field angle related data to construct the field angle frame corresponding to the detection target.
  • the non-boundary position target can be effectively determined through the non-boundary image feature, and the field angle frame corresponding to the non-boundary position target can be identified to ensure the detection effect of the instance segmentation.
  • the detection target includes a boundary position target; as shown in FIG. 5, step 203 includes:
  • Step 502 based on the image features of the boundary area, identify the boundary position target in the panoramic image.
  • Step 504 Identify the object attributes between the first detection object and the second detection object based on the image features of the boundary area.
  • the first detection object and the second detection object are boundary position objects in relative positions in the panoramic image.
  • Step 506 when the object attribute indicates that the first detection object and the second detection object are the same detection object, obtain the initial field of view frame corresponding to the first detection object and the second detection object, and the field of view frame includes the boundary range of the panoramic image area.
  • Step 508 according to the initial field of view frame corresponding to the first detection object and the second detection object, obtain the field of view frame corresponding to the boundary position object in the panoramic image.
  • the boundary position target refers to the detection target that is divided into left and right ends in the panoramic image, and a complete boundary position target is generally arranged at the left and right ends of the panoramic image.
  • the target attribute is specifically used to judge whether two detection targets in relative positions, that is, whether the first detection target and the second detection target are the same target.
  • the two detection targets in relative positions are the same target, the two detection targets The target properties of are the same.
  • the target attributes of the two detection targets are different.
  • the image features of the boundary regions corresponding to these objects can be extracted by preset object deformation adaptive convolution operators. Based on the extracted features of the panoramic image, it is further determined which targets belong to the detection target, and the target attributes corresponding to the two detection targets at relative positions are identified. For example, the width of an image is latitude 0-2 ⁇ , and the height of the image is a panoramic image of longitude 0- ⁇ .
  • the detection targets at relative positions it specifically refers to the detection targets including the same Y-axis coordinates. For example, if it is recognized that the coordinates of a detection target A include (0,0.5 ⁇ ), it can be determined that the detection target B including the coordinates (2 ⁇ ,0.5 ⁇ ) is the boundary position target of the relative position of the detection target A.
  • the field of view frame corresponding to the boundary position target in the panoramic image can be obtained according to the initial field of view frame corresponding to the first detection target and the second detection target . Since the targets at both ends are the same target, it is necessary to remove the initial field of view frame corresponding to one of the boundary position targets.
  • the initial field of view frame corresponding to another boundary position target is used as the final field of view frame.
  • the field angle frame corresponding to the boundary position object can be effectively detected to ensure the detection effect of the instance segmentation.
  • step 308 includes:
  • Step 601 according to the position of the initial field of view frame, map the initial field of view frame of any boundary in the panoramic image to the corresponding boundary of the boundary where the initial field of view frame is located.
  • Step 603 Filter the initial field of view frame in the corresponding boundary through a non-maximum value filtering algorithm, and obtain the field of view frame corresponding to the boundary position target in the panoramic image.
  • non-maximum value filtering algorithm is also called non-maximum value suppression.
  • non-maximum value suppression it is to suppress elements that are not maximum values, which can be understood as a local maximum search.
  • the initial field-of-view bounding box on one boundary can be completely mapped to the other boundary. You can map all the initial field of view borders on the left border to the right border, or map all the initial field of view borders on the right border to the left border, so that the initial field of view borders of the same detection target will overlap.
  • the initial field of view frame in the corresponding boundary is filtered through the non-maximum value filtering algorithm, so that repeated frames can be filtered out, and the field of view frame corresponding to the boundary position target in the panoramic image can be obtained directly.
  • the repeated field of view frames corresponding to the same detection are filtered out through non-maximum value filtering, which can ensure the accuracy of instance segmentation.
  • step 209 it also includes:
  • Step 702 acquiring a rotated panoramic image corresponding to the panoramic image.
  • Step 704 extracting a rotation image segmentation result corresponding to the rotation panorama image.
  • Step 706 update the panoramic image segmentation result according to the rotation image segmentation result.
  • the rotated panoramic image refers to a panoramic image obtained by forward-projecting the panoramic image back to a spherical surface, then rotating the spherical surface, and then back-projecting.
  • the results of panoramic image segmentation can also be updated by superposition of results.
  • the original panoramic image is orthographically projected back to the spherical surface, and after rotation, the rotated panoramic image is obtained by inverse projection. Input this rotated panorama image into the model again to obtain a new output result, which is the image segmentation result of the rotated image. Then, the result is superimposed with the previous image segmentation results to obtain an average value, so as to update the image segmentation results and obtain a final output result with better accuracy.
  • the image segmentation result can be effectively optimized, thereby improving the accuracy of the image segmentation result.
  • an image segmentation device for a panoramic image including:
  • the data acquisition module 801 is configured to acquire a panoramic image and extract image features of the panoramic image.
  • the viewing angle processing module 803 is configured to identify the viewing angle frame corresponding to the detection target in the panoramic image based on the image features.
  • the branch detection module 805 is configured to acquire local image features corresponding to the field of view frame based on the image features.
  • the region processing module 807 is configured to obtain a local image segmentation result corresponding to the local image feature based on the result of the pooling processing after performing pooling processing on the local image features through a spherical projection-based pooling processing operator.
  • the result obtaining module 809 is configured to obtain the panoramic image segmentation result corresponding to the panoramic image according to the local image segmentation result corresponding to the local image feature.
  • the data acquisition module 801 is specifically configured to: extract the image features of the non-boundary area of the panoramic image through a preset conventional convolution operator; extract the image features of the border area of the panoramic image through a preset target deformation adaptation operator.
  • the detected target includes a non-boundary position target; the field of view processing module 803 is specifically configured to: identify the field of view frame corresponding to the non-boundary position target in the panoramic image based on the non-boundary image feature.
  • the detection target includes a boundary position target
  • the field of view processing module 803 is specifically configured to: identify the boundary position target in the panoramic image based on the boundary area image features; identify the first detection target and the second detection target based on the boundary area image features
  • Two target attributes between detection targets, the first detection target and the second detection target are boundary position targets in relative positions in the panoramic image; when the target attribute indicates that the first detection target and the second detection target are the same detection target, the acquisition
  • the initial field of view frame corresponding to the first detection target and the second detection target, and the field of view frame includes the area outside the boundary range of the panoramic image; according to the initial field of view frame corresponding to the first detection target and the second detection target, the panoramic image is acquired The field of view frame corresponding to the target at the middle boundary position.
  • the field of view processing module 803 is further configured to: according to the position of the initial field of view frame, map the initial field of view frame of any boundary in the panoramic image to the corresponding border where the initial field of view frame is located. Boundary: Filter the initial field of view frame in the corresponding boundary through a non-maximum value filtering algorithm to obtain the field of view frame corresponding to the boundary position target in the panoramic image.
  • it also includes a detection result update module, configured to: acquire a rotated panoramic image corresponding to the panoramic image; extract a rotated image segmentation result corresponding to the rotated panoramic image; update the panoramic image segmentation result according to the rotated image segmentation result.
  • a detection result update module configured to: acquire a rotated panoramic image corresponding to the panoramic image; extract a rotated image segmentation result corresponding to the rotated panoramic image; update the panoramic image segmentation result according to the rotated image segmentation result.
  • Each module in the above-mentioned device for image segmentation of a panoramic image may be fully or partially realized by software, hardware or a combination thereof.
  • the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, and can also be stored in the memory of the computer device in the form of software, so that the processor can invoke and execute the corresponding operations of the above-mentioned modules.
  • a computer device is provided.
  • the computer device may be a server, and its internal structure may be as shown in FIG. 9 .
  • the computer device includes a processor, memory and a network interface connected by a system bus. Wherein, the processor of the computer device is used to provide calculation and control capabilities.
  • the memory of the computer device includes a non-volatile storage medium and an internal memory.
  • the non-volatile storage medium stores an operating system, computer programs and databases.
  • the internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium.
  • the database of the computer device is used to store traffic forwarding data.
  • the network interface of the computer device is used to communicate with an external terminal via a network connection.
  • FIG. 9 is only a block diagram of a part of the structure related to the solution of this application, and does not constitute a limitation on the computer equipment on which the solution of this application is applied.
  • the specific computer equipment can be More or fewer components than shown in the figures may be included, or some components may be combined, or have a different arrangement of components.
  • a computer device including a memory and a processor, a computer program is stored in the memory, and the processor implements the following steps when executing the computer program:
  • the local image segmentation results corresponding to the local image features are obtained;
  • the panoramic image segmentation result corresponding to the panoramic image is obtained based on the pooling processing result.
  • the following steps are also implemented when the processor executes the computer program: extracting the image features of the non-boundary region of the panoramic image through a preset conventional convolution operator; extracting the boundary region image of the panoramic image through a preset target deformation adaptation operator feature.
  • identifying the field angle frame corresponding to the detection target in the panoramic image includes: based on the non-boundary image features, identifying the non-boundary position target corresponding to the panoramic image field of view border.
  • the processor executes the computer program, the following steps are also implemented: based on the image features of the boundary area, identifying the boundary position target in the panoramic image; based on the image feature of the boundary area, identifying the target between the first detection target and the second detection target attribute, the first detection target and the second detection target are boundary position targets in relative positions in the panoramic image; when the target attribute indicates that the first detection target and the second detection target are the same detection target, the first detection target and the second detection target are obtained Detect the initial field of view frame corresponding to the target, and the field of view frame includes the area outside the boundary range of the panoramic image; according to the initial field of view frame corresponding to the first detection target and the second detection target, obtain the corresponding field of view of the boundary position target in the panoramic image Field corner border.
  • the following steps are further implemented: according to the position of the initial field of view frame, the initial field of view frame of any boundary in the panoramic image is mapped to the corresponding border where the initial field of view frame is located. Boundary: Filter the initial field of view frame in the corresponding boundary through a non-maximum value filtering algorithm to obtain the field of view frame corresponding to the boundary position target in the panoramic image.
  • the following steps are further implemented: acquiring a rotated panoramic image corresponding to the panoramic image; extracting a rotated image segmentation result corresponding to the rotated panoramic image; updating the panoramic image segmentation result according to the rotated image segmentation result.
  • a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:
  • the local image segmentation results corresponding to the local image features are obtained;
  • the panoramic image segmentation result corresponding to the panoramic image is obtained based on the pooling processing result.
  • the following steps are also implemented: extracting the image features of the non-boundary area of the panoramic image through a preset conventional convolution operator; extracting the boundary area of the panoramic image through a preset target deformation adaptation operator image features.
  • identifying the field angle frame corresponding to the detection target in the panoramic image includes: identifying the non-boundary position target in the panoramic image based on the non-boundary image features Corresponding field of view border.
  • the following steps are further implemented: based on the image features of the boundary area, identifying the boundary position target in the panoramic image; based on the image feature of the boundary area, identifying the distance between the first detection target and the second detection target Target attribute, the first detection target and the second detection target are boundary position targets in relative positions in the panoramic image; when the target attribute indicates that the first detection target and the second detection target are the same detection target, the first detection target and the second detection target are obtained 2.
  • the initial field of view frame corresponding to the detection target includes the area outside the boundary range of the panoramic image; according to the initial field of view frame corresponding to the first detection target and the second detection target, obtain the corresponding position of the boundary position target in the panoramic image Field of view border.
  • the following steps are further implemented: according to the position of the initial field of view frame, the initial field of view frame of any border in the panoramic image is mapped to the border where the initial field of view frame is located.
  • Corresponding boundary filter the initial field of view frame in the corresponding boundary through a non-maximum value filtering algorithm, and obtain the field of view frame corresponding to the boundary position target in the panoramic image.
  • the following steps are further implemented: acquiring a rotating panoramic image corresponding to the panoramic image; extracting a rotating image segmentation result corresponding to the rotating panoramic image; updating the panoramic image segmentation result according to the rotating image segmentation result.
  • any references to memory, storage, database or other media used in the various embodiments provided in the present application may include at least one of non-volatile memory and volatile memory.
  • the non-volatile memory may include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory or optical memory, and the like.
  • Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory.
  • the RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM).

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)
  • Studio Devices (AREA)

Abstract

本申请涉及一种全景图像的图像分割方法、装置、计算机设备和存储介质。其中方法通过获取全景图像，提取全景图像的图像特征；基于图像特征，识别全景图像中检测目标对应的视场角边框；基于图像特征，获取视场角边框对应的局部图像特征；通过基于球面投影的池化处理算子对局部图像特征进行池化处理后，基于池化处理的结果获取局部图像特征对应的局部图像分割结果；根据局部图像特征对应的局部图像分割结果，获取全景图像对应的全景图像分割结果。本申请可以有效对视场角边框形式的局部图像特征进行池化处理，来得到局部图像特征对应的局部图像分割结果，进而得到最终的实例分割结果，保证实例分割的准确性。

Description

全景图像的图像分割方法、装置、计算机设备和存储介质 技术领域
本申请涉及计算机领域，特别是涉及一种全景图像的图像分割方法、装置、计算机设备和存储介质。
背景技术
随着人工智能技术的发展，计算机视觉技术也得到了越来越广泛的应用。计算机视觉是一门研究如何使机器“看”的科学，更进一步的说，就是指用摄影机和电脑代替人眼对目标进行识别、跟踪和测量等机器视觉，并进一步做图形处理，使电脑处理成为更适合人眼观察或传送给仪器检测的图像。作为一个科学学科，计算机视觉研究相关的理论和技术，试图建立能够从图像或者多维数据中获取信息的人工智能系统。实例分割是计算机视觉中的一种，实例分割是在目标检测的基础上，检测出目标所在位置的所有像素。即，它要求同时输出目标(也即实例)所在的区域以及该区域内属于该目标的所有像素点。
全景图像是一种特殊的图像，宽高比一般为2:1，由多张图像拼接而成。它按照经纬展开法，图像的宽就是纬度0-2π，图像的高就是经度0-π。所以，它能记录水平360度，俯仰180度的全部信息。目前，对全景图像进实例分割时，一般是对全景图像的平面展开图像进行实例分割，所以全景图像中部分物体会发生畸变，导致检测结果的矩形框并不能合理地框住发生形变和延展的目标，从而导致检测结果出现偏差。
目前的实例分割算法一般采用目标的预选矩形框(Bounding-Box)来进行实例分割，在实例分割过程中采用卷积神经网络(CNN)构建检测模型，对每个目标的矩形框表示，回归它的矩形框坐标，同时预测它的类别。然而这种检测方法更适合用在没有畸变的普通平面图像，在全景图像中，由于全景畸变的存在，上述检测方式无法实现准确地全景图像中的实例分割。
发明内容
基于此，有必要针对上述技术问题，提供一种准确的全景图像的图像分割方法、装置、计算机设备和存储介质。
一种全景图像的图像分割方法，所述方法包括：
获取全景图像，提取所述全景图像的图像特征；
基于所述图像特征，识别所述全景图像中检测目标对应的视场角边框；
基于所述图像特征，获取所述视场角边框对应的局部图像特征；
通过基于球面投影的池化处理算子对所述局部图像特征进行池化处理后，基于池化处理的结果获取所述局部图像特征对应的局部图像分割结果；
根据所述局部图像特征对应的局部图像分割结果，获取所述全景图像对应的全景图像分割结果。
在其中一个实施例中，所述图像特征包括非边界区域图像特征以及边界区域图像特征；
所述提取所述全景图像的图像特征包括：
通过预设常规卷积算子提取所述全景图像的非边界区域图像特征；
通过预设目标形变适应算子提取所述全景图像的边界区域图像特征。
在其中一个实施例中，所述检测目标包括非边界位置目标；
所述基于所述图像特征，识别所述全景图像中检测目标对应的视场角边框包括：
基于所述非边界图像特征，识别所述全景图像中非边界位置目标对应的视场角边框。
在其中一个实施例中，所述检测目标包括边界位置目标；
所述基于所述图像特征，识别所述全景图像中检测目标对应的视场角边框包括：
基于所述边界区域图像特征，识别所述全景图像中边界位置目标；
基于所述边界区域图像特征识别第一检测目标与第二检测目标之间的目标属性，所述第一检测目标与所述第二检测目标为所述全景图像中处于相对位置的边界位置目标；
当所述目标属性表征所述第一检测目标与所述第二检测目标为同一检测目标时，获取所述第一检测目标与所述第二检测目标对应的初始视场角边框，所述视场角边框包括所述全景图像边界范围外区域；
根据所述第一检测目标与所述第二检测目标对应的初始视场角边框，获取所述全景图像中边界位置目标对应的视场角边框。
在其中一个实施例中，所述根据所述第一检测目标与所述第二检测目标对应的初始视场角边框，获取所述全景图像中边界位置目标对应的视场角边框包括：
根据所述初始视场角边框的位置，将所述全景图像中任意一个边界的初始视场角边框映射至所述初始视场角边框所在边界的对应边界；
通过非极大值过滤算法对所述对应边界中的初始视场角边框进行过滤，获取所述全景图像中边界位置目标对应的视场角边框。
在其中一个实施例中，所述根据所述局部图像特征对应的局部图像分割结果，基于池化处理的结果获取所述全景图像对应的全景图像分割结果之后，还包括：
获取所述全景图像对应的旋转全景图像；
提取所述旋转全景图像对应的旋转图像分割结果；
根据所述旋转图像分割结果更新所述全景图像分割结果。
一种全景图像的图像分割装置，所述装置包括：
数据获取模块，用于获取全景图像，提取所述全景图像的图像特征；
视场角处理模块，用于基于所述图像特征，识别所述全景图像中检测目标对应的视场角边框；
分支检测模块，用于基于所述图像特征，获取所述视场角边框对应的局部图像特征；
区域处理模块，用于通过基于球面投影的池化处理算子对所述局部图像特征进行池化处理后，基于池化处理的结果获取所述局部图像特征对应的局部图像分割结果；
结果获取模块，用于根据所述局部图像特征对应的局部图像分割结果，获取所述全景图像对应的全景图像分割结果。
在其中一个实施例中，所述图像特征包括非边界区域图像特征以及边界区域图像特征，所述数据获取模块具体用于：通过预设常规卷积算子提取所述全景图像的非边界区域图像特征；通过预设目标形变适应算子提取所述全景图像的边界区域图像特征。
一种计算机设备，包括存储器和处理器，所述存储器存储有计算机程序，所述处理器执行所述计算机程序时实现以下步骤：
获取全景图像，提取所述全景图像的图像特征；
基于所述图像特征，识别所述全景图像中检测目标对应的视场角边框；
基于所述图像特征，获取所述视场角边框对应的局部图像特征；
通过基于球面投影的池化处理算子对所述局部图像特征进行池化处理后，基于池化处理的结果获取所述局部图像特征对应的局部图像分割结果；
根据所述局部图像特征对应的局部图像分割结果，获取所述全景图像对应的全景图像分割结果。
一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现以下步骤：
获取全景图像，提取所述全景图像的图像特征；
基于所述图像特征，识别所述全景图像中检测目标对应的视场角边框；
基于所述图像特征，获取所述视场角边框对应的局部图像特征；
通过基于球面投影的池化处理算子对所述局部图像特征进行池化处理后，基于池化处理的结果获取所述局部图像特征对应的局部图像分割结果；
根据所述局部图像特征对应的局部图像分割结果，获取所述全景图像对应的全景图像分割结果。
上述全景图像的图像分割方法、装置、计算机设备和存储介质，通过获取全景图像，提取全景图像的图像特征；基于图像特征，识别全景图像中检测目标对应的视场角边框；基于图像特征，获取视场角边框对应的局部图像特征；通过基于球面投影的池化处理算子对局部图像特征进行池化处理后，基于池化处理的结果获取局部图像特征对应的局部图像分割结果；根据局部图像特征对应的局部图像分割结果，获取全景图像对应的全景图像分割结果。本申请在对全景图像检测时，通过视场角边框来进行局部图像特征的提取，基于定义于球面的视场角边框来对全景图像进行处理，能获得更好的特征提取效果，同时通过基于球面投影的池化处理算子对局部图像特征进行池化处理后，可以有效对视场角边框形式的局部图像特征进行池化处理，来得到局部图像特征对应的局部图像分割结果，进而得到最终的实例分割结果，保证实例分割的准确性。
附图说明
图1为一个实施例中全景图像的图像分割方法的应用环境图；
图2为一个实施例中全景图像的图像分割方法的流程示意图；
图3为一个实施例中全景图像目标检测中的矩形边框示意图；
图4为一个实施例中全景图像目标检测中的视场角边框示意图；
图5为一个实施例中图2中步骤203的子流程示意图；
图6为一个实施例中图3中步骤308的子流程示意图；
图7为一个实施例中更新全景图像分割结果步骤的流程示意图；
图8为一个实施例中全景图像的图像分割装置的结构框图；
图9为一个实施例中计算机设备的内部结构图。
具体实施方式
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。
申请人发现，目前存在的全景图像中一般存在全景畸变的现象，全景畸变是指在全景图像的扫描成像过程中，由于像距保持不变，物距随扫描角度的增大而增大，从而导致图像上从中心到两边比例尺逐渐缩小。现有针对全景图像的实例分割算法大多数都采用目标的Bounding-Box(BBox)。然而在全景图 像中，由于畸变的存在，采用BBox的图像分割方法中，检测出的矩形框并不能合理地框住发生形变和延展的检测目标，从而导致实例分割效果不佳。针对此情况，申请人提出了本申请的图像分割方法。
本申请提供的全景图像的图像分割方法，可以应用于如图1所示的应用环境中。其中，终端102通过网络与服务器104进行通信。其中，当终端102方的数据处理工作人员需要对全景图像中的目标进行实例分割时，可以将全景图像发送至服务器104，由服务器104来对终端102所提交的全景图像进行实例分割。服务器104获取全景图像，提取全景图像的图像特征；基于图像特征，识别全景图像中检测目标对应的视场角边框；基于图像特征，获取视场角边框对应的局部图像特征；通过基于球面投影的池化处理算子对局部图像特征进行池化处理后，基于池化处理的结果获取局部图像特征对应的局部图像分割结果；根据局部图像特征对应的局部图像分割结果，获取全景图像对应的全景图像分割结果。其中，终端102可以但不限于是各种个人计算机、笔记本电脑、智能手机、平板电脑和便携式可穿戴设备，服务器104可以用独立的服务器或者是多个服务器组成的服务器集群来实现。
在一个实施例中，如图2所示，提供了一种全景图像的图像分割方法，以该方法应用于图1中的服务器104为例进行说明，包括以下步骤：
步骤201，获取全景图像，提取全景图像的图像特征。
其中，全景图像是一种特殊的图像，宽高比一般为2:1，由多张图像拼接而成。它按照经纬展开法，图像的宽就是纬度0-2π，图像的高就是经度0-π。所以，它能记录水平360度，俯仰180度的全部信息。目前，对全景图像进行全景图像的实例分割的话，由于全景图像中部分物体会被分割到图像水平方向的左右两边，导致无法将其检测为同一物体，同时因为全景畸变的存在，通过矩形框来进行实例分割的检测方法无法有效地框住检测目标，从而影响全景图像的实例分割的准确率。可以通过本申请的全景图像的图像分割方法来实现针对全景图像精准的实例分割。图像特征主要有图像的颜色特征、纹理特征、形状特征和空间关系特征。其中，颜色特征是一种全局特征，描述了图像或图像区域所对应的景物的表面性质；纹理特征也是一种全局特征，它也描述了图像或图像区域所对应景物的表面性质；形状特征有两类表示方法，一类是轮廓特征，另一类是区域特征，图像的轮廓特征主要针对物体的外边界，而图像的区域特征则关系到整个形状区域；空间关系特征，是指图像中分割出来的多个目标之间的相互的空间位置或相对方向关系，这些关系也可分为连接/邻接关系、交叠/重叠关系和包含/包容关系等。本申请具体可以通过预先构建的卷积神经网络来提取全景图像的图像特征，从而实现全景图像的实例分割。卷积神经网络具体包括了一个主干网络、一个检测分支以及一个分割分支，输入的全景图像先通过主干网络中提取特征，之后分别通过一个检测分支和一个分割分支来进行实例分割。本实施例的卷积神经网络具体可 以通过对Mask R-CNN，CascadeMask R-CNN等平面图实例分割卷积网络进行改造实现。
具体地，当终端102方需要进行全景图像的实例分割时，可以通过终端102向服务器104提交全景图像，以通过服务器104进行全景图像对应的实例分割，确定全景图像内的检测目标的类型以及检测目标的位置。服务器104接收该全景图像。即可通过预设的卷积神经网络对全景图像进行卷积处理，首先可以通过卷积神经网络中的主干网络来提取全景图像的图像特征。而后基于提取的图像特征来进行后续处理。
步骤203，基于图像特征，识别全景图像中检测目标对应的视场角边框。
步骤205，基于图像特征，获取视场角边框对应的局部图像特征。
其中，视场角边框即BFoV(Bounding Field-of-View)，BFoV把全景图像视为一个球面，用目标所在的纬经度坐标表示其中心点，用它水平和竖直方向上的两个视场角(Field-of-Views)表示它所占的空间。BFoV具体定义为(φ,θ,h,w)。φ和θ分别是目标在球面上的纬度和经度坐标；h和w表示目标在水平和竖直方向上的两个视场角，类似于高和宽。而局部图像特征是指从全景图像中分割出的视场角边框部分所对应的图像特征。
具体地，在通过卷积神经网络的主干网络提取全景图像的图像特征后。现有技术中是通过检测分支来提取全景图像中检测目标对应的矩形边框，而本申请的方案中，可以把检测分支修改为提取目标的视场角边框。在全景图像中，矩形框对全景畸变的畸变目标而言，它框入的区域除了检测到的畸变目标之外，还包括不规则的畸变目标周围的全景图像背景内容。因此，在输入是全景图像的情况下矩形边框引入了更多的背景干扰信息，影响了后续分割分支的效果。而使用视场角边框替换掉矩形边框后，如图3所示，基于矩形边框(BBOX)的目标检测方法，在检测的矩形框中框入了全景图像中左右两侧的检测目标(带绿植的墙面)，然而在左边的边框内，除了检测目标外，由于检测目标的全景畸变，还包括了背景部分的天空。而如图4所示，使用视场角边框后，视场角边框对上下区域的畸变能延展开来包含住检测目标，从而减少框入的背景内容，提高实例分割的准确率。而在识别全景图像中检测目标对应的视场角边框后，即可基于图像特征，对全景图像进行分割，得到视场角边框部分所对应的局部图像特征。具体地，当得到全景图像中检测目标对应的视场角边框后，即可进一步确定视场角边框对应图像范围，将图像范围内的图像特征，作为视场角边框对应的局部图像特征。
步骤207，通过基于球面投影的池化处理算子对局部图像特征进行池化处理后，基于池化处理的结果获取局部图像特征对应的局部图像分割结果。
步骤209，根据局部图像特征对应的局部图像分割结果，获取全景图像对应的全景图像分割结果。
其中，池化处理即Pooling，是卷积神经网络处理中对信息抽象的过程，主要用于降低计算的复杂度。
具体地，由于在检测分支时，将矩形边框替换为了基于球面的视场角边框，因此，为了保证后续识别的准确率，可以用基于球面投影设计的池化处理算子，来代替原本卷积神经网络中用于矩形框检测的RoI Align/RoI Pooling算子，使得卷积神经网络可以处理视场角边框内的检测目标。在基于球面投影的池化处理算子对局部图像特征进行池化处理后，可以通过卷积神经网络的分割分支来对各个局部图像特征对应的局部图像分割结果进行处理，从而得到各个视场角边框所对应的实例分割结果，实例分割结果具体包括了视场角边框的位置以及视场角边框内检测目标的分类结果。而后综合各个视场角边框所对应的实例分割结果，即可得到全景图像所对应的完整实例分割结果。
上述全景图像的图像分割方法，通过获取全景图像，提取全景图像的图像特征；基于图像特征，识别全景图像中检测目标对应的视场角边框；基于图像特征，获取视场角边框对应的局部图像特征；通过基于球面投影的池化处理算子对局部图像特征进行池化处理后，获取局部图像特征对应的局部图像分割结果；根据局部图像特征对应的局部图像分割结果，获取全景图像对应的全景图像分割结果。本申请在对全景图像检测时，通过视场角边框来进行局部图像特征的提取，基于定义于球面的视场角边框来对全景图像进行处理，能获得更好的特征提取效果，同时通过基于球面投影的池化处理算子对局部图像特征进行池化处理后，可以有效对视场角边框形式的局部图像特征进行池化处理，来得到局部图像特征对应的局部图像分割结果，进而得到最终的实例分割结果，保证实例分割的准确性。
在其中一个实施例中，图像特征包括非边界区域图像特征以及边界区域图像特征；步骤201包括：通过预设常规卷积算子提取全景图像的非边界区域图像特征；通过预设目标形变适应算子提取全景图像的边界区域图像特征。
其中，算子是神经网络计算的基本单元，而卷积操作是对图像处理时，经常用到的一种操作。它具有增强原信号特征，并且能降低噪音的作用。而预设目标形变适应卷积算子是指本申请通过对现有的实例分割的卷积神经网络模型进行改性，将部分常规的卷积算子替换为能适应目标形变的卷积算子，如可变形卷积、等矩形投影卷积以及球面卷积等类型的卷积算子，这些预设目标形变适应卷积算子通过使用全景图片训练得到。
具体地，当终端102方需要进行全景图像的实例分割时，可以通过终端102向服务器104提交全景图像，以通过服务器104进行全景图像对应的实例分割，确定全景图像内的检测目标的类型以及检测目标的位置。服务器104接收该全景图像。即可通过包含预设目标形变适应卷积算子的卷积神经网络对全景图像进行卷积处理。
具体地，本申请具体通过把部分传统卷积算子替换为预设目标形变适应卷积算子，构建一个更适应全景图像的卷积模型。通过预设目标形变适应卷积算子对全景图像的边界处检测目标的形变有更好的适应能力。通过预设目标形变适应卷积算子来全景图像边界部分进行卷积处理，获取相应的边界区域图像特征。而对于非边界位置的目标，则可以通过卷积神经网络的其他常规目标检测卷积算子来进行检测。本实施例中，通过预设目标形变适应算子来提取全景图像的图像特征，可以有效保证特征提取的准确性。
在其中一个实施例中，检测目标包括非边界位置目标，步骤203包括：基于非边界图像特征，识别全景图像中非边界位置目标对应的视场角边框。
其中，非边界位置目标是指未被分割至全景图像两端的初始检测目标，非边界目标为一个完整的目标，一般位于全景图像的中间位置。可以基于图像特征，确定全景图像中的检测目标所在位置，从而确定全景图像中哪些初始检测目标属于非边界位置目标。
具体地，可以先确定全景图像中的非边界图像特征，而后基于这些非边界图像特征确定全景图像中哪些检测目标数据非边界位置目标，进而识别这些非边界位置目标对应的视场角边框。其中检测视场角边框的过程具体可以基于热力图来实现，通过卷积神经网络提取出全景图像中检测目标的热力图，检测目标的偏移量数据以及检测目标的视场角等相关数据，其中热力图中标注有每个位置上存在目标的置信度，先通过热力图过滤掉低置信度的检测目标，根据检测目标的偏移量数据确定出检测目标的位置，而后根据检测目标的视场角相关数据，构建出检测目标对应的视场角边框。本实施例中，通过非边界图像特征，可以有效确定非边界位置目标，并识别非边界位置目标对应的视场角边框，保证实例分割的检测效果。
在其中一个实施例中，检测目标包括边界位置目标；如图5所示，步骤203包括：
步骤502，基于边界区域图像特征，识别全景图像中边界位置目标。
步骤504，基于边界区域图像特征识别第一检测目标与第二检测目标之间的目标属性，第一检测目标与第二检测目标为全景图像中处于相对位置的边界位置目标。
步骤506，当目标属性表征第一检测目标与第二检测目标为同一检测目标时，获取第一检测目标与第二检测目标对应的初始视场角边框，视场角边框包括全景图像边界范围外区域。
步骤508，根据第一检测目标与第二检测目标对应的初始视场角边框，获取全景图像中边界位置目标对应的视场角边框。
其中，边界位置目标是指全景图像中被分割至左右两端的检测目标，一个完整的边界位置目标一般分列在全景图像的左右两端。通过预设目标形变适应卷积算子对全景图像提取特征，能有效地从全景图 像中提取出边界位置目标对应的边界区域图像特征。目标属性具体用于判断处于相对位置的两个检测目标，即第一检测目标与第二检测目标是否为同一个目标，当处于相对位置的两个检测目标为同一目标时，这两个检测目标的目标属性为相同。而处于相对位置的两个检测目标不为同一目标时，这两个检测目标的目标属性为不同。
具体地，在识别边界位置处的视场角边框时，由于目标已经可能已经被分割在了全景图像中相对的两个边界，产生了目标的形变。因此，此时可以通过预设目标形变适应卷积算子来提取这些目标对应的边界区域图像特征。基于提取出的全景图像特征，来进一步地确定中哪些目标属于检测目标，并识别出处于相对位置处的两个检测目标所对应的目标属性。如对于一个图像的宽为纬度0-2π，图像的高为经度0-π的全景图像。可以在图像的左下端点为原点、以图像的宽度方向为X轴，以图像的高度方向为Y轴，建立二维平面坐标系。则该全景图像中边界位置为X＝0的左边界以及X＝2π的右边界。而对于处于相对位置的检测目标，具体是指包含相同Y轴坐标的检测目标。如识别出一个检测目标A的坐标包括(0,0.5π)，从而可以确定包含坐标(2π,0.5π)的检测目标B是检测目标A相对位置的边界位置目标。而后即可基于卷积神经网络提取出的边界区域图像特征来进一步地识别判断，确定两个处于相对位置的检测目标是否相同的检测目标。而在当目标属性表征处于相对位置的检测目标为同一目标时，即可根据第一检测目标与第二检测目标对应的初始视场角边框，获取全景图像中边界位置目标对应的视场角边框。由于两端的目标为同一个目标，需要去除其中一个边界位置目标对应的初始视场角边框。而将另外一个边界位置目标对应的初始视场角边框作为最终的视场角边框。本实施例中，通过边界区域图像特征，可以有效对边界位置目标对应的视场角边框进行有效检测，保证实例分割的检测效果。
在其中一个实施例中，如图6所示，步骤308包括：
步骤601，根据初始视场角边框的位置，将全景图像中任意一个边界的初始视场角边框映射至初始视场角边框所在边界的对应边界。
步骤603，通过非极大值过滤算法对对应边界中的初始视场角边框进行过滤，获取全景图像中边界位置目标对应的视场角边框。
其中，非极大值过滤算法又称非极大值抑制，顾名思义就是抑制不是极大值的元素，可以理解为局部最大搜索。
具体地，在初始视场角边框提取完成后，由于处于相对位置的左右两边的边界位置目标都各自带有一个初始视场角边框，而这两个边界位置目标实际上是同一个检测目标，此时为了保证实例分割的准确性，需要排除重复的视场角边框。因此，可以将其中一个边界上的初始视场角边框全部映射到另外一个 边界。可以把左边界的始视场角边框全部映射到右边界，也可以把右边界的始视场角边框全部映射到左边界，如此，同一个检测目标的初始视场角边框就会重叠。而后通过非极大值过滤算法对对应边界中的初始视场角边框进行过滤，即可过滤掉重复的边框，直接获取全景图像中边界位置目标对应的视场角边框。本实施例，通过非极大值过滤来过滤掉相同检测对应的重复视场角边框，可以保证实例分割的准确性。
在其中一个实施例中，如图7所示，步骤209之后，还包括：
步骤702，获取全景图像对应的旋转全景图像。
步骤704，提取旋转全景图像对应的旋转图像分割结果。
步骤706，根据旋转图像分割结果更新全景图像分割结果。
其中，旋转全景图像是指将全景图像正投影回球面，而后对球面进行旋转后，再逆投影得到的全景图像。
具体地，为了提高实例分割的准确率，还可以通过结果叠加来更新全景图像分割结果。首先将原先的全景图像正投影回球面，进行旋转后，再用逆投影得到旋转全景图像。把这张旋转全景图像再次输入模型，获得新的输出结果，即旋转图像的图像分割结果。而后将该结果和之前的图像分割结果叠加求平均值，从而更新图像分割结果，得到精度更好的最终输出结果。本实施例，通过旋转全景图像的再次分割，可以有效对图像分割结果进行优化，从而提高图像分割结果的准确率。
应该理解的是，虽然图2-7的流程图中的各个步骤按照箭头的指示依次显示，但是这些步骤并不是必然按照箭头指示的顺序依次执行。除非本文中有明确的说明，这些步骤的执行并没有严格的顺序限制，这些步骤可以以其它的顺序执行。而且，图2-7中的至少一部分步骤可以包括多个步骤或者多个阶段，这些步骤或者阶段并不必然是在同一时刻执行完成，而是可以在不同的时刻执行，这些步骤或者阶段的执行顺序也不必然是依次进行，而是可以与其它步骤或者其它步骤中的步骤或者阶段的至少一部分轮流或者交替地执行。
在一个实施例中，如图8所示，提供了一种全景图像的图像分割装置，包括：
数据获取模块801，用于获取全景图像，提取全景图像的图像特征。
视场角处理模块803，用于基于图像特征，识别全景图像中检测目标对应的视场角边框。
分支检测模块805，用于基于图像特征，获取视场角边框对应的局部图像特征。
区域处理模块807，用于通过基于球面投影的池化处理算子对局部图像特征进行池化处理后，基于池化处理的结果获取局部图像特征对应的局部图像分割结果。
结果获取模块809，用于根据局部图像特征对应的局部图像分割结果，获取全景图像对应的全景图像分割结果。
在其中一个实施例中，数据获取模块801具体用于：通过预设常规卷积算子提取全景图像的非边界区域图像特征；通过预设目标形变适应算子提取全景图像的边界区域图像特征。
在其中一个实施例中，检测目标包括非边界位置目标；视场角处理模块803具体用于：基于非边界图像特征，识别全景图像中非边界位置目标对应的视场角边框。
在其中一个实施例中，检测目标包括边界位置目标；视场角处理模块803具体用于：基于边界区域图像特征，识别全景图像中边界位置目标；基于边界区域图像特征识别第一检测目标与第二检测目标之间的目标属性，第一检测目标与第二检测目标为全景图像中处于相对位置的边界位置目标；当目标属性表征第一检测目标与第二检测目标为同一检测目标时，获取第一检测目标与第二检测目标对应的初始视场角边框，视场角边框包括全景图像边界范围外区域；根据第一检测目标与第二检测目标对应的初始视场角边框，获取全景图像中边界位置目标对应的视场角边框。
在其中一个实施例中，视场角处理模块803还用于：根据初始视场角边框的位置，将全景图像中任意一个边界的初始视场角边框映射至初始视场角边框所在边界的对应边界；通过非极大值过滤算法对对应边界中的初始视场角边框进行过滤，获取全景图像中边界位置目标对应的视场角边框。
在其中一个实施例中，还包括检测结果更新模块，用于：获取全景图像对应的旋转全景图像；提取旋转全景图像对应的旋转图像分割结果；根据旋转图像分割结果更新全景图像分割结果。
关于全景图像的图像分割装置的具体限定可以参见上文中对于全景图像的图像分割方法的限定，在此不再赘述。上述全景图像的图像分割装置中的各个模块可全部或部分通过软件、硬件及其组合来实现。上述各模块可以硬件形式内嵌于或独立于计算机设备中的处理器中，也可以以软件形式存储于计算机设备中的存储器中，以便于处理器调用执行以上各个模块对应的操作。
在一个实施例中，提供了一种计算机设备，该计算机设备可以是服务器，其内部结构图可以如图9所示。该计算机设备包括通过系统总线连接的处理器、存储器和网络接口。其中，该计算机设备的处理器用于提供计算和控制能力。该计算机设备的存储器包括非易失性存储介质、内存储器。该非易失性存储介质存储有操作系统、计算机程序和数据库。该内存储器为非易失性存储介质中的操作系统和计算机程序的运行提供环境。该计算机设备的数据库用于存储流量转发数据。该计算机设备的网络接口用于与外部的终端通过网络连接通信。该计算机程序被处理器执行时以实现一种全景图像的图像分割方法。
本领域技术人员可以理解，图9中示出的结构，仅仅是与本申请方案相关的部分结构的框图，并不 构成对本申请方案所应用于其上的计算机设备的限定，具体的计算机设备可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。
在一个实施例中，提供了一种计算机设备，包括存储器和处理器，存储器中存储有计算机程序，该处理器执行计算机程序时实现以下步骤：
获取全景图像，提取全景图像的图像特征；
基于图像特征，识别全景图像中检测目标对应的视场角边框；
基于图像特征，获取视场角边框对应的局部图像特征；
通过基于球面投影的池化处理算子对局部图像特征进行池化处理后，获取局部图像特征对应的局部图像分割结果；
根据局部图像特征对应的局部图像分割结果，基于池化处理的结果获取全景图像对应的全景图像分割结果。
在一个实施例中，处理器执行计算机程序时还实现以下步骤：通过预设常规卷积算子提取全景图像的非边界区域图像特征；通过预设目标形变适应算子提取全景图像的边界区域图像特征。
在一个实施例中，处理器执行计算机程序时还实现以下步骤：基于图像特征，识别全景图像中检测目标对应的视场角边框包括：基于非边界图像特征，识别全景图像中非边界位置目标对应的视场角边框。
在一个实施例中，处理器执行计算机程序时还实现以下步骤：基于边界区域图像特征，识别全景图像中边界位置目标；基于边界区域图像特征识别第一检测目标与第二检测目标之间的目标属性，第一检测目标与第二检测目标为全景图像中处于相对位置的边界位置目标；当目标属性表征第一检测目标与第二检测目标为同一检测目标时，获取第一检测目标与第二检测目标对应的初始视场角边框，视场角边框包括全景图像边界范围外区域；根据第一检测目标与第二检测目标对应的初始视场角边框，获取全景图像中边界位置目标对应的视场角边框。
在一个实施例中，处理器执行计算机程序时还实现以下步骤：根据初始视场角边框的位置，将全景图像中任意一个边界的初始视场角边框映射至初始视场角边框所在边界的对应边界；通过非极大值过滤算法对对应边界中的初始视场角边框进行过滤，获取全景图像中边界位置目标对应的视场角边框。
在一个实施例中，处理器执行计算机程序时还实现以下步骤：获取全景图像对应的旋转全景图像；提取旋转全景图像对应的旋转图像分割结果；根据旋转图像分割结果更新全景图像分割结果。
在一个实施例中，提供了一种计算机可读存储介质，其上存储有计算机程序，计算机程序被处理器执行时实现以下步骤：
获取全景图像，提取全景图像的图像特征；
基于图像特征，识别全景图像中检测目标对应的视场角边框；
基于图像特征，获取视场角边框对应的局部图像特征；
通过基于球面投影的池化处理算子对局部图像特征进行池化处理后，获取局部图像特征对应的局部图像分割结果；
根据局部图像特征对应的局部图像分割结果，基于池化处理的结果获取全景图像对应的全景图像分割结果。
在一个实施例中，计算机程序被处理器执行时还实现以下步骤：通过预设常规卷积算子提取全景图像的非边界区域图像特征；通过预设目标形变适应算子提取全景图像的边界区域图像特征。
在一个实施例中，计算机程序被处理器执行时还实现以下步骤：基于图像特征，识别全景图像中检测目标对应的视场角边框包括：基于非边界图像特征，识别全景图像中非边界位置目标对应的视场角边框。
在一个实施例中，计算机程序被处理器执行时还实现以下步骤：基于边界区域图像特征，识别全景图像中边界位置目标；基于边界区域图像特征识别第一检测目标与第二检测目标之间的目标属性，第一检测目标与第二检测目标为全景图像中处于相对位置的边界位置目标；当目标属性表征第一检测目标与第二检测目标为同一检测目标时，获取第一检测目标与第二检测目标对应的初始视场角边框，视场角边框包括全景图像边界范围外区域；根据第一检测目标与第二检测目标对应的初始视场角边框，获取全景图像中边界位置目标对应的视场角边框。
在一个实施例中，计算机程序被处理器执行时还实现以下步骤：根据初始视场角边框的位置，将全景图像中任意一个边界的初始视场角边框映射至初始视场角边框所在边界的对应边界；通过非极大值过滤算法对对应边界中的初始视场角边框进行过滤，获取全景图像中边界位置目标对应的视场角边框。
在一个实施例中，计算机程序被处理器执行时还实现以下步骤：获取全景图像对应的旋转全景图像；提取旋转全景图像对应的旋转图像分割结果；根据旋转图像分割结果更新全景图像分割结果。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，上述的计算机程序可存储于一非易失性计算机可读取存储介质中，该计算机程序在执行时，可包括如上述各方法的实施例的流程。其中，本申请所提供的各实施例中所使用的对存储器、存储、数据库或其它介质的任何引用，均可包括非易失性和易失性存储器中的至少一种。非易失性存储器可包括只读存储器(Read-OnlyMemory，ROM)、磁带、软盘、闪存或光存储器等。易失性存储器 可包括随机存取存储器(RandomAccessMemory，RAM)或外部高速缓冲存储器。作为说明而非局限，RAM可以是多种形式，比如静态随机存取存储器(StaticRandomAccessMemory，SRAM)或动态随机存取存储器(DynamicRandomAccessMemory，DRAM)等。
以上实施例的各技术特征可以进行任意的组合，为使描述简洁，未对上述实施例中的各个技术特征所有可能的组合都进行描述，然而，只要这些技术特征的组合不存在矛盾，都应当认为是本说明书记载的范围。
以上实施例仅表达了本申请的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对发明专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本申请构思的前提下，还可以做出若干变形和改进，这些都属于本申请的保护范围。因此，本申请专利的保护范围应以所附权利要求为准。

Claims (10)

  1. 一种全景图像的图像分割方法，所述方法包括：
    获取全景图像，提取所述全景图像的图像特征；
    基于所述图像特征，识别所述全景图像中检测目标对应的视场角边框；
    基于所述图像特征，获取所述视场角边框对应的局部图像特征；
    通过基于球面投影的池化处理算子对所述局部图像特征进行池化处理后，基于池化处理的结果获取所述局部图像特征对应的局部图像分割结果；
    根据所述局部图像特征对应的局部图像分割结果，获取所述全景图像对应的全景图像分割结果。
  2. 根据权利要求1所述的方法，其特征在于，所述图像特征包括非边界区域图像特征以及边界区域图像特征；
    所述提取所述全景图像的图像特征包括：
    通过预设常规卷积算子提取所述全景图像的非边界区域图像特征；
    通过预设目标形变适应算子提取所述全景图像的边界区域图像特征。
  3. 根据权利要求2所述的方法，其特征在于，所述检测目标包括非边界位置目标；
    所述基于所述图像特征，识别所述全景图像中检测目标对应的视场角边框包括：
    基于所述非边界图像特征，识别所述全景图像中非边界位置目标对应的视场角边框。
  4. 根据权利要求2所述的方法，其特征在于，所述检测目标包括边界位置目标；
    所述基于所述图像特征，识别所述全景图像中检测目标对应的视场角边框包括：
    基于所述边界区域图像特征，识别所述全景图像中边界位置目标；
    基于所述边界区域图像特征识别第一检测目标与第二检测目标之间的目标属性，所述第一检测目标与所述第二检测目标为所述全景图像中处于相对位置的边界位置目标；
    当所述目标属性表征所述第一检测目标与所述第二检测目标为同一检测目标时，获取所述第一检测目标与所述第二检测目标对应的初始视场角边框，所述视场角边框包括所述全景图像边界范围外区域；
    根据所述第一检测目标与所述第二检测目标对应的初始视场角边框，获取所述全景图像中边界位置目标对应的视场角边框。
  5. 根据权利要求4所述的方法，其特征在于，所述根据所述第一检测目标与所述第二检测目标对应的初始视场角边框，获取所述全景图像中边界位置目标对应的视场角边框包括：
    根据所述初始视场角边框的位置，将所述全景图像中任意一个边界的初始视场角边框映射至所述初始视场角边框所在边界的对应边界；
    通过非极大值过滤算法对所述对应边界中的初始视场角边框进行过滤，获取所述全景图像中边界位置目标对应的视场角边框。
  6. 根据权利要求1所述的方法，其特征在于，所述根据所述局部图像特征对应的局部图像分割结果，基于池化处理的结果获取所述全景图像对应的全景图像分割结果之后，还包括：
    获取所述全景图像对应的旋转全景图像；
    提取所述旋转全景图像对应的旋转图像分割结果；
    根据所述旋转图像分割结果更新所述全景图像分割结果。
  7. 一种全景图像的图像分割装置，其特征在于，所述装置包括：
    数据获取模块，用于获取全景图像，提取所述全景图像的图像特征；
    视场角处理模块，用于基于所述图像特征，识别所述全景图像中检测目标对应的视场角边框；
    分支检测模块，用于基于所述图像特征，获取所述视场角边框对应的局部图像特征；
    区域处理模块，用于通过基于球面投影的池化处理算子对所述局部图像特征进行池化处理后，基于池化处理的结果获取所述局部图像特征对应的局部图像分割结果；
    结果获取模块，用于根据所述局部图像特征对应的局部图像分割结果，获取所述全景图像对应的全景图像分割结果。
  8. 根据权利要求7所述的装置，其特征在于，所述图像特征包括非边界区域图像特征以及边界区域图像特征，所述数据获取模块具体用于：通过预设常规卷积算子提取所述全景图像的非边界区域图像特征；通过预设目标形变适应算子提取所述全景图像的边界区域图像特征。
  9. 一种计算机设备，包括存储器和处理器，所述存储器存储有计算机程序，其特征在于，所述处理器执行所述计算机程序时实现权利要求1至6中任一项所述方法的步骤。
  10. 一种计算机可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现权利要求1至6中任一项所述的方法的步骤。
PCT/CN2022/125243 2021-10-22 2022-10-14 全景图像的图像分割方法、装置、计算机设备和存储介质 WO2023066143A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111235009.7 2021-10-22
CN202111235009.7A CN114004839A (zh) 2021-10-22 2021-10-22 全景图像的图像分割方法、装置、计算机设备和存储介质

Publications (1)

Publication Number Publication Date
WO2023066143A1 true WO2023066143A1 (zh) 2023-04-27

Family

ID=79923556

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/125243 WO2023066143A1 (zh) 2021-10-22 2022-10-14 全景图像的图像分割方法、装置、计算机设备和存储介质

Country Status (2)

Country Link
CN (1) CN114004839A (zh)
WO (1) WO2023066143A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114004839A (zh) * 2021-10-22 2022-02-01 影石创新科技股份有限公司 全景图像的图像分割方法、装置、计算机设备和存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107844750A (zh) * 2017-10-19 2018-03-27 华中科技大学 一种水面全景图像目标检测识别方法
CN109272442A (zh) * 2018-09-27 2019-01-25 百度在线网络技术（北京）有限公司 全景球面图像的处理方法、装置、设备和存储介质
US20190130204A1 (en) * 2017-10-31 2019-05-02 The University Of Florida Research Foundation, Incorporated Apparatus and method for detecting scene text in an image
CN110163271A (zh) * 2019-05-13 2019-08-23 武汉大学 一种基于球面投影网格和球面卷积的全景影像目标检测方法
CN110503651A (zh) * 2019-08-09 2019-11-26 北京航空航天大学 一种图像显著对象分割方法及装置
CN111524150A (zh) * 2020-07-03 2020-08-11 支付宝(杭州)信息技术有限公司 图像处理的方法及装置
CN114004839A (zh) * 2021-10-22 2022-02-01 影石创新科技股份有限公司 全景图像的图像分割方法、装置、计算机设备和存储介质

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107844750A (zh) * 2017-10-19 2018-03-27 华中科技大学 一种水面全景图像目标检测识别方法
US20190130204A1 (en) * 2017-10-31 2019-05-02 The University Of Florida Research Foundation, Incorporated Apparatus and method for detecting scene text in an image
CN109272442A (zh) * 2018-09-27 2019-01-25 百度在线网络技术（北京）有限公司 全景球面图像的处理方法、装置、设备和存储介质
CN110163271A (zh) * 2019-05-13 2019-08-23 武汉大学 一种基于球面投影网格和球面卷积的全景影像目标检测方法
CN110503651A (zh) * 2019-08-09 2019-11-26 北京航空航天大学 一种图像显著对象分割方法及装置
CN111524150A (zh) * 2020-07-03 2020-08-11 支付宝(杭州)信息技术有限公司 图像处理的方法及装置
CN114004839A (zh) * 2021-10-22 2022-02-01 影石创新科技股份有限公司 全景图像的图像分割方法、装置、计算机设备和存储介质

Also Published As

Publication number Publication date
CN114004839A (zh) 2022-02-01

Similar Documents

Publication Publication Date Title
WO2021227360A1 (zh) 一种交互式视频投影方法、装置、设备及存储介质
WO2021227359A1 (zh) 一种无人机投影方法、装置、设备及存储介质
WO2019218824A1 (zh) 一种移动轨迹获取方法及其设备、存储介质、终端
US8126268B2 (en) Edge-guided morphological closing in segmentation of video sequences
US9454796B2 (en) Aligning ground based images and aerial imagery
US8565525B2 (en) Edge comparison in segmentation of video sequences
WO2019192358A1 (zh) 一种全景视频合成方法、装置及电子设备
WO2020134528A1 (zh) 目标检测方法及相关产品
CN111401266B (zh) 绘本角点定位的方法、设备、计算机设备和可读存储介质
US20090028432A1 (en) Segmentation of Video Sequences
US20090016603A1 (en) Contour Finding in Segmentation of Video Sequences
CN107945112A (zh) 一种全景图像拼接方法及装置
CN111915483B (zh) 图像拼接方法、装置、计算机设备和存储介质
TWI639136B (zh) 即時視訊畫面拼接方法
WO2021017589A1 (zh) 一种基于梯度域映射的图像融合方法
CN111553841B (zh) 一种基于最佳缝合线更新的实时视频拼接方法
WO2023066143A1 (zh) 全景图像的图像分割方法、装置、计算机设备和存储介质
JP2019509526A (ja) 多数のカメラを用いた最適の球形映像獲得方法
CN115546027B (zh) 图像缝合线确定方法、装置以及存储介质
WO2021142843A1 (zh) 图像扫描方法及装置、设备、存储介质
CN113436251B (zh) 一种基于改进的yolo6d算法的位姿估计系统及方法
CN112333468B (zh) 图像处理方法、装置、设备及存储介质
Duchowski et al. Using pose estimation to map gaze to detected fiducial markers
WO2023066142A1 (zh) 全景图像的目标检测方法、装置、计算机设备和存储介质
CN114399423B (zh) 图像内容移除方法、系统、介质、设备及数据处理终端

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22882752

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE