CN113838116B

CN113838116B - Method and device for determining target view, electronic equipment and storage medium

Info

Publication number: CN113838116B
Application number: CN202111152595.9A
Authority: CN
Inventors: 王烁; 焦少慧; 李泉
Original assignee: Beijing Youzhuju Network Technology Co Ltd
Current assignee: Beijing Youzhuju Network Technology Co Ltd
Priority date: 2021-09-29
Filing date: 2021-09-29
Publication date: 2023-01-31
Anticipated expiration: 2041-09-29
Also published as: CN113838116A

Abstract

The embodiment of the disclosure provides a method, a device, an electronic device and a storage medium for determining a target view, wherein the method comprises the following steps: determining a target local three-dimensional point cloud corresponding to the target area according to a plurality of images to be processed of each sampling point to be selected in the target area; determining at least one target sampling point from the to-be-selected sampling points and target visual angle information in the target sampling points according to the obtained labeling information in the target local three-dimensional point cloud; processing the target point cloud of each target visual angle information to obtain a target image to be spliced corresponding to each target visual angle information; and obtaining a target panoramic mosaic image corresponding to each to-be-selected sampling site by carrying out mosaic processing on the target to-be-mosaic image and/or the to-be-processed image of each to-be-selected sampling site. The technical scheme provided by the embodiment of the disclosure provides a way for users to mark interference factors such as mirror surfaces, windows and the like in the area, and realizes accurate construction of the panoramic image of the target area.

Description

Method and device for determining target view, electronic equipment and storage medium

Technical Field

The embodiment of the disclosure relates to the technical field of computers, and in particular, to a method and an apparatus for determining a target view, an electronic device, and a storage medium.

Background

At present, in many digital services related to a house, real data of the house needs to be collected and modeled, so as to provide a model or a picture corresponding to the house for a user.

However, in the prior art, when the house data is collected by using the collection device, interference of various external information on the collection device is not considered, so that a three-dimensional schematic diagram most approximate to a real house source cannot be drawn, and thus a displayed image and a real image have a certain difference, and further a user has a poor watching effect.

Disclosure of Invention

The embodiment of the disclosure provides a method and a device for determining a target view, an electronic device and a storage medium, so as to achieve the best matching between a drawn target view and a view of a real scene, and further improve the technical effect of user experience.

In a first aspect, an embodiment of the present disclosure provides a method for determining a target view, where the method includes:

determining a target local three-dimensional point cloud corresponding to a target region according to a plurality of images to be processed of each sampling point to be selected in the target region;

determining at least one target sampling point from the to-be-selected sampling points and target visual angle information corresponding to the marking information in the target sampling points according to the obtained marking information in the target local three-dimensional point cloud;

processing the target point cloud of each target visual angle information to obtain a target image to be spliced corresponding to each target visual angle information;

and obtaining a target panoramic mosaic image corresponding to each to-be-selected sampling site by carrying out mosaic processing on the target to-be-mosaic image and/or the to-be-processed image of each to-be-selected sampling site.

In a second aspect, an embodiment of the present disclosure further provides an apparatus for determining a target view, where the apparatus includes:

the target local three-dimensional point cloud determining module is used for determining a target local three-dimensional point cloud corresponding to a target area according to a plurality of images to be processed of sampling points to be selected in the target area;

the target sampling point determining module is used for determining at least one target sampling point from the to-be-selected sampling points and target visual angle information corresponding to the marking information in the target sampling points according to the obtained marking information in the target local three-dimensional point cloud;

the target image to be spliced determining module is used for processing the target point cloud of each target visual angle information to obtain a target image to be spliced corresponding to each target visual angle information;

and the target panoramic stitched image determining module is used for carrying out stitching processing on the target to-be-stitched images and/or the images to be processed of the to-be-selected sampling sites to obtain target panoramic stitched images corresponding to the to-be-selected sampling sites.

In a third aspect, an embodiment of the present disclosure further provides an electronic device, where the electronic device includes:

one or more processors;

a storage device for storing one or more programs,

when executed by the one or more processors, cause the one or more processors to implement a method of determining a target view as in any of the embodiments of the present disclosure.

In a fourth aspect, the disclosed embodiments also provide a storage medium containing computer-executable instructions, which when executed by a computer processor, are used to perform the method for determining a target view according to any one of the disclosed embodiments.

According to the technical scheme of the embodiment of the disclosure, firstly, a target local three-dimensional point cloud corresponding to a target area is determined according to a plurality of images to be processed of each sampling point to be selected in the target area so as to mark a specific part of the target local three-dimensional point cloud; determining at least one target sampling point from the to-be-selected sampling points and target visual angle information corresponding to the marking information in the target sampling points according to the obtained marking information in the target local three-dimensional point cloud, so as to determine a visual angle corresponding to the marking part in the sampling points; the target point cloud of each target visual angle information is processed to obtain the target to-be-spliced image corresponding to each target visual angle information, and then the target to-be-spliced image and/or the to-be-processed image of each to-be-selected sampling site are spliced to obtain the target panoramic spliced image corresponding to each to-be-selected sampling site, so that the problem that the influence of external factors on model construction is not considered in the prior art is solved, the constructed target panoramic spliced image is different from the actual panoramic image of a target area to a certain extent, the image displayed on display software is different from the actual image, and the user experience is poor, the automatic construction of the panoramic image of the target area is realized, meanwhile, the influence of interference factors in the area on the accuracy of the constructed panoramic image is avoided, the accuracy of splicing of the panoramic image is improved, and the technical effect of the user experience is improved.

Drawings

The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and features are not necessarily drawn to scale.

Fig. 1 is a flowchart illustrating a method for determining a target view according to a first embodiment of the disclosure;

fig. 2 is a schematic flowchart of a method for determining a target view according to a second embodiment of the present disclosure;

fig. 3 is a schematic diagram of a method for determining a target view according to a second embodiment of the present disclosure;

fig. 4 is a flowchart illustrating a method for determining a target view according to a third embodiment of the present disclosure;

fig. 5 is a schematic diagram of a method for determining a target view according to a third embodiment of the present disclosure;

fig. 6 is a schematic diagram of a method for determining a target view according to a third embodiment of the present disclosure;

fig. 7 is a schematic flowchart of a method for determining a target view according to a fourth embodiment of the present disclosure;

fig. 8 is a block diagram of an apparatus for determining a target view according to a fifth embodiment of the present disclosure;

fig. 9 is a schematic structural diagram of an electronic device according to a sixth embodiment of the present disclosure.

Detailed Description

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.

It should be understood that the various steps recited in the method embodiments of the present disclosure may be performed in a different order, and/or performed in parallel. Moreover, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present disclosure is not limited in this respect.

The term "include" and variations thereof as used herein are open-ended, i.e., "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions for other terms will be given in the following description.

It should be noted that the terms "first", "second", and the like in the present disclosure are only used for distinguishing different devices, modules or units, and are not used for limiting the order or interdependence of the functions performed by the devices, modules or units.

It is noted that references to "a" or "an" in this disclosure are intended to be illustrative rather than limiting, and that those skilled in the art will appreciate that references to "one or more" are intended to be exemplary and not limiting unless the context clearly indicates otherwise.

Example one

Fig. 1 is a schematic flowchart of a method for determining a target view according to an embodiment of the present disclosure, where the embodiment is applicable to a situation where a specific portion in a three-dimensional point cloud is processed in a targeted manner to construct a panoramic view, and the method may be executed by an apparatus for determining a target view, where the apparatus may be implemented in the form of software and/or hardware, and the hardware may be an electronic device, such as a mobile terminal, a PC terminal, or a server.

In order to clearly understand the technical solution of the embodiment of the present invention, an application scenario may be first exemplarily described. The technical scheme can be applied to any scene needing to construct the panoramic view for the target area, for example, in various application programs related to the house sources, the view corresponding to each house source can be displayed, and the view corresponding to each house source can be determined based on the technical scheme disclosed by the invention and displayed in the display interface.

As shown in fig. 1, the method of the present embodiment includes:

s110, determining a target local three-dimensional point cloud corresponding to the target area according to the plurality of images to be processed of the acquisition points to be selected in the target area.

For example, in an application program related to house rental, a real picture of a house needs to be presented to a user on a rental page corresponding to each house, and thus, a spatial area corresponding to the house including at least one room is the target area.

In this embodiment, before constructing the panoramic stitched image for the target area, the to-be-processed image corresponding to the target area may be obtained first, where the to-be-processed image is an image that is collected from the target area and reflects a real picture of the target area, and for the house in the above example, a photo taken for each room in the house may be used as the to-be-processed image. It can be understood that, in order to make the collected real data more complete, the image to be processed may have multiple images, and further, for each acquired image to be processed, a specific identifier may be marked on the image to be processed to distinguish the image to be processed in the subsequent processing process, for example, the image to be processed corresponding to the bedroom in the house is marked with the identifier a, and the image to be processed corresponding to the kitchen in the house is marked with the identifier B. It should be noted that, in order to obtain three-dimensional information corresponding to the target region in the subsequent process, at least the depth information of the target region and the object in the region needs to be included in the multiple images to be processed.

In the present embodiment, the image to be processed is acquired from a sampling point to be selected in the target area, where the sampling point to be selected refers to a plurality of viewpoints for acquiring the image to be processed in the target area, each viewpoint has a specific view range, for example, when the doorway of the bedroom of the house in the above example is used as the sampling point, the sampling point has at least the view range of the inside of the bedroom, and when the midpoint of the kitchen is used as the sampling point, the sampling point has at least the view range of the whole kitchen. It can be understood that after a plurality of image acquisition devices (such as cameras) are erected on each to-be-selected sampling site to perform data acquisition, to-be-processed images corresponding to the visual field range of each to-be-selected sampling site can be obtained. Also, in order to make the acquired real data more complete, a plurality of images to be processed can be acquired based on each sampling point to be selected in the target region.

It should be noted that after obtaining a plurality of images to be processed from the sampling sites to be selected, the images may be uploaded to the system in real time for processing, or the images may be stored in a specific repository to wait for the system to retrieve the images to be processed. It will be understood by those skilled in the art that the specific processing manner may be selected according to actual needs, and the embodiments of the present disclosure are not specifically limited herein.

Furthermore, a target local three-dimensional point cloud can be determined according to a plurality of images to be processed of each to-be-selected sampling point in the target area. It should be understood by those skilled in the art that the Point Cloud refers to a Point Data set of an appearance surface of an object obtained by a measuring instrument in a reverse engineering, and the three-dimensional Point Cloud is a more common and more basic three-dimensional model, which can at least reflect a three-dimensional geometric shape of an area or an object corresponding to the model, and is constructed based on Point Cloud Data (PCD), and the Point Cloud Data comes from a plurality of images to be processed containing depth information. In this embodiment, there may be a plurality of sampling points to be selected in the target region, and a plurality of images to be processed may be obtained at each sampling point to be selected, so that, for each sampling point to be selected, after enough point cloud data (images to be processed) is obtained at the point, a corresponding target three-dimensional point cloud may be established to reflect a certain part of the three-dimensional structure in the target region. Further, a part of the target three-dimensional point cloud is screened out, namely the target local three-dimensional point cloud in the embodiment of the disclosure.

The sampling site to be selected corresponding to the bedroom of the house in the above example is explained, after the point cloud data of the bedroom is acquired at the sampling site, a target three-dimensional point cloud of the bedroom can be constructed, further, the three-dimensional model is divided into three parts according to the region division rule, wherein the three parts are respectively a part containing a window, a part containing a bed and a part containing a wardrobe, and each part in the three-dimensional model can be used as a target local three-dimensional point cloud.

It should be noted that there are various ways to obtain the target three-dimensional point cloud based on multiple images to be processed, for example, a corresponding point cloud fusion program may be written based on a programming language, and the point cloud data is processed by executing the program, so as to generate a target three-dimensional point cloud model corresponding to the sampling site to be selected in the three-dimensional mapping software. Furthermore, the target local three-dimensional point cloud may be screened or partitioned from the target three-dimensional point cloud model in a manual or automatic manner, and a specific screening or partitioning rule may also be formulated according to task needs.

And S120, determining at least one target sampling point from the to-be-selected sampling points and target view angle information corresponding to the marking information in the target sampling points according to the obtained marking information in the target local three-dimensional point cloud.

The labeling information may be information in the form of a symbol or a text in the target local three-dimensional point cloud, such as a line segment added in the three-dimensional point cloud, or a text description associated with a specific position of the three-dimensional point cloud. In this embodiment, the target local three-dimensional point cloud may be labeled manually, or the system may automatically label the target local three-dimensional point cloud according to a preset rule.

It should be noted that the labeling information in the target local three-dimensional point cloud at least indicates that the system needs to perform targeted processing on the part associated with the labeling information in the subsequent process, such as removing or replacing the part associated with the labeling information. Specifically, in the process of constructing a three-dimensional model or a panoramic image for a target area, there are inevitably a plurality of interference factors affecting the construction of the three-dimensional model/panoramic image in an image to be processed serving as a data base, for example, in the house of the above example, there may be a plurality of mirrors and glass windows, and when data acquisition is performed on a room including the mirrors and the glass windows by using an acquisition device, the devices may generate strong reflection and projection on a mirror-like object, so that the generated image to be processed includes wrong depth information. Based on this, labeling information may be printed on the portions corresponding to the mirror surface and the glass window based on the target local three-dimensional point cloud of each room.

In this embodiment, each target local three-dimensional point cloud may correspond to a specific portion in the target area, and a picture of the specific portion is acquired based on the to-be-selected sampling point, so it can be understood that the target local three-dimensional point cloud also corresponds to each to-be-selected sampling point providing the to-be-processed image.

In the practical application process, each target local three-dimensional point cloud does not contain the labeling information, so that the target sampling points can be screened out from the to-be-selected sampling points corresponding to the target local three-dimensional point clouds, namely, only the labeling information exists in the target local three-dimensional point cloud generated based on the to-be-processed image provided by the target sampling points. It should be noted that, in the embodiments of the present disclosure, at least one target sampling point, for example, a corresponding to-be-selected sampling point is arranged for each room in a house, only one target sampling point can be determined when a mirror or a glass window exists in only one room, and correspondingly, when mirrors or glass windows exist in a plurality of rooms, a plurality of target sampling points can be determined.

It should be noted that each room may correspond to a plurality of sampling points to be selected. The number of the sampling points to be selected can be determined according to the shooting view of the camera, and correspondingly, the sampling points capable of shooting a mirror or a glass window can be used as target sampling points.

It should be further noted that, while the target sampling point is determined, the target view angle information corresponding to the labeling information may be determined in the target sampling point, so as to process the point cloud and the pixel point corresponding to the target view angle information, so that the obtained target panoramic image or target point cloud corresponding to each to-be-selected sampling point is most similar to the actual situation of the house. The following details the determination of the target view information for each target sampling point:

when the picture of the current area is acquired based on the target acquisition site, acquiring equipment deployed on the site needs to capture the picture in the current area from multiple visual angles, so that the obtained multiple images to be processed can display the picture of the current area in an all-around manner; meanwhile, the labeling information only exists at a specific position of the target local three-dimensional point cloud or is associated with the specific position, and the labeling information usually only appears in specific view angles under the condition that the target local three-dimensional point cloud corresponds to the pictures of multiple view angles in the current area. Taking the bedroom in the house as an example, the corresponding target local three-dimensional point cloud is created based on the to-be-processed images of the left, middle and right viewing angles, and when the sampling point corresponding to the bedroom is determined to be the target sampling point, the to-be-processed image including the window and the middle part is taken as the target viewing angle information based on the glass window marking information in the point cloud.

S130, processing the target point cloud of each target visual angle information to obtain a target image to be spliced corresponding to each target visual angle information.

In this embodiment, after the target view angle information is determined, the point clouds corresponding to the view angles are selected as the target point clouds from the corresponding target local three-dimensional point clouds, and then the target point clouds are subjected to the targeted processing.

Illustratively, after determining that an image to be processed, which corresponds to a bedroom and includes a mirror, is used as a target view angle, a part corresponding to the image can be determined as a target point cloud in a target local three-dimensional point cloud of the bedroom, and further, the part of the point cloud can be removed, which can be understood as removing a part of the mirror, which affects the construction of a subsequent panoramic image, in the three-dimensional model.

In this embodiment, the above-mentioned targeted processing procedure of the target point cloud may be performed based on a pre-programmed program, and may also be performed by using specific software, for example, setting configuration items in the software according to the target view information, so that the software processes the target point cloud based on specific editing parameters.

Further, in order to obtain a panoramic view of a target area in a subsequent process, a corresponding part needs to be screened out from an image to be processed, and the corresponding part is subjected to the same logic execution targeted processing, so that the processed image is used as a target image to be spliced; or, directly executing conversion operation based on the processed target local three-dimensional point cloud, and further obtaining the target image to be spliced.

The target image to be spliced is a two-dimensional image used for constructing a panoramic image of the current sampling point, and can at least reflect a picture of a part of visual angles in an area corresponding to the current sampling point. Therefore, the process of obtaining the target image to be stitched based on the target local three-dimensional point cloud can be substantially understood as a process of obtaining two-dimensional images under multiple viewing angles based on a three-dimensional model. Continuing with the above example, based on the target local three-dimensional point cloud corresponding to the bedroom from which the label part has been removed, a two-dimensional image lacking a mirror part can be obtained as a target image to be stitched, so as to avoid the influence of the mirror on the finally obtained room panoramic image.

It should be noted that, for a target local three-dimensional point cloud without labeling information, the scheme based on the embodiment of the present disclosure may not determine corresponding target view information, and may directly store an image to be processed corresponding to the three-dimensional point cloud as an image to be stitched. Illustratively, the two local three-dimensional point clouds of the two targets corresponding to the kitchen and the study room are not provided with marking information, which indicates that no mirror or glass window influencing the construction of the panoramic image exists in the two rooms, so that the images to be processed collected at the to-be-selected acquisition sites of the two rooms can be directly used as the images to be spliced and stored at the specific positions of the repository for subsequent processing.

And S140, splicing the target to-be-spliced images and/or the images to be processed of the to-be-selected sampling sites to obtain target panoramic spliced images corresponding to the to-be-selected sampling sites.

In this embodiment, after the images to be stitched corresponding to the sampling points to be selected are obtained, the images need to be stitched, so as to obtain a target panoramic stitched image corresponding to each sampling point to be selected in the target area. It will be appreciated by those skilled in the art that the panoramic image may be based on one image providing the user with multiple views of the target scene, for example, a panoramic image of 360 ° range horizontally may provide the user with all of the views around the point where the image was taken.

From the foregoing description, it can be determined that three situations may occur in the image stitching process, and the following description takes one sampling point in the target region as an example.

The first situation is that labeling information exists in a target local three-dimensional point cloud of a certain to-be-selected sampling point in a target area, and meanwhile, the labeling information relates to all target viewing angles of the sampling point.

The second situation is that labeling information exists in a target local three-dimensional point cloud of a certain to-be-selected sampling point in a target area, and meanwhile, the labeling information only relates to a partial view angle (namely a target view angle) of the sampling point. Further, a target panoramic spliced image reflecting real pictures of multiple visual angles in the current area is spliced on the basis of the target to-be-spliced image (of the target visual angle) and the to-be-processed images (of other visual angles).

And in the third situation, labeling information does not exist in the target local three-dimensional point cloud of a certain to-be-selected sampling point in the target area, according to the scheme of the embodiment of the disclosure, to-be-processed images at all visual angles of the sampling point can be directly selected as to-be-spliced images, and then target panoramic spliced images reflecting real pictures of multiple visual angles in the current area are spliced directly on the basis of the to-be-processed images.

It should be understood by those skilled in the art that the foregoing example only illustrates a certain sampling point in the target area, and in an actual application process, the processing manner for each sampling point to be selected may follow the scheme of the embodiment of the present disclosure, so as to obtain a target panoramic stitched image corresponding to each sampling point to be selected. Based on the target panoramic spliced images with the interference factors eliminated, a user can select a specific area in the target area and select a plurality of visual angles in the specific area, so that the real picture of the target area can be completely and accurately known.

Example two

Fig. 2 is a schematic flow chart of a method for determining a target view according to a second embodiment of the present disclosure, in which at least three depth cameras on a rotation axis are used to capture images at each to-be-selected capture point on the basis of the second embodiment, so that to-be-processed images of each scene can be efficiently and comprehensively acquired; determining the common-view information of any two single-point cloud models, so that the same parts in the two models can be removed conveniently; according to the target global point cloud, the top view point cloud part is determined to be used as the target local three-dimensional point cloud, so that the situation that image information is disordered when the labeling information is excessive is avoided, and the manual labeling operation of a user in the point cloud is conveniently executed. The specific implementation manner can be referred to the technical scheme of the embodiment. The technical terms that are the same as or corresponding to the above embodiments are not repeated herein.

As shown in fig. 2, the method specifically includes the following steps:

s210, determining each sampling site to be selected in the target area; and acquiring a plurality of images to be processed shot at the current sampling site to be selected based on at least three depth cameras aiming at the sampling site to be selected.

In this embodiment, the sampling point to be selected in the target region may be determined according to task needs, or the sampling point to be selected may be determined based on a result of automatic selection by the system. For example, when a worker enters a house to take a picture of each room according to task requirements, in order to efficiently and comprehensively obtain a real picture of a scene, the midpoint of each room may be selected as a sampling point to be selected according to experience, or a specific point location may be selected as a sampling point to be selected in each room according to an operation instruction issued by the system.

Further, when data acquisition is performed based on each acquisition point to be selected, an RGBD depth camera may be used. Before describing the depth camera, the image captured by the camera will be described first.

For an image collected by a camera, the image can be divided into a color image and a depth image, the color image is also called an RGB image, an RGB color space is used as a basis for forming the color image, three components of R, G and B respectively correspond to colors of three channels of red, green and blue, and the final display effect of the image is determined by the superposition of the three components. The depth image is also called a distance image, and is different from the storage brightness value of the pixel points in the gray level image, the pixel points in the depth image store depth values, for each pixel point, the depth value represents the distance from the point to the camera, and further, the distance between the target object in the image and the camera can be determined through the depth values of the plurality of points. It should be understood by those skilled in the art that the depth value is only related to the distance, and is not related to the environment, light, direction, and other factors, so that the depth image can truly and accurately represent the geometric depth information of the object in the image, and provide a data base for the subsequent construction of the three-dimensional model, for example, when a camera shoots a certain room to obtain a plurality of corresponding depth images, the computer can restore the three-dimensional point cloud corresponding to the room based on the images.

It will thus be appreciated that an RGBD depth camera may be understood as a combination of a colour camera and a camera capable of acquiring depth images, including structured light cameras and Time of flight (ToF) cameras. Furthermore, after the current area is shot by the RGBD depth camera, a color image and a depth image can be output, and the images are integrated to obtain an image to be processed corresponding to the current area.

In this embodiment, in order to improve the accuracy of the constructed three-dimensional point cloud of each target, at least three depth cameras may be deployed at each sampling point to be selected, and meanwhile, the at least three depth cameras may be fixedly disposed on the same rotation axis, and the camera view angles of the at least three depth cameras are different. Specifically, a rotating shaft may be vertically disposed on the sampling point, and three depth cameras may be aimed at the upper, middle, and lower directions of the current area to perform shooting respectively, so as to obtain real image information of the top, horizontal, and bottom views of the current area. It will be appreciated that by deploying at least three depth cameras, the limitation of a single camera shooting angle can be avoided.

In this embodiment, after the sampling point to be selected deploys the rotating shaft and the at least three depth cameras, the rotating shaft needs to be controlled to rotate so that the depth cameras can acquire pictures of multiple viewing angles in the current area. Specifically, taking the current sampling point to be selected as a rotation center point of a rotation shaft, taking a preset rotation angle interval as a step length, and adjusting the shooting angles of at least three depth cameras fixed on the rotation shaft so as to obtain images to be processed corresponding to all the shooting angles through shooting; and when the rotation angle of the rotating shaft reaches a preset rotation angle threshold, obtaining a plurality of images to be processed corresponding to the current sampling points to be selected.

For example, the midpoint of a bedroom is selected as a sampling point to be selected, and after a rotating shaft and a depth camera are erected at the sampling point, the current angle of the rotating shaft is taken as 0 °, the interval of the rotating angles is set to be 10 °, and the rotating shaft is controlled to rotate by adjusting the step length by taking 10 ° as an angle. It is understood that at least three depth cameras respectively shoot corresponding color images and depth images when the rotating shaft rotates to 0 degrees, 10 degrees, 20 degrees, 8230degrees and 350 degrees, and then the images are taken as images to be processed corresponding to a bedroom.

S220, determining a single-point locus cloud model corresponding to each sampling locus to be selected according to the multiple images to be processed corresponding to the sampling loci to be selected and the camera parameters of at least three depth cameras.

In this embodiment, it is further required to determine camera parameters of the depth cameras, where the camera parameters include internal parameters of each depth camera and external parameters of at least three depth cameras, and the 3D reconstruction based on OpenCV may determine that the internal parameters of the depth cameras are internal parameter matrices, where f in the matrices is an internal parameter matrix _x The length of each f on an imaging plane with a focal length f represents how many pixels, and the external parameters comprise rotation and translation of the optical center of the camera.

Further, based on the to-be-processed image and the camera parameters corresponding to each to-be-selected sampling point, a point cloud model corresponding to each to-be-selected sampling point can be obtained, and it can be understood that the point cloud model corresponding to a single sampling point is a single-point cloud model, and the model can only reflect the multi-view image of the sampling point in a three-dimensional manner. Specifically, scattered and disordered point cloud data in the image can be extracted based on the image to be processed and the camera parameters, and the point cloud data is processed by using a specific point cloud fusion program or software, so that a single-point locus cloud model after point cloud fusion is obtained.

And S230, determining common view information of any two single-point cloud models, and determining a global three-dimensional point cloud corresponding to the target area according to the common view information.

In this embodiment, after determining the corresponding single-point location model for each to-be-selected location point, each location point model needs to be integrated to obtain a global three-dimensional point cloud reflecting the whole target area, for example, a single-point location point cloud model of each room is integrated to obtain a three-dimensional model representing the whole house, which is a global three-dimensional point cloud.

However, cameras at different sampling points inevitably acquire the same picture in the image acquisition process, and errors occur in the whole single-point cloud model, so that repeated parts in each point cloud model need to be processed.

Optionally, based on a triggering operation of a user on each single-point site cloud model, determining common view information of the two single-point site cloud models; or traversing each single-point cloud model based on a preset feature extraction method, and determining common-view information of any two single-point cloud models; and processing each single-point location point cloud model through a point cloud registration algorithm and common view information, and determining a global three-dimensional point cloud corresponding to the target area.

The repeated parts in any two single-point cloud models are represented by common-view information, and it can be understood that the common-view information reflects the pictures captured by both the two acquisition point cameras and the same parts in the point cloud models corresponding to the two acquisition points.

In this embodiment, there are two ways of determining the common-view information, the first way is that a user may drag, based on a specific application program, each single-point-site cloud model displayed in the application program, and specifically, after observing the common-view information part in any two single-point-site cloud models, the user may manually drag the common-view information part of one single-point-site cloud model to coincide with the common-view information part of another single-point-site cloud model. The second way is to match features in an image based on a computer, and those skilled in the art should understand that image feature matching may be implemented based on a variety of feature extraction and matching algorithms, and details of the embodiments of the present disclosure are not described herein.

Further, after the common-view information of any two sampling points is determined, point clouds in the corresponding single-point cloud model can be registered, the point cloud registration criterion of the model can be based on the common part of the scene, multi-frame images acquired at different time, angles and illumination intensities are overlapped and matched into a unified coordinate system, corresponding translation vectors and rotation matrixes are calculated, and redundant information in the translation vectors and the rotation matrixes is eliminated. It can be understood that point cloud configuration is essentially to solve a transformation relation between two piles of point clouds, namely a rotation relation R and a translation relation t, and registration of three-dimensional depth information is divided into three methods, namely coarse registration, fine registration, global registration and the like according to different image input conditions and reconstruction output requirements.

In the actual application process, point cloud registration can be performed based on an Iterative Closest Point (ICP) method, and the idea of the ICP algorithm is as follows: finding out the nearest point pairs in the two groups of point cloud sets, calculating the error of the nearest point pairs after transformation according to the estimated transformation relation (R and t), and determining the final transformation relation through continuous iteration until the error is less than a certain threshold or the iteration times are reached. Based on the scheme of the embodiment, after the common view information of any two single-point cloud models is determined, point cloud registration can be achieved based on an ICP (inductively coupled plasma) algorithm. Further, the point clouds of all the registered areas are balanced, and the global three-dimensional point cloud reflecting the whole picture of the target area can be obtained.

It should be noted that, by taking one of the single-point site cloud models as a reference, the common-view information between each of the other single-point site cloud models and the single-point site cloud model can be determined, so as to obtain the common-view information of any two single-point site cloud models.

And S240, determining a target local three-dimensional point cloud corresponding to the target area according to the global three-dimensional point cloud.

In this embodiment, after obtaining the global three-dimensional point cloud of the target area, in order to minimize the labeling information in the point cloud, it is necessary to determine the target local three-dimensional point cloud from the global three-dimensional point cloud. It can be understood that in the target local three-dimensional point cloud, the labeling information can be completely displayed only by less space, and the disorder of image information caused by excessive labeling information is avoided by labeling in the target local three-dimensional point cloud.

In the practical application process, a part corresponding to the overlooking visual angle can be selected from the global three-dimensional point cloud to be used as the target local three-dimensional point cloud. Specifically, determining a ground point cloud to be fitted according to the global three-dimensional point cloud; fitting each ground point cloud to be fitted based on a random sampling consistency algorithm to obtain a fitted ground, and determining a normal vector corresponding to the fitted ground; determining a rotation matrix corresponding to each point cloud based on the normal vector and the first target direction; processing the global three-dimensional point cloud based on the rotation matrix, and determining a target global point cloud of a target area; and determining a top view point cloud corresponding to the target area according to the target global point cloud, and taking the top view point cloud as a target local three-dimensional point cloud.

For example, a part corresponding to the top view of the house can be used as a target local three-dimensional point cloud, and therefore, the ground part can be determined from the whole point cloud model of the house as the ground point cloud to be fitted. Further, in order to avoid the oblique and distorted images in the obtained target local three-dimensional point cloud, the ground main direction needs to be calculated, that is, the point cloud is corrected to the ground as the xoz plane. Specifically, a ground normal vector may be calculated by fitting the ground using a Random Sample Consensus (RANSAC) algorithm. Further, the ground normal is corrected to the global y direction, a rotation matrix corresponding to each point cloud is obtained through calculation, and the point cloud rotates in a rigid coordinate system according to the rotation matrix obtained through calculation, so that the target global point cloud of the target area can be determined. And finally, selecting a part corresponding to the top view according to the target global point cloud to obtain the target local three-dimensional point cloud, wherein in the practical application process, the point cloud with the height less than half meter can be screened and fitted to obtain the target local three-dimensional point cloud corresponding to the top view.

It can be understood that by determining the corresponding part of the top view in the target global point cloud as the target local three-dimensional point cloud, the situation of image information confusion when the labeling information is too much is avoided, and the manual labeling operation of the user in the point cloud is also facilitated.

The above steps can be understood as referring to fig. 3, determining a single point location cloud model corresponding to each acquisition location point according to the acquired to-be-processed image of each acquisition location point to be selected, and processing the co-view information of any two determined single point location cloud models, optionally performing point cloud balancing processing, to obtain a target point cloud top view, that is, a target local point cloud. Further, mirror objects, such as windows and/or mirrors, etc., are marked in the overhead point cloud by corresponding tools.

And S250, determining at least one target sampling point from the to-be-selected sampling points and target visual angle information corresponding to the marking information in the target sampling points according to the obtained marking information in the target local three-dimensional point cloud.

And S260, processing the target point cloud of each target visual angle information to obtain a target image to be spliced corresponding to each target visual angle information.

And S270, splicing the target to-be-spliced images and/or the images to be processed of the to-be-selected sampling sites to obtain target panoramic spliced images corresponding to the to-be-selected sampling sites.

According to the technical scheme of the embodiment, at least three depth cameras on the rotating shaft are used for shooting at each to-be-selected sampling site, so that to-be-processed images of each scene can be efficiently and comprehensively acquired; determining the common-view information of any two single-point cloud models, so that the same parts in the two models can be removed conveniently; according to the target global point cloud, the top view point cloud part is determined to be used as the target local three-dimensional point cloud, so that the situation that image information is disordered when the labeling information is excessive is avoided, and the manual labeling operation of a user in the point cloud is conveniently executed.

EXAMPLE III

Fig. 4 is a schematic flowchart of a method for determining a target view according to a third embodiment of the present disclosure, in which, on the basis of the third embodiment, a marking tool is used to mark a local three-dimensional point cloud of a target, and then target view information is determined based on mirror marking information; processing the single-point cloud model based on the target visual angle information to obtain cavity information, and further processing the to-be-processed images based on the cavity information to obtain to-be-spliced images for splicing the panoramic view; and finally, performing differential selection and processing on the images to be spliced of different sampling sites to ensure that the generated panoramic spliced image is accurate and complete. The specific implementation manner can be referred to the technical scheme of the embodiment. The technical terms that are the same as or corresponding to the above embodiments are not repeated herein.

As shown in fig. 4, the method specifically includes the following steps:

s310, determining a target local three-dimensional point cloud corresponding to the target area according to the plurality of images to be processed of the acquisition points to be selected in the target area.

S320, marking the mirror surface, the mark glass and the mark to-be-processed area in the target local three-dimensional point cloud based on the marking tool to obtain mirror surface marking information, glass marking information and area marking information.

The marking tool may be a tool developed in an application program for performing a marking operation, for example, a button is developed in the application program, and a user may trigger the marking tool by clicking the button, and then perform the marking operation in the target local three-dimensional point cloud by using the tool.

In the practical application process, at least one of the mirror surface, the glass and the to-be-processed area in the target local three-dimensional point cloud can be marked by utilizing the marking tool. Further, based on the labeled content, corresponding labeled information may also be obtained, and it can be understood that, when the labeled target object is at least one of a river surface, glass, and a region to be processed, the corresponding labeled information includes at least one of mirror surface labeled information, glass labeled information, and region labeled information, which is specifically described below.

In this embodiment, mirror surface labeling information may be determined in the target local three-dimensional point cloud by using a labeling tool, where the mirror surface labeling information may reflect a position of a mirror surface in a target area, and includes vector information corresponding to a cross section of the mirror surface. For example, for a target local three-dimensional point cloud corresponding to a top view portion of a house, a user may draw a corresponding line segment on a mirror surface of each room by using a marking tool, and it may be understood that each line segment represents a cross section of the mirror surface, and at the same time, an orientation of the mirror surface may be determined based on the line segment.

It will be understood by those skilled in the art that in the target local three-dimensional point cloud, there may be glass and various regions that affect the color value ambiguity of the image by reflected light, and therefore, when there is glass and the above regions in the three-dimensional point cloud, it is also necessary to mark them in a way of processing the mirror. On the other hand, for various regions affecting the ambiguity of the color values of the image by the reflected light, the regions can be directly marked, and it can be understood that the marked regions are regions to be processed. On the other hand, similar to the mirror labeling information, the glass labeling information obtained by labeling the glass includes vector information corresponding to the cross section of the glass, and the region labeling information obtained by labeling the region to be processed includes vector information corresponding to the cross section of the region to be processed, and based on these vector information, the orientation of the glass or the orientation of the region to be processed can be determined as well.

For example, for a target local three-dimensional point cloud corresponding to a top view part of a house, for a window of each room and a region to be processed in which a glass decoration in the room is located, a user may draw a plurality of line segments in a manner similar to a marked mirror surface by using a marking tool, each line segment represents a cross section of the window or the glass decoration, and similarly, an orientation of the window or the glass decoration may be determined based on each line segment.

In this embodiment, based on the mirror surface labeling information, the glass labeling information and the region labeling information, in the subsequent panoramic image creating process, the ambiguity problem of the collected image color values caused by the reflection of light rays by a mirror or a window can be avoided, and then the problems of mismatching of characteristic values and inaccurate splicing of panoramic views are avoided. It should be noted that, for objects such as a mirror surface, glass, and a region to be processed, no matter which one or more of the objects are included in the target local three-dimensional point cloud, the objects can be processed in a targeted manner, and then the influence of the objects on the finally generated three-dimensional point cloud or panoramic stitched image is eliminated based on the processing result (corresponding labeling information).

S330, determining a target sampling site corresponding to the current marking information aiming at each marking information; and determining the target visual rotation angle of the current target sampling site for the corresponding marking information aiming at each target sampling site, and taking the target visual rotation angle as the target visual angle information of the current target sampling site.

Illustratively, after the labeling information of each mirror surface is determined, for a single mirror surface, the mirror surface may be used as a central point to obtain information of a plurality of point locations within a circumference of an acquisition range, and it can be understood that these point locations are target acquisition points, and when an effective depth range of an acquisition device (depth camera) is 5m, the acquisition range is a range within a circumference with a radius of 5m and with the mirror surface as a center. It should be understood by those skilled in the art that the manner of determining the target sampling point based on the glass labeling information and the region labeling information is similar to that in the above example, and the details of the embodiments of the present disclosure are not repeated herein.

Furthermore, for each target sampling point, a connection line between the point and the starting point of the mirror surface, the glass or the area to be processed needs to be calculated, and meanwhile, the direction of the mirror surface, the glass or the area to be processed is combined for visual judgment. Since the rotation axis can be rotated by 360 ° to capture the real image around the current region, the process of visual judgment can be understood as the angle at which the current rotation axis is located when the device can capture the mirror surface when the current region is captured at the target capture point by the capture device. In this embodiment, the angle corresponding to the rotation axis when the mirror surface, the glass, or the to-be-processed area is photographed is the target visible rotation angle, and correspondingly, the to-be-processed image photographed at the angle may be a picture reflecting the target view angle.

It should be noted that, for the sampling points where the labeling information does not exist, the visible rotation angle of the target may not be determined, and for the to-be-processed images acquired by these sampling points, the system may directly process them or store them in a specific repository.

And S340, aiming at each target visual angle information, obtaining the hole information by cutting the single-point cloud model of the target sampling point corresponding to the current target visual angle information.

In this embodiment, after the target view information is determined, a portion corresponding to the target view information may be cut on the model to obtain cavity information, where the cavity information corresponds to the cut point cloud and represents the removed portion in the single-point cloud model.

For example, after the target view angle information corresponding to the bedroom is obtained, the point cloud model corresponding to the bedroom can be cut to remove the point cloud corresponding to the mirror or window part of the bedroom, and then the cavity information is generated on the model.

For example, referring to fig. 5, the target sampling point may be determined from the plurality of sampling points to be selected according to the mirror labeling information. And determining whether each camera visual angle can see mirror marking information or not for each target sampling point, if so, cutting the point cloud corresponding to the target visual angle information, otherwise, not processing the target visual angle information, and thus, the point cloud is not processed. After the point cloud is cut, the hole information corresponding to the cutting part can be obtained. Further, referring to fig. 6, for each hole information, position information of the hole may be obtained, and the position is back-projected to a corresponding image to be processed, a corresponding pixel point of the corresponding pixel point is determined, and after the pixel at the pixel point is removed, all images to be processed of the sampling point to be selected are stitched, so as to obtain a panoramic stitched image.

S350, projecting the hole information to the corresponding image to be processed, determining pixel points to be eliminated in the image to be processed, and eliminating the pixel points to be eliminated from the image to be processed to obtain the image to be spliced corresponding to the target visual angle information.

In this embodiment, in order to construct a panoramic view for a target area, after determining the hole information on the single-point-location cloud model, the hole information needs to be projected into an image to be processed, which can be understood as projecting the hole information onto a phase plane acquired by an original camera. Further, a mask corresponding to the hole is calculated based on the acquired to-be-processed image, after the mask is determined, the system does not perform feature extraction and other processing on pixels at the mask in the subsequent process, and the pixel information is directly discarded (for example, each pixel point is directly set to zero) to obtain the to-be-spliced image at the target view angle. It should be noted that, in the actual application process, the pictures at the multiple target view angles are processed in the above manner, and then multiple images to be stitched corresponding to the target view angles are obtained.

And S360, for each sampling point to be selected, splicing the target images to be spliced and/or the images to be processed of the current sampling point to be selected to obtain a target panoramic spliced image corresponding to the current sampling point to be selected.

In this embodiment, the determined images to be stitched for generating the target panoramic stitched image also have differences for different sampling points to be selected. Illustratively, when a sampling point corresponding to a bedroom of a house is taken as a current sampling point to be selected, images to be spliced corresponding to the sampling point are respectively a target image to be spliced from which a part (mirror surface) corresponding to cavity information is removed and a to-be-processed image without cavity information, and are spliced on the basis of the two images, so that a target panoramic spliced image corresponding to the bedroom can be obtained. When the sampling site corresponding to the house prescription is taken as the current sampling site to be selected, as no mirror surface and window exist in the kitchen and no cavity information exists in the corresponding single-point site cloud model, the image to be spliced corresponding to the sampling site is the image to be processed which is initially acquired by the acquisition equipment, and the target panoramic spliced image corresponding to the kitchen can be obtained by splicing based on the images.

By carrying out differential selection and processing on the images to be spliced of different sampling sites to be selected, accurate and complete target panoramic spliced images corresponding to the sampling sites to be selected can be obtained.

According to the technical scheme of the embodiment, marking is carried out in the target local three-dimensional point cloud based on a marking tool, and then target visual angle information is determined based on mirror marking information; processing the single-point cloud model based on the target visual angle information to obtain cavity information, and further processing the to-be-processed images based on the cavity information to obtain to-be-spliced images for splicing the panoramic view; and finally, performing differential selection and processing on the images to be spliced of different sampling sites to ensure that the generated panoramic spliced image is accurate and complete.

Example four

Fig. 7 is a schematic flow chart of a method for determining a target view according to a fourth embodiment of the present disclosure, and based on the foregoing embodiments, when cavity information is obtained, a corresponding single-point site cloud model is filled, so that not only can other application programs directly call the complete single-point site cloud model of the collection site, but also when the single-point site cloud model of the collection site needs to be used subsequently, computing resources are not wasted and are built again; the multiple target panoramic stitched images are displayed on the target display interface, a channel for viewing real pictures of a target area is provided for a user in a concise mode, and further, the single-point locus cloud model is called in the panoramic image switching process to generate the jump animation, so that the smoothness of image switching is enhanced in the aspect of user subjectivity, the user can know in advance which part of the target area the next panoramic stitched image comes from, and the human-computer interaction experience of the user is improved. The specific implementation manner can be referred to the technical scheme of the embodiment. The technical terms that are the same as or corresponding to the above embodiments are not repeated herein.

As shown in fig. 7, the method specifically includes the following steps:

s410, determining a target local three-dimensional point cloud corresponding to the target area according to the plurality of images to be processed of the acquisition points to be selected in the target area.

And S420, determining at least one target sampling point from the to-be-selected sampling points and target visual angle information corresponding to the labeling information in the target sampling points according to the obtained labeling information in the target local three-dimensional point cloud.

And S430, aiming at each target visual angle information, obtaining the hole information by cutting the single-point locus cloud model of the target sampling locus corresponding to the current target visual angle information.

S440, projecting the hole information to the corresponding image to be processed, determining pixel points to be eliminated in the image to be processed, and eliminating the pixel points to be eliminated from the image to be processed to obtain the image to be spliced corresponding to the target visual angle information.

S450, splicing the target to-be-spliced images and/or the images to be processed of the to-be-selected sampling sites to obtain target panoramic spliced images corresponding to the to-be-selected sampling sites.

S460, when the cavity information is obtained, obtaining a to-be-filled single-point site cloud model of a point cloud model of a target acquisition site corresponding to the current target visual angle information; and updating the single-point cloud model corresponding to the current target acquisition point through filling the unit point cloud model to be filled.

In this embodiment, after the single-point cloud model is processed and the hole information is obtained, in order to keep the single-point cloud model corresponding to the sampling point complete, the model from which part of the point cloud is removed may be filled and reconstructed. In the actual model filling process, the removed part of the point cloud can be repaired based on a specific algorithm, the repaired point cloud is further filled on the single-point site cloud model corresponding to the point cloud model, and the removed part of the point cloud model can be manually filled by selecting a specific filling model from the model base. Illustratively, when the single-point cloud model corresponding to the bedroom sampling point is filled and reconstructed, the removed mirror surface part can be repaired to be made of materials which cannot influence light transmission, and then the repaired point cloud is filled on the original point cloud model, or the wall surface point cloud with the width corresponding to the mirror surface is directly selected from the model base and directly covered on the original point cloud model, so that the integrity of the bedroom single-point cloud model is ensured. Those skilled in the art should understand that the specific filling processing manner of the point cloud model may be selected according to actual situations, and the embodiment of the present disclosure is not specifically limited herein.

Further, after filling and reconstructing the cavity information portion of the single-point location point cloud model, the model of the exact portion of the point cloud (the cavity information corresponding portion) stored in the target repository needs to be updated based on the updated point cloud model. By the method, other application programs can directly call the complete single-point site cloud model of the sampling site, and computing resources are not wasted to construct the point cloud model again when the single-point site cloud model of the current sampling site is needed to be used subsequently.

With continued reference to fig. 5, after the point cloud is cropped, the void may be reconstructed to obtain a single-point cloud model after point cloud correction.

And S470, overlapping and displaying the target panoramic stitching images corresponding to the sampling points to be selected in a target display interface so that a user can browse the panoramic images corresponding to the target area in a triggering mode.

In this embodiment, after the target panoramic stitched images corresponding to the sampling points to be selected are determined, the images may be displayed in a target display interface in a stacked manner for a user to view. Illustratively, in an application program related to house leasing, after three corresponding target panoramic stitched images are constructed for a bedroom, a kitchen and a bathroom of a certain house based on the scheme of the embodiment of the disclosure, the images can be associated with an identifier of the house, three layers are stacked in a target display interface of the application program for display, when a user clicks the house to check detailed house information, a panoramic stitched image corresponding to the bedroom of the house can be displayed in the target display interface, and when sliding or selection operation of the user is detected, the kitchen panoramic stitched image and the bathroom panoramic stitched image in the image stack can be sequentially displayed in the target display interface.

A channel for viewing the real picture of the target area is provided for a user in a concise mode by displaying a plurality of target panoramic stitching images corresponding to the sampling points to be selected on a target display interface.

And S480, when jumping from the current target panoramic mosaic image to the next target panoramic mosaic image is detected, displaying a jumping animation.

Because a plurality of target panoramic spliced images are displayed on a target display interface in a laminated manner, when different images are switched in the interface according to a user instruction, corresponding jump animations need to be set in order to prevent the switching process from being too abrupt. In this embodiment, when one target panorama stitched image has been displayed on the target display interface, the single-point locus cloud model corresponding to the next target panorama stitched image in the stack may be used as a call object of the jump animation.

Illustratively, when a user browses a panoramic mosaic image of a bedroom of a house and triggers display of a kitchen panoramic mosaic image through sliding operation, image switching occurs in a target display interface, at this time, a single-point cloud model corresponding to a kitchen sampling point is a calling object of a jump animation, that is, in the image switching process, an application program calls the single-point cloud model corresponding to the kitchen sampling point stored in a storage library and displays the model in the target display interface.

The single-point locus cloud model is called to generate the jump animation in the panoramic image switching process, so that the smoothness of image switching is enhanced on the subjectivity of a user, the abrupt feeling brought to the user is avoided, meanwhile, the user can know in advance which part of the target area the next panoramic spliced image comes from, and the human-computer interaction experience of the user is improved.

In the embodiment of the present disclosure, the target area may be a house view area, and therefore, the scheme of the embodiment of the present disclosure may be applied to a variety of scenes in which a house panoramic stitched image and/or a point cloud model needs to be constructed. It should be understood by those skilled in the art that, except for the house view area, when there is no major technical obstacle in the construction of hardware devices, the scheme of the embodiment may also be applied to a variety of scenes in which a panoramic stitched image and/or a point cloud model of a certain area in reality needs to be constructed.

According to the technical scheme, when the cavity information is obtained, the corresponding single-point site cloud model is filled, so that other application programs can directly call the complete single-point site cloud model of the acquisition site, and the calculation resources are not wasted and are built again when the single-point site cloud model of the acquisition site needs to be used subsequently; the multiple target panoramic stitched images are displayed on the target display interface, a channel for viewing real pictures of a target area is provided for a user in a concise mode, and further, the single-point locus cloud model is called in the panoramic image switching process to generate the jump animation, so that the smoothness of image switching is enhanced in the aspect of user subjectivity, the user can know in advance which part of the target area the next panoramic stitched image comes from, and the human-computer interaction experience of the user is improved.

EXAMPLE five

Fig. 8 is a block diagram of a device for determining a target view according to a fifth embodiment of the disclosure, which is capable of executing a method for determining a target view according to any embodiment of the disclosure, and has corresponding functional modules and beneficial effects of the execution method. As shown in fig. 8, the apparatus specifically includes: a target local three-dimensional point cloud determining module 510, a target sampling point determining module 520, a target to-be-stitched image determining module 530 and a target panoramic stitched image determining module 540.

A target local three-dimensional point cloud determining module 510, configured to determine a target local three-dimensional point cloud corresponding to a target region according to a plurality of images to be processed of sampling points to be selected in the target region.

And a target sampling point determining module 520, configured to determine, according to the obtained labeling information in the target local three-dimensional point cloud, at least one target sampling point from the to-be-selected sampling points, and target view angle information corresponding to the labeling information in the target sampling points.

And a target to-be-spliced image determining module 530, configured to obtain a target to-be-spliced image corresponding to each target view information by processing the target point cloud of each target view information.

And the target panoramic stitched image determining module 540 is configured to perform stitching processing on the target to-be-stitched image and/or the to-be-processed image of each to-be-selected sampling point to obtain a target panoramic stitched image corresponding to each to-be-selected sampling point.

On the basis of the technical schemes, the device for determining the target view further comprises a module for determining the sampling position to be selected and a module for determining the image to be processed.

And the sampling site to be selected determining module is used for determining each sampling site to be selected in the target area.

The device comprises a to-be-processed image determining module, a processing module and a processing module, wherein the to-be-processed image determining module is used for acquiring a plurality of to-be-processed images shot at the current to-be-selected sampling site based on at least three depth cameras aiming at each to-be-selected sampling site; the at least three depth cameras are fixedly arranged on the same rotating shaft, and the camera view angles of the at least three depth cameras are different.

Optionally, the to-be-processed image determining module is further configured to adjust the shooting angles of at least three depth cameras fixed on the rotating shaft by taking the current to-be-selected sampling point as a rotation center point of the rotating shaft and taking a preset rotation angle interval as a step length, so as to obtain to-be-processed images corresponding to the shooting angles by shooting; and when the rotation angle of the rotating shaft reaches a preset rotation angle threshold, obtaining a plurality of images to be processed corresponding to the current sampling points to be selected.

On the basis of the above technical solutions, the target local three-dimensional point cloud determining module 510 includes a single-point location point cloud model determining unit, a global three-dimensional point cloud determining unit, and a target local three-dimensional point cloud determining unit.

The single-point site cloud model determining unit is used for determining a single-point site cloud model corresponding to each to-be-selected sampling site according to the multiple to-be-processed images corresponding to the to-be-selected sampling sites and the camera parameters of the at least three depth cameras; wherein the camera parameters include an internal parameter for each depth camera and an external parameter for each of the at least three depth cameras.

And the global three-dimensional point cloud determining unit is used for determining the common-view information of any two single-point cloud models and determining the global three-dimensional point cloud corresponding to the target area according to the common-view information.

And the target local three-dimensional point cloud determining unit is used for determining the target local three-dimensional point cloud corresponding to the target area according to the global three-dimensional point cloud.

Optionally, the global three-dimensional point cloud determining unit is further configured to determine common view information of the two single-point cloud models based on a triggering operation of a user on each single-point cloud model; or traversing each single-point cloud model based on a preset feature extraction method, and determining common-view information of any two single-point cloud models; and processing each single-point location point cloud model through a point cloud registration algorithm and the common view information, and determining a global three-dimensional point cloud corresponding to the target area.

Optionally, the target local three-dimensional point cloud determining unit is further configured to determine a ground point cloud to be fitted according to the global three-dimensional point cloud; fitting each ground point cloud to be fitted based on a random sampling consistency algorithm to obtain a fitted ground, and determining a normal vector corresponding to the fitted ground; determining a rotation matrix corresponding to each point cloud based on the normal vector and the first target direction; processing the global three-dimensional point cloud based on the rotation matrix, and determining a target global point cloud of the target area; and determining a top view point cloud corresponding to the target area according to the target global point cloud, and taking the top view point cloud as the target local three-dimensional point cloud.

On the basis of the above technical solutions, the labeling information includes at least one of mirror labeling information, glass labeling information, and region labeling information, and the apparatus for determining a target view further includes a mirror labeling information determining module.

The mirror surface labeling information determining module is used for marking a mirror surface, marking glass and marking a to-be-processed area in the target local three-dimensional point cloud based on a labeling tool to obtain mirror surface labeling information, glass labeling information and area labeling information; the mirror surface labeling information comprises vector information corresponding to the cross section of the mirror surface, the glass labeling information comprises vector information corresponding to the cross section of the glass, the area labeling comprises vector information corresponding to the cross section of the area to be processed, and the target panoramic mosaic of each sampling point to be selected in the target area is determined according to the labeling information.

Optionally, the mirror marking information determining module is further configured to determine, for each piece of marking information, a target sampling point corresponding to the current marking information; and determining a target visual rotation angle of the current target sampling site for corresponding marking information aiming at each target sampling site, and taking the target visual rotation angle as target visual angle information of the current target sampling site.

On the basis of the above technical solutions, the target to-be-stitched image determining module 530 includes a hole information determining unit and a projecting unit.

The hole information determining unit is used for cutting a single-point cloud model of a target sampling point corresponding to the current target visual angle information to obtain hole information aiming at each target visual angle information; wherein the hole information corresponds to the cropped point cloud.

And the projection unit is used for projecting the hole information into the corresponding image to be processed, determining pixel points to be eliminated in the image to be processed, and eliminating the pixel points to be eliminated from the image to be processed to obtain the image to be spliced corresponding to the target visual angle information.

Optionally, the target panoramic stitched image determining module 540 is further configured to, for each sampling point to be selected, stitch the target to-be-stitched image of the current sampling point to be selected and/or the to-be-processed image to obtain a target panoramic stitched image corresponding to the current sampling point to be selected.

On the basis of the technical schemes, the device for determining the target view further comprises a module for determining the to-be-filled single-point site cloud model and a module for updating the single-point site cloud model.

And the module for determining the point cloud model of the single point to be filled is used for obtaining the point cloud model of the target sampling point corresponding to the current target visual angle information when the cavity information is obtained.

And the single-point site cloud model updating module is used for updating the single-point site cloud model corresponding to the current target acquisition site through filling processing of the unit point cloud model to be filled.

On the basis of the technical solutions, the device for determining the target view further comprises a display module.

And the display module is used for displaying the target panoramic stitching image corresponding to each sampling point to be selected in a target display interface in a laminated manner so that a user can conveniently trigger and browse the panoramic image corresponding to the target area.

On the basis of the technical solutions, the device for determining the target view further comprises a detection module.

The device comprises a detection module and a display module, wherein the detection module is used for displaying a jump animation when jumping from a current target panoramic mosaic image to a next target panoramic mosaic image is detected, and the jump animation corresponds to a single-point locus cloud model corresponding to the next target panoramic mosaic image.

On the basis of the technical schemes, the target area is a house view area.

According to the technical scheme of the embodiment of the disclosure, firstly, a target local three-dimensional point cloud corresponding to a target area is determined according to a plurality of images to be processed of sampling points to be selected in the target area so as to mark a specific part of the target local three-dimensional point cloud; determining at least one target sampling point from the sampling points to be selected and target visual angle information corresponding to the labeling information in the target sampling points according to the obtained labeling information in the target local three-dimensional point cloud, so as to determine a visual angle corresponding to a labeling part in the sampling points; the target point cloud of each target visual angle information is processed to obtain the target to-be-spliced image corresponding to each target visual angle information, and then the target to-be-spliced image and/or the to-be-processed image of each to-be-selected sampling site are spliced to obtain the target panoramic spliced image corresponding to each to-be-selected sampling site, so that the problem that the influence of external factors on model construction is not considered in the prior art is solved, the constructed target panoramic spliced image is different from the actual panoramic image of a target area to a certain extent, the image displayed on display software is different from the actual image, and the user experience is poor, the automatic construction of the panoramic image of the target area is realized, meanwhile, the influence of interference factors in the area on the accuracy of the constructed panoramic image is avoided, the accuracy of splicing of the panoramic image is improved, and the technical effect of the user experience is improved.

The device for determining the target view, provided by the embodiment of the disclosure, can execute the method for determining the target view, provided by any embodiment of the disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

It should be noted that, the units and modules included in the apparatus are merely divided according to functional logic, but are not limited to the above division as long as the corresponding functions can be implemented; in addition, specific names of the functional units are only used for distinguishing one functional unit from another, and are not used for limiting the protection scope of the embodiments of the present disclosure.

EXAMPLE six

Fig. 9 is a schematic structural diagram of an electronic device according to a sixth embodiment of the present disclosure. Referring now to fig. 9, a schematic diagram of an electronic device (e.g., the terminal device or the server in fig. 9) 600 suitable for implementing embodiments of the present disclosure is shown. The terminal device in the embodiments of the present disclosure may include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a vehicle terminal (e.g., a car navigation terminal), and the like, and a stationary terminal such as a digital TV, a desktop computer, and the like. The electronic device shown in fig. 9 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.

As shown in fig. 9, the electronic device 600 may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 601 that may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 602 or a program loaded from a storage means 606 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the electronic apparatus 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An editing/output (I/O) interface 605 is also connected to bus 604.

Generally, the following devices may be connected to the I/O interface 605: editing devices 606 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 607 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 606 including, for example, magnetic tape, hard disk, etc.; and a communication device 609. The communication means 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. While fig. 9 illustrates an electronic device 600 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.

In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer readable medium, the computer program containing program code for performing the method illustrated by the flow chart. In such embodiments, the computer program may be downloaded and installed from a network through the communication device 609, or installed from the storage device 606, or installed from the ROM 602. The computer program, when executed by the processing device 601, performs the above-described functions defined in the methods of the embodiments of the present disclosure.

The names of messages or information exchanged between devices in the embodiments of the present disclosure are for illustrative purposes only, and are not intended to limit the scope of the messages or information.

The electronic device provided by the embodiment of the present disclosure belongs to the same inventive concept as the method for determining the target view provided by the above embodiment, and technical details that are not described in detail in the embodiment can be referred to the above embodiment, and the embodiment has the same beneficial effects as the above embodiment.

EXAMPLE seven

The disclosed embodiments provide a computer storage medium having a computer program stored thereon, which when executed by a processor implements the method for determining a target view provided by the above embodiments.

It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.

In some embodiments, the clients, servers may communicate using any currently known or future developed network Protocol, such as HTTP (HyperText Transfer Protocol), and may interconnect with any form or medium of digital data communication (e.g., a communications network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network.

The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device.

The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to:

Computer program code for carrying out operations for the present disclosure may be written in any combination of one or more programming languages, including but not limited to an object oriented programming language such as Java, smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units described in the embodiments of the present disclosure may be implemented by software or hardware. Where the name of a unit does not in some cases constitute a limitation of the unit itself, for example, the first retrieving unit may also be described as a "unit for retrieving at least two internet protocol addresses".

The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems on a chip (SOCs), complex Programmable Logic Devices (CPLDs), and the like.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

According to one or more embodiments of the present disclosure, [ example one ] there is provided a method of determining a target view, the method comprising:

According to one or more embodiments of the present disclosure, [ example two ] there is provided a method of determining a target view, further comprising:

optionally, determining each sampling site to be selected in the target region;

acquiring a plurality of images to be processed shot at the current sampling site to be selected based on at least three depth cameras aiming at the sampling site to be selected; the at least three depth cameras are fixedly arranged on the same rotating shaft, and the camera view angles of the at least three depth cameras are different.

According to one or more embodiments of the present disclosure, [ example three ] there is provided a method of determining a target view, further comprising:

optionally, taking the current sampling point to be selected as a rotation central point of the rotation shaft, taking a preset rotation angle interval as a step length, and adjusting the shooting angles of at least three depth cameras fixed on the rotation shaft to obtain images to be processed corresponding to each shooting angle through shooting;

and when the rotation angle of the rotating shaft reaches a preset rotation angle threshold, obtaining a plurality of images to be processed corresponding to the current sampling points to be selected.

According to one or more embodiments of the present disclosure, [ example four ] there is provided a method of determining a target view, further comprising:

optionally, determining a single-point location cloud model corresponding to each to-be-selected sampling location according to the multiple to-be-processed images corresponding to the to-be-selected sampling locations and the camera parameters of the at least three depth cameras; wherein the camera parameters include an internal parameter of each depth camera and external parameters of each of at least three depth cameras;

determining common-view information of any two point cloud models of single points, and determining a global three-dimensional point cloud corresponding to the target area according to the common-view information;

and determining a target local three-dimensional point cloud corresponding to the target area according to the global three-dimensional point cloud.

According to one or more embodiments of the present disclosure, [ example five ] there is provided a method of determining a target view, further comprising:

optionally, based on a triggering operation of a user on each single-point site cloud model, determining common view information of the two single-point site cloud models; or the like, or, alternatively,

traversing each single-point location point cloud model based on a preset feature extraction method, and determining common-view information of any two single-point location point cloud models;

and processing each single-point location point cloud model through a point cloud registration algorithm and the common view information, and determining a global three-dimensional point cloud corresponding to the target area.

According to one or more embodiments of the present disclosure, [ example six ] there is provided a method of determining a target view, further comprising:

optionally, determining a ground point cloud to be fitted according to the global three-dimensional point cloud;

fitting each ground point cloud to be fitted based on a random sampling consistency algorithm to obtain a fitted ground, and determining a normal vector corresponding to the fitted ground;

determining a rotation matrix corresponding to each point cloud based on the normal vector and the first target direction;

processing the global three-dimensional point cloud based on the rotation matrix, and determining a target global point cloud of the target area;

and determining a top view point cloud corresponding to the target area according to the target global point cloud, and taking the top view point cloud as the target local three-dimensional point cloud.

According to one or more embodiments of the present disclosure, [ example seven ] there is provided a method of determining a target view, further comprising:

optionally, the labeling information includes at least one of mirror labeling information, glass labeling information, and area labeling information; marking a mirror surface, marking glass and marking a to-be-processed area in the target local three-dimensional point cloud based on a marking tool to obtain mirror surface marking information, glass marking information and area marking information; the mirror surface labeling information comprises vector information corresponding to the cross section of the mirror surface, the glass labeling information comprises vector information corresponding to the cross section of the glass, the area labeling comprises vector information corresponding to the cross section of the area to be processed, and the target panoramic mosaic of each sampling point to be selected in the target area is determined according to the labeling information.

According to one or more embodiments of the present disclosure [ example eight ] there is provided a method of determining a target view, further comprising:

optionally, for each piece of labeling information, determining a target sampling point corresponding to the current labeling information;

and determining a target visual rotation angle of the current target sampling site for corresponding marking information aiming at each target sampling site, and taking the target visual rotation angle as target visual angle information of the current target sampling site.

According to one or more embodiments of the present disclosure, [ example nine ] there is provided a method of determining a target view, further comprising:

optionally, for each target visual angle information, obtaining hole information by cutting a single-point locus cloud model of a target sampling locus corresponding to the current target visual angle information; wherein the cavity information corresponds to a cropped point cloud;

and projecting the hole information to the corresponding image to be processed, determining pixel points to be eliminated in the image to be processed, and eliminating the pixel points to be eliminated from the image to be processed to obtain the image to be spliced corresponding to the target visual angle information.

According to one or more embodiments of the present disclosure, [ example ten ] there is provided a method of determining a target view, further comprising:

optionally, for each sampling point to be selected, the target to-be-stitched image of the current sampling point to be selected and/or the to-be-processed image are stitched, so as to obtain a target panoramic stitched image corresponding to the current sampling point to be selected.

According to one or more embodiments of the present disclosure [ example eleven ] there is provided a method of determining a target view, further comprising:

optionally, when the cavity information is obtained, obtaining a to-be-filled single-point cloud model of a target acquisition point corresponding to the current target view information;

and updating the single-point site cloud model corresponding to the current target sampling site by filling the unit point cloud model to be filled.

According to one or more embodiments of the present disclosure, [ example twelve ] there is provided a method of determining a target view, further comprising:

optionally, the target panoramic stitched image corresponding to each sampling point to be selected is displayed in a target display interface in a laminated manner, so that a user can browse the panoramic image corresponding to the target area in a triggering manner.

According to one or more embodiments of the present disclosure, [ example thirteen ] there is provided a method of determining a target view, further comprising:

optionally, when it is detected that the target panorama stitched image jumps to a next target panorama stitched image from a current target panorama stitched image, a jump animation is displayed, where the jump animation corresponds to a single-point locus cloud model corresponding to the next target panorama stitched image.

According to one or more embodiments of the present disclosure, [ example fourteen ] there is provided a method of determining a target view, further comprising:

optionally, the target area is a house view area.

According to one or more embodiments of the present disclosure, [ example fifteen ] there is provided an apparatus of determining a target view, further comprising:

the target to-be-spliced image determining module is used for processing the target point cloud of each target visual angle information to obtain a target to-be-spliced image corresponding to each target visual angle information;

The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the disclosure herein is not limited to the particular combination of features described above, but also encompasses other embodiments in which any combination of the features described above or their equivalents does not depart from the spirit of the disclosure. For example, the above features and the technical features disclosed in the present disclosure (but not limited to) having similar functions are replaced with each other to form the technical solution.

Further, while operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A method of determining a target view, comprising:

determining at least one target sampling point from the sampling points to be selected and target visual angle information corresponding to the label information in the target sampling points according to the obtained label information in the target local three-dimensional point cloud;

splicing the target to-be-spliced images and/or the images to be processed of the to-be-selected sampling sites to obtain target panoramic spliced images corresponding to the to-be-selected sampling sites;

the method for determining at least one target sampling point from the to-be-selected sampling points according to the obtained labeling information in the target local three-dimensional point cloud and the target visual angle information corresponding to the labeling information in the target sampling points comprises the following steps:

marking a mirror surface, marking glass and marking a to-be-processed area in the target local three-dimensional point cloud based on a marking tool to obtain mirror surface marking information, glass marking information and area marking information; the mirror marking information comprises vector information corresponding to the cross section of the mirror, the glass marking information comprises vector information corresponding to the cross section of the glass, and the area marking information comprises vector information corresponding to the cross section of the area to be processed;

aiming at each marking information, determining a target sampling site corresponding to the current marking information;

and determining the target visual rotation angle of the current target sampling site for the corresponding marking information aiming at each target sampling site, and taking the target visual rotation angle as the target visual angle information of the current target sampling site.

2. The method according to claim 1, wherein before determining the target local three-dimensional point cloud corresponding to the target region according to the plurality of images to be processed of the sampling points to be selected in the target region, the method comprises:

determining each sampling site to be selected in the target area;

3. The method of claim 2, wherein the acquiring a plurality of images to be processed based on at least three depth cameras taken at a currently selected sampling site comprises:

taking the current sampling point to be selected as a rotation central point of the rotating shaft, taking a preset rotation angle interval as a step length, and adjusting the shooting angles of at least three depth cameras fixed on the rotating shaft so as to obtain images to be processed corresponding to all the shooting angles through shooting;

4. The method according to claim 1, wherein the determining a target local three-dimensional point cloud corresponding to a target region according to a plurality of images to be processed of each sampling point to be selected in the target region comprises:

determining a single-point locus cloud model corresponding to each sampling locus to be selected according to a plurality of images to be processed corresponding to each sampling locus to be selected and camera parameters of at least three depth cameras; wherein the camera parameters include an internal parameter for each depth camera and an external parameter for each of the at least three depth cameras;

5. The method of claim 4, wherein determining co-view information for any two point cloud models, and determining a global three-dimensional point cloud corresponding to the target region according to the co-view information comprises:

determining common view information of the two single-point site cloud models based on the triggering operation of a user on each single-point site cloud model; or the like, or, alternatively,

6. The method of claim 4, wherein determining a target local three-dimensional point cloud corresponding to the target region from the global three-dimensional point cloud comprises:

determining a ground point cloud to be fitted according to the global three-dimensional point cloud;

7. The method according to claim 1, wherein the obtaining of the target to-be-stitched image corresponding to each target view information by processing the target point cloud of each target view information comprises:

aiming at each target visual angle information, obtaining cavity information by cutting a single-point cloud model of a target sampling point corresponding to the current target visual angle information; wherein the cavity information corresponds to a cropped point cloud;

8. The method according to claim 1, wherein the obtaining of the target panoramic stitched image corresponding to each to-be-selected sampling site by stitching the target to-be-stitched image and/or the to-be-processed image of each to-be-selected sampling site comprises:

and for each sampling point to be selected, splicing the target images to be spliced of the current sampling point to be selected and/or the images to be processed to obtain a target panoramic spliced image corresponding to the current sampling point to be selected.

9. The method of claim 7, after determining the hole information, further comprising:

when the cavity information is obtained, obtaining a to-be-filled single-point cloud model of a target acquisition point corresponding to the current target visual angle information;

and updating the single-point site cloud model corresponding to the current target sampling site through filling the single-point site cloud model to be filled.

10. The method of claim 9, further comprising:

and displaying the target panoramic stitching image corresponding to each sampling point to be selected in a target display interface in a laminated manner so that a user can conveniently trigger and browse the panoramic image corresponding to the target area.

11. The method of claim 10, further comprising:

and when jumping from the current target panoramic stitched image to the next target panoramic stitched image is detected, displaying a jumping animation, wherein the jumping animation corresponds to the single-point locus cloud model corresponding to the next target panoramic stitched image.

12. The method of any one of claims 1-11, wherein the target area is a house view area.

13. An apparatus for determining a target view, comprising:

the target local three-dimensional point cloud determining module is used for determining a target local three-dimensional point cloud corresponding to a target area according to a plurality of images to be processed of each sampling point to be selected in the target area;

the target panoramic mosaic image determining module is used for carrying out mosaic processing on the target to-be-mosaic images and/or the images to be processed of the sampling sites to be selected to obtain target panoramic mosaic images corresponding to the sampling sites to be selected;

the device also comprises a mirror marking information determining module, a processing module and a processing module, wherein the mirror marking information determining module is used for marking a mirror, marking glass and marking a to-be-processed area in the target local three-dimensional point cloud based on a marking tool to obtain mirror marking information, glass marking information and area marking information; the mirror surface labeling information comprises vector information corresponding to the cross section of the mirror surface, the glass labeling information comprises vector information corresponding to the cross section of glass, and the region labeling comprises vector information corresponding to the cross section of a region to be processed so as to obtain the labeling information;

14. An electronic device, characterized in that the electronic device comprises:

one or more processors;

a storage device for storing one or more programs,

when executed by the one or more processors, cause the one or more processors to implement a method of determining a target view as recited in any of claims 1-12.

15. A storage medium containing computer executable instructions for performing the method of determining a target view as claimed in any one of claims 1-12 when executed by a computer processor.