WO2023035548A1 - 目标环境的信息管理方法及相关增强现实的显示方法、电子设备、存储介质、计算机程序和计算机程序产品 - Google Patents

目标环境的信息管理方法及相关增强现实的显示方法、电子设备、存储介质、计算机程序和计算机程序产品 Download PDF

Info

Publication number
WO2023035548A1
WO2023035548A1 PCT/CN2022/074966 CN2022074966W WO2023035548A1 WO 2023035548 A1 WO2023035548 A1 WO 2023035548A1 CN 2022074966 W CN2022074966 W CN 2022074966W WO 2023035548 A1 WO2023035548 A1 WO 2023035548A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
information
voxels
voxel
semantic information
Prior art date
Application number
PCT/CN2022/074966
Other languages
English (en)
French (fr)
Inventor
盛崇山
Original Assignee
上海商汤智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海商汤智能科技有限公司 filed Critical 上海商汤智能科技有限公司
Publication of WO2023035548A1 publication Critical patent/WO2023035548A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
    • G06T17/205—Re-meshing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality

Definitions

  • the present disclosure relates to the technical field of augmented reality, and in particular, relates to an information management method of a target environment and a related augmented reality display method, electronic equipment, storage media, computer programs and computer program products.
  • Augmented Reality (AR) technology is a technology that ingeniously integrates virtual information with the real world. It uses multimedia, 3D modeling, real-time tracking and registration, intelligent interaction, sensing and other technical means. Computer-generated text, images, 3D models, music, video and other virtual information are simulated and applied to the real world. The two kinds of information complement each other, thereby realizing the "enhancement" of the real world.
  • the present disclosure provides a management method of a target environment and a related augmented reality display method.
  • the first aspect of the present disclosure provides an information management method for a target environment.
  • the method includes: acquiring three-dimensional information of the target environment; dividing the three-dimensional information into voxels to obtain multiple target voxels corresponding to the target environment;
  • the attribute information of the target environment corresponding to the target voxel determines the semantic information of each target voxel.
  • the target voxels obtained by dividing the 3D information of the target environment into voxels, and then determining the attribute information of the target environment corresponding to each target voxel as the semantic information of each target voxel, so that the target
  • the semantic information of the voxel can reflect the attribute information of the target environment, realize the integration of the information of the real environment into the semantic information, and improve the sense of reality of the fusion of virtual and real.
  • the above-mentioned attribute information of the target environment includes at least one of spatial attribute information, social attribute information, and object attribute information;
  • the semantic information of the target voxel corresponds to the attribute information of the target environment, and the semantic information of the target voxel includes spatial semantic information At least one of information, social semantic information and object semantic information;
  • the above-mentioned attribute information of the target environment corresponding to each target voxel determines the semantic information of each target voxel, including at least one of the following:
  • the above-mentioned object semantic information of each target voxel is determined according to the object attribute information of the target environment corresponding to each target voxel.
  • the spatial semantic information, social semantic information, and object semantic information of the target voxel can be correspondingly determined.
  • the above spatial attribute information includes real path information and object blocking information; social attribute information includes area category information, object attribute information includes object material information; spatial semantic information includes path semantic information and blocking semantic information, social semantic information includes category semantic information Information, object semantic information includes material semantic information; the above-mentioned spatial attribute information of the target environment corresponding to each target voxel determines the spatial semantic information of each target voxel, including: according to the corresponding to each target voxel The actual path information and object blocking information of the target environment, determine the path semantic information and blocking semantic information of each target voxel; the above-mentioned according to the social attribute information of the target environment corresponding to each target voxel, determine each target voxel The social semantic information of each target voxel, including: according to the area category information of the target environment corresponding to each target voxel, determine the category semantic information of each target voxel; the above-mentioned object attributes according to the target environment corresponding to each target voxel
  • the path semantic information and blocking semantic information of the target voxel can be determined accordingly.
  • the category semantic information of each target voxel can be correspondingly determined.
  • the object material semantic information of each target voxel can be correspondingly determined.
  • determining the path semantic information of each target voxel includes: determining the intersection target voxel at the intersection of different paths in the target environment, based on the intersection The actual path information of different paths determines the path semantic information of the intersecting target voxels.
  • the path semantic information of the intersection target voxel can include the semantic information of the intersection path, which facilitates subsequent path information search.
  • the above three-dimensional information includes point cloud information or grid information; divide the three-dimensional information into voxels to obtain multiple target voxels corresponding to the target environment, including: divide the point cloud information or grid information according to the preset
  • the division method performs voxel division to obtain multiple original voxels corresponding to the target environment. Judging whether each original voxel satisfies the point cloud division requirement or the grid division requirement, and obtains the judgment result; the plurality of original voxels in the described each original voxel in the described judgment result meet the point cloud division requirement or the grid division requirement.
  • the multiple original voxels are used as multiple target voxels corresponding to the target environment.
  • the above-mentioned multiple original voxels as the target voxels corresponding to the target environment include: determining the surrounding original voxels surrounded by other original voxels among the multiple original voxels that meet the requirements; excluding the surrounding original voxels The other original voxels of the voxel serve as multiple target voxels corresponding to the target environment.
  • the above-mentioned original voxels that meet the requirements are used as target voxels corresponding to the target environment, including: dividing the point cloud information or grid information into voxels according to the preset division method, and obtaining multiple voxels corresponding to the target environment.
  • An original voxel determining in the original voxels surrounding original voxels surrounded by other original voxels; excluding the other original voxels surrounding the original voxels as a plurality of target voxels corresponding to the target environment.
  • the storage space occupied can be reduced when storing the target voxels.
  • the information management method of the target environment further includes: obtaining other semantic information related to the target environment; Semantic information is stored in target voxels corresponding to the target environment.
  • the target voxel can store other semantic information associated with the entire target environment.
  • the second aspect of the present disclosure provides an augmented reality display method, the method includes: acquiring the semantic information of the target voxel in the target environment, wherein the semantic information of the target voxel is managed through the information management of the target environment described in the first aspect above Obtained by the method; display semantic information at the corresponding position of the target voxel of the target environment.
  • the third aspect of the present disclosure also provides an augmented reality display method, the method includes: acquiring the semantic information of the target voxel in the environment where the virtual object is located, wherein the semantic information of the target voxel is described by the above first aspect The information management method of the target environment is obtained; based on the semantic information of the target voxel, the corresponding behavior of the virtual object is determined and the corresponding behavior of the virtual object is displayed.
  • the behavior of virtual objects can be made more natural and realistic.
  • the fourth aspect of the present disclosure provides an electronic device, the electronic device includes a processor and a memory coupled to each other, wherein the processor is used to execute the computer program stored in the memory to implement the information management method for the target environment described in the first aspect above, or The augmented reality display method described in the second aspect and the third aspect above.
  • the fifth aspect of the present disclosure provides a computer-readable storage medium, on which program instructions are stored.
  • the program instructions are executed by a processor, the information management method for the target environment described in the first aspect above is implemented, or the second aspect, the first aspect The augmented reality display method described in three aspects.
  • a sixth aspect of the present disclosure provides a computer program, including computer-readable codes.
  • the processor in the electronic device executes to realize the target environment described in the above-mentioned first aspect.
  • the seventh aspect of the present disclosure provides a computer program product, including computer program instructions.
  • the computer program instructions are executed by a computer, the information management method for the target environment described in the first aspect above, or the second aspect or the third aspect above is implemented. Describes the augmented reality display method.
  • the target voxels obtained by dividing the three-dimensional information of the target environment into voxels, and then determining the attribute information of the target environment corresponding to each target voxel as the semantic information of each target voxel, so that
  • the semantic information of the target voxel can reflect the attribute information of the target environment, realize the integration of the information of the real environment into the semantic information, and improve the sense of reality of the fusion of virtual and real.
  • Fig. 1 is a first schematic flow chart of an embodiment of an information management method in a target environment of the present disclosure
  • Fig. 2 is a second schematic flowchart of an embodiment of an information management method of the disclosed target environment
  • Fig. 3 is a schematic flow chart of another embodiment of an information management method of the disclosed target environment
  • FIG. 4 is a schematic flowchart of an embodiment of an augmented reality display method of the present disclosure
  • FIG. 5 is a schematic flow chart of another embodiment of the augmented reality display method of the present disclosure.
  • Fig. 6 is a schematic framework diagram of an embodiment of an information management device in the target environment of the present disclosure
  • Fig. 7 is a schematic frame diagram of an embodiment of an electronic device of the present disclosure.
  • FIG. 8 is a schematic diagram of an embodiment of a computer-readable storage medium of the present disclosure.
  • system and “network” are often used interchangeably herein.
  • the term “and/or” in this article is just an association relationship describing associated objects, which means that there can be three relationships, for example, A and/or B can mean: A exists alone, A and B exist simultaneously, and there exists alone B these three situations.
  • the character "/” in this article generally indicates that the contextual objects are an “or” relationship.
  • “many” herein means two or more than two.
  • FIG. 1 is a schematic flowchart of a first embodiment of an information management method in a target environment of the present disclosure. As shown in Figure 1, the method may include the following steps:
  • Step S11 Obtain 3D information of the target environment.
  • the 3D information of the target environment can be considered as 3D data obtained based on the target environment and capable of expressing the situation of the target environment.
  • the three-dimensional information may be point cloud information established based on the target environment, or grid information, and the like.
  • the target environment can be any environment in the real world.
  • the execution subject of the information management method of the target environment may be an information management device of the target environment, for example, the information management method of the target environment may be executed by a terminal device, or a server or other processing device, wherein the terminal device Can be user equipment, mobile device, user terminal, terminal, cellular phone, personal digital processing, handheld device, computing device, vehicle-mounted device, wearable device, and the like.
  • the method for managing the information of the target caliber can be realized by calling the computer calibration instructions stored in the memory by the processor.
  • the information management apparatus of the target environment is set on the electronic equipment, and the electronic equipment is used as an example for illustration.
  • the acquisition of the three-dimensional information of the target environment by the electronic device may be to collect image information of the target environment, generate a high-precision map based on the image information of the target environment, and then obtain point cloud information of the target environment based on the high-precision map or is grid information and so on.
  • Step S12 Divide the three-dimensional information into voxels to obtain multiple target voxels corresponding to the target environment.
  • the electronic device may divide the three-dimensional information into voxels, so as to obtain multiple target voxels corresponding to the target environment.
  • the method of dividing the three-dimensional information into voxels may be to rasterize the three-dimensional information, or to rasterize the point cloud information, or to rasterize the grid information.
  • Voxels are the smallest unit of semantic information storage. Voxels can store the information of a grid, and can also be used to store attribute information of various target environments such as paths, obstacles, materials, and music.
  • the electronic device can project the grid on the reconstructed dense grid or dense point cloud, and recalculate the height of the grid points, so that the rasterized grid of voxels is more appropriate to the reconstructed grid, realizing the physical world align.
  • Step S13 Determine the semantic information of each target voxel according to the attribute information of the target environment corresponding to each target voxel.
  • the attribute information of the target environment can be considered as information that can be interpreted by humans in the target environment.
  • the known information related to the target environment can be regarded as the attribute information of the target environment.
  • the attribute information of the target environment may include attribute information automatically generated by the system and attribute information determined by human input.
  • the attribute information of the target environment may include at least one of spatial attribute information, social attribute information, and object attribute information.
  • the semantic information of the target voxel may correspond to the attribute information of the target environment, and the semantic information of the target voxel may also include at least one of spatial semantic information, social semantic information and object semantic information.
  • the spatial attribute information may be considered as information characterized by the space where the target environment is located.
  • the spatial attribute information may include spatial information such as the location, size, and area of the square, which region it belongs to, and which road it is connected to.
  • Social attribute information can be considered as various artificially defined information and information related to people in the target environment.
  • the target environment is a dangerous area, a no-noisy area, and so on.
  • Object attribute information can be considered as information represented by objects existing in the target environment. For example, if there is a road paved with gravel in the target environment, the object attribute information may be gravel.
  • the object attribute information may be the material, shape, etc. of the sculpture.
  • the attribute information of the block may also include a transparent material and the like.
  • each target voxel is a part of the target environment, and each target voxel may also be different in the target environment. Therefore, the semantic information of each target voxel may not only include attribute information of the target environment as a whole, but also include specific attribute information of the target environment corresponding to the target voxel. For example, if the target environment is a corridor, the semantic information of all target voxels corresponding to the corridor can include the attribute information of the corridor, and the target voxels corresponding to the corridor on the corridor can also include the semantic information of the corridor passage.
  • the target voxel corresponding to the wall in can also include the semantic information of the corridor wall.
  • the target voxels obtained by dividing the 3D information of the target environment into voxels, and then determining the attribute information of the target environment corresponding to each target voxel as the semantic information of each target voxel, so that the target
  • the semantic information of the voxel can reflect the attribute information of the target environment, realizing the effect of integrating the information of the real environment into the semantic information.
  • FIG. 2 is a second schematic flowchart of an embodiment of an information management method in a target environment of the present disclosure.
  • the three-dimensional information includes point cloud information or grid information of the target environment.
  • the implementation of "dividing the three-dimensional information into voxels to obtain multiple target voxels corresponding to the target environment" mentioned in step S12 may include steps S121 to S123.
  • Step S121 Divide the point cloud information or grid information into voxels according to a preset division method to obtain a plurality of original voxels corresponding to the target environment.
  • the preset division method may be a general rasterization method, which is not limited in the embodiment of the present disclosure.
  • the point cloud information can be a dense point cloud
  • the grid information can be a dense grid
  • the preset division method is an octree division.
  • the electronic device can divide the dense point
  • the cloud or dense grid is divided into the smallest unit of 30cm ⁇ 30cm ⁇ 30cm, so as to obtain multiple original voxels corresponding to the target environment.
  • Step S122 Judging whether each original voxel satisfies the requirement of point cloud division or grid division, and obtains the judgment result.
  • the electronic device needs to judge whether each original voxel meets the requirements of point cloud division or mesh division, and will Raw voxels that do not meet the requirements are excluded.
  • the point cloud division requirement is, for example, that the point cloud information contained in the original voxel is less than a preset threshold
  • the mesh division requirement is, for example, any surface or slice in the original voxel.
  • the original voxels that do not meet the requirements can be considered as voxels that cannot provide enough information, so they need to be excluded.
  • Step S123 When the judging result indicates that the plurality of original voxels meet the requirements of point cloud division or mesh division, the plurality of original voxels are used as the plurality of target voxels corresponding to the target environment.
  • the original voxels that meet the requirements of point cloud division or grid division can be used for the corresponding semantic information determined later, so that the original voxels that meet the requirements can be used as the corresponding target environment. target voxel.
  • the realization of "using multiple original voxels as multiple target voxels corresponding to the target environment" mentioned in step S123 above may include: step S1231 and step S1232.
  • Step S1231 Determine the surrounding original voxels surrounded by other original voxels among the original voxels meeting the requirements.
  • Step S1232 Exclude other original voxels surrounding the original voxel as target voxels corresponding to the target environment.
  • the original voxel may not be stored, that is, the surrounding original voxel may not be used as the target voxel.
  • the surrounding original voxels may also be finally used as the target voxels.
  • step S123 of "determining whether each original voxel meets the requirement of point cloud division or grid division” it is also possible to first determine in the original voxel which is replaced by other original voxels The surrounding original voxels are surrounded, and then other original voxels except the surrounding original voxels are determined as the original voxels for judging whether they meet the requirements of point cloud division or grid division. That is, in some embodiments of the present disclosure, the enclosing original voxels will not be used for subsequent determination of point cloud division requirements or grid division requirements.
  • the number of original voxels that need to be judged whether point cloud division or grid division is required can be reduced, which helps to improve the speed of the information management method for the target environment of the present disclosure, and the surrounding original voxels are not used for Subsequent judgment means that the surrounding original voxels will not be used as target voxels, so that when storing the target voxels, the storage space can be reduced.
  • the implementation of "dividing the three-dimensional information into voxels to obtain multiple target voxels corresponding to the target environment" mentioned in step S12 may include steps S124 to S126.
  • Step S124 Divide the point cloud information or grid information into voxels according to a preset division method to obtain a plurality of original voxels corresponding to the target environment.
  • step S121 For a detailed description of this step, please refer to the above step S121, which will not be repeated here.
  • Step S125 Determine the surrounding original voxels surrounded by other original voxels in the original voxels.
  • the enclosing original voxels surrounded by other original voxels that is, all six faces of the voxels are connected with other original voxels.
  • Step S126 Exclude other original voxels surrounding the original voxel as multiple target voxels corresponding to the target environment.
  • the surrounding original voxels may not be seen by the user in the subsequent augmented reality display, the surrounding original voxels may not be used as the target voxels, so that when storing the target voxels, the Reduce storage space usage.
  • the implementation of "determining the semantic information of each target voxel according to the attribute information of the target environment corresponding to each target voxel" mentioned in step S13 may include: The spatial attribute information of the target environment corresponding to each target voxel is determined to determine the spatial semantic information of each target voxel.
  • the space attribute information of the target environment may be information characterized by the space where the target environment is located.
  • the target environment is a road, and there is a shop on the side of the road; the target environment corresponding to the target voxel is the wall of the shop, then based on the space where the wall is located, it can be determined that the spatial semantic information of the target voxel is a road The wall of a store.
  • the spatial attribute information may include real path information and object blocking information; correspondingly, the spatial semantic information may include path semantic information and blocking semantic information.
  • the real path information can be considered as the relevant road and passage information of the target environment; then the path semantic information can be regarded as the semantic information that can represent the road information and passage information in the real environment, and the path semantic information is, for example, a target The road to which the target environment corresponding to the voxel belongs, what roads the road is connected to, the destinations it can lead to, and so on.
  • a value of 1 represents a road network voxel, so how the road network is connected can be determined according to the value of a voxel being 1.
  • Electronic devices can connect adjacent voxels. For example, you can select the intersection of two road networks and connect the two intersections. All voxels that intersect with the two intersections will set corresponding road network information. Such as 1. In this way, compared with the way of direct wire connection, in some search examples, it has a better guiding effect.
  • object blocking information may be considered as information on whether an object will collide when passing through the space of the target environment corresponding to the target voxel. For example, if the space where the target environment corresponding to a certain target voxel is located is a fire hydrant on the side of the road, it can be considered that passing through this space will collide with the fire hydrant. Then the corresponding blocking semantic information may be that there is a blocking situation in this area. For another example, when a virtual entity walks in the real space, it needs to obtain information about its surroundings, such as where there is no obstacle, it can walk, and it is safe; or, where there is an obstacle, it cannot walk, or it can jump on it.
  • the electronic device can determine blocking information according to the voxel values.
  • the "determining the spatial semantic information of each target voxel according to the spatial attribute information of the target environment corresponding to each target voxel" mentioned in step S13 may include: The realistic path information and object blocking information of the environment determine the path semantic information and blocking semantic information of each target voxel.
  • the path semantic information and blocking semantic information of the target voxel can be determined accordingly.
  • the connecting intersections of different roads and passages may be considered as a part of any one of the connected roads and passages, respectively.
  • the intersection of road A and road B can be considered as part of road A, or part of road B.
  • passage A is a hall corridor
  • passage B is a stair passage
  • the intersection of passage A and passage B can be considered as a part of passage A or passage B.
  • the target voxel at the intersection of different paths it is defined as the intersection target voxel.
  • the step of determining its path semantic information may include: determining the intersection target voxel at the intersection of different paths in the target environment, and determining the path semantic information of the intersection target voxel based on the actual path information of different paths at the intersection . Exemplarily, it is determined which roads the intersection of the path where the intersection target voxel is located can belong to, and then use the path semantic information of these roads as the path semantic information of the intersection target voxel.
  • the path semantic information of the intersection target voxel can include the semantic information of the intersection path, which facilitates subsequent path information search.
  • the implementation of "determining the semantic information of each target voxel according to the attribute information of the target environment corresponding to each target voxel" mentioned in the above steps may include:
  • the social attribute information of the target environment corresponding to the voxel determines the social semantic information of each target voxel. For example, if the target environment corresponding to the target voxel is a designated bicycle parking area, then the social attribute information represented by the target environment is the bicycle parking area, and the corresponding social semantic information is also the bicycle parking area. For another example, if the target environment corresponding to the target voxel is the reading room of the library, then the social attribute information of the target environment is the reading room, and the corresponding social semantic information is also the reading area and so on.
  • the social semantic information of each target voxel can be correspondingly determined.
  • the voxel division can make the information edited by the electronic device be based on three-dimensional space information, so that it can have a more spatial structure and make the fusion of virtual and real more realistic.
  • the social attribute information includes area category information.
  • Social semantic information includes category semantic information.
  • the category semantic information of each target voxel may be determined according to the area category information of the target environment corresponding to each target voxel.
  • the target environment can be divided into different areas according to different criteria. For example, from the perspective of being able to speak, it can be divided into a quiet area, a speaking area, and so on. From the perspective of passers-by walking safety, it can be divided into careful walking area, forbidden walking area, free passage area and so on.
  • the classification standard can be set in advance, which is not limited here.
  • the area category information is the type to which the target environment belongs.
  • the category semantic information may also be type information to which the target environment belongs.
  • the category semantic information of a target voxel may also be category semantic information including multiple different classification standards. Therefore, according to the area category information of the target environment corresponding to each target voxel, the category semantic information of each target voxel can be determined accordingly.
  • the implementation of "determining the semantic information of each target voxel according to the attribute information of the target environment corresponding to each target voxel" mentioned in the above steps may include: The object attribute information of the target environment corresponding to the voxel determines the object semantic information of each target voxel.
  • the object attribute information of the target environment may be information characterized by target objects existing in the target environment.
  • the target object may be any object in the target environment, and the object attribute information of the target object may include various information related to the target object.
  • the information represented by the car may be the size, brand, size, etc. of the car.
  • the corresponding object semantic information may also be various information related to the target object. Therefore, by determining the object attribute information of the target environment corresponding to each target voxel, the object semantic information of each target voxel can be determined accordingly.
  • the object attribute information includes object material information.
  • Object semantic information includes object material semantic information.
  • the electronic device may determine the object material semantic information of each target voxel according to the object material information of the target environment corresponding to each target voxel.
  • the object material information includes the constituent materials of the object, for example, the material of the surface of the object.
  • the target object is a large plush doll, the surface of which is plush. Then it can be determined that the object material information of the plush doll can be that the surface is fluff, and then the corresponding object material semantic information can be the surface of fluff.
  • the object material information may be sand, and the corresponding object material semantic information is the sand surface. Therefore, by determining the object material information of the target environment corresponding to each target voxel, the object material semantic information of each target voxel can be correspondingly determined.
  • the object material information is stored in voxels, so that the electronic device can determine the object material information and the object semantic material information according to the voxel value.
  • FIG. 3 is a schematic flowchart of another embodiment of an information management method in a target environment of the present disclosure.
  • the information management method of the target environment in the embodiments of the present disclosure can also be Including: step S21 and step S22.
  • Step S21 Obtain other semantic information related to the target environment.
  • other semantic information may be other semantic information related to the target environment as a whole.
  • other semantic information may also be other semantic information of the tourist attraction.
  • Other semantic information can be determined manually or automatically matched by the system, and the determination method is not limited.
  • Step S22 Store other semantic information in the target voxel corresponding to the target environment.
  • the target voxel is obtained based on the target environment, and the electronic device can store other semantic information in the target voxel corresponding to the target environment, so that the target voxel can store information related to the entire target environment Other associated semantic information.
  • other semantic information may include vectorization information; for the target environment, the electronic device may associate the vectorization information with the voxels of the target environment; thus, the electronic device may combine the vectorization information of the road network related to the target environment Stored together in the overall structure of the target environment.
  • FIG. 4 is a schematic flowchart of an embodiment of an augmented reality display method of the present disclosure.
  • the augmented reality display method includes the following steps:
  • Step S31 Obtain the semantic information of the target voxel in the target environment.
  • the device that executes the steps of the augmented reality display method is, for example, a mobile phone, AR glasses and other devices.
  • the electronic device can obtain the image of the surrounding environment, and based on image recognition technology, match the environment in the image with the established target environment, and then obtain the semantic information of the target voxel in the target environment .
  • the semantic information of the target voxel can be the semantic information obtained through the embodiment of the above-mentioned target environment information management method.
  • Step S32 Display semantic information at the corresponding position of the target voxel in the target environment.
  • the electronic device can display the semantic information at the corresponding position of the target voxel in the target environment.
  • FIG. 5 is a schematic flowchart of another embodiment of an augmented reality display method according to the present disclosure.
  • the augmented reality display method includes the following steps:
  • Step S41 Obtain semantic information of target voxels in the environment where the virtual object is located.
  • the virtual object may be a virtual object simulated by an electronic device, for example, a virtual character simulated by a mobile phone executing the augmented reality display method of the present disclosure.
  • the electronic device after the electronic device generates the virtual object, it can acquire the semantic information of the target voxel in the environment where the virtual object is located, so that the virtual object can determine its environment through the obtained semantic information.
  • the virtual object is placed in the space, and the electronic device can obtain the surrounding voxels and the global voxels of the positioning position according to the positioning position of the virtual object.
  • Semantic information According to these semantic information, electrical equipment can perform sound and visual display in the fusion of virtual and real, and realize intelligent behaviors such as wayfinding, obstacle avoidance, and area reminders in the space (such as shopping mall toilet reminders, escalator safety reminders), and improve virtual-real fusion. Effect.
  • Step S42 Based on the semantic information of the target voxel, determine the corresponding behavior of the virtual object.
  • controlling the behavior of the virtual object is to determine the corresponding relationship between the semantic information of the target voxel and the preset response of the virtual object, so that the virtual object can target the target voxel It responds to the semantic information of pixels, making virtual objects more realistic.
  • Step S43 Display the corresponding behavior of the virtual object.
  • the corresponding behavior of the virtual object is a response to the target environment
  • the corresponding behavior of the virtual object can be displayed so that the corresponding behavior can be perceived by the user.
  • the virtual object is a virtual character
  • the target environment includes tiled ground and sandy ground, and a sculpture stands on the sandy ground
  • the semantic information of the target voxel in the target environment includes tiled ground, Sandy ground, sculptures.
  • the behavior of the virtual object can be controlled according to the semantic information of the gravel road.
  • the virtual character walks in front of the sculpture, the virtual character can be controlled to bypass, because the sculpture can block semantic information.
  • the virtual character walks on the tile floor, the virtual character can be controlled to walk slowly because the tile floor is slippery.
  • the behavior of virtual objects can be made more natural and realistic.
  • FIG. 6 is a schematic framework diagram of an embodiment of an information management device in a target environment provided by an embodiment of the present disclosure.
  • the information management device 60 includes an acquisition section 61 , a division section 62 and a determination section 63 .
  • the acquiring part 61 is configured to acquire the three-dimensional information of the target environment;
  • the dividing part 62 is configured to divide the three-dimensional information into voxels to obtain multiple target voxels corresponding to the target environment;
  • the determining module 63 is configured to The attribute information of the target environment corresponding to each target voxel determines the semantic information of each target voxel.
  • the above-mentioned attribute information of the target environment includes at least one of spatial attribute information, social attribute information and object attribute information;
  • the semantic information of the target voxel corresponds to the attribute information of the target environment, including spatial semantic information, social semantic information and object semantic information At least one of the information;
  • the above-mentioned determination part 63 is configured to determine the semantic information of each target voxel according to the attribute information of the target environment corresponding to each target voxel, including at least one of the following: according to each target voxel
  • the spatial attribute information of the target environment corresponding to the target voxel determines the spatial semantic information of each target voxel; according to the social attribute information of the target environment corresponding to each target voxel, determines the social semantic information of each target voxel;
  • the object semantic information of each target voxel is determined according to the object attribute information of the target environment corresponding to each target voxel.
  • the above spatial attribute information includes real path information and object blocking information; social attribute information includes area category information, object attribute information includes object material information; spatial semantic information includes path semantic information and blocking semantic information, social semantic information includes category semantic information Information, object semantic information includes material semantic information; the above-mentioned determining part 63 is configured to determine the spatial semantic information of each target voxel according to the spatial attribute information of the target environment corresponding to each target voxel, including: The actual path information and object blocking information of the target environment corresponding to each target voxel determine the path semantic information and blocking semantic information of each target voxel; the above-mentioned determination part 63 is configured to correspond to each target voxel The social attribute information of the target environment, determine the social semantic information of each target voxel, including: according to the regional category information of the target environment corresponding to each target voxel, determine the category semantic information of each target voxel; the above The determination part 63 is configured to determine the object semantic information of each target
  • the above-mentioned determining part 63 is configured to determine the path semantic information of each target voxel according to the actual path information of the target environment corresponding to each target voxel, including: determining the intersection of different paths in the target environment The target voxel determines the path semantic information of the intersection target voxel based on the actual path information of different paths at the intersection.
  • the above-mentioned three-dimensional information includes point cloud information or grid information
  • the above-mentioned division part 62 is configured to divide the three-dimensional information into voxels to obtain multiple target voxels corresponding to the target environment, including: dividing the point cloud Information or grid information is divided into voxels according to the preset division method to obtain multiple original voxels corresponding to the target environment; judge whether each original voxel meets the requirements of point cloud division or grid division, and obtain the judgment result; When the judgment result indicates that a plurality of original voxels in each original voxel meet the requirements of point cloud division or grid division, the plurality of original voxels are used as a plurality of target volumes corresponding to the target environment white.
  • the above-mentioned dividing part 62 is configured to use the original voxels that meet the requirements as a plurality of target voxels corresponding to the target environment, including: determining the surrounding original voxels surrounded by other original voxels among the original voxels that meet the requirements voxels; will exclude other original voxels surrounding the original voxel as multiple target voxels corresponding to the target environment.
  • the above-mentioned division part 62 is configured to divide the three-dimensional information into voxels to obtain a plurality of target voxels corresponding to the target environment, including: dividing point cloud information or grid information into voxels according to a preset division method. Divide to obtain multiple original voxels corresponding to the target environment; determine the surrounding original voxels surrounded by other original voxels in the original voxels; exclude other original voxels surrounding the original voxels as multiple original voxels corresponding to the target environment target voxels.
  • the above-mentioned information management device 60 also includes a semantic information acquisition part, after the determination part 63 is configured to determine the semantic information of each target voxel according to the attribute information of the target environment corresponding to each target voxel, the semantic information
  • the obtaining part is configured to obtain other semantic information related to the target environment; and store the other semantic information in the target voxel corresponding to the target environment.
  • the present disclosure also discloses an augmented reality display device, which includes a first acquisition part and a first display part.
  • the first acquiring part is configured to acquire the semantic information of the target voxel in the target environment, wherein the semantic information of the target voxel is obtained through the above embodiment of the information management method for the target environment.
  • the first display part is configured to display semantic information at the corresponding position of the target voxel in the target environment.
  • the present disclosure also discloses another augmented reality display device, which includes a second acquisition part, a determination part and a second display part.
  • the second acquisition part is configured to acquire the semantic information of the target voxel in the environment where the virtual object is located, wherein the semantic information of the target voxel is obtained through the above embodiment of the information management method for the target environment.
  • the determination part is configured to determine the corresponding behavior of the virtual object based on the semantic information of the target voxel; the second display part is configured to display the corresponding behavior of the virtual object.
  • a "part" may be a part of a circuit, a part of a processor, a part of a program or software, etc., of course it may also be a unit, a module or a non-modular one.
  • FIG. 7 is a schematic frame diagram of an embodiment of the electronic device of the present disclosure.
  • the electronic device 70 includes a memory 71 and a processor 72 coupled to each other, and the processor 72 is used to execute the program instructions stored in the memory 71, so as to realize the information management method of any of the above-mentioned target environments or the display method of augmented reality. step.
  • the electronic device 70 may include, but is not limited to: a microcomputer and a server.
  • the electronic device 70 may also include mobile devices such as notebook computers and tablet computers, which are not limited here.
  • the processor 72 is used to control itself and the memory 71 to implement the steps of an embodiment of an information management method for any target environment or an augmented reality display method.
  • the processor 72 may also be called a CPU (Central Processing Unit, central processing unit).
  • the processor 72 may be an integrated circuit chip with signal processing capability.
  • the processor 72 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
  • a general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.
  • the processor 72 may be jointly implemented by an integrated circuit chip.
  • FIG. 8 is a schematic framework diagram of an embodiment of a computer-readable storage medium of the present disclosure.
  • the computer-readable storage medium 80 stores program instructions 81 that can be executed by the processor, and the program instructions 81 are used to implement the steps of any one of the above target environment information management methods or augmented reality display method embodiments.
  • the electronic device obtains the target voxels obtained by dividing the three-dimensional information of the target environment into voxels, and then determines the attribute information of the target environment corresponding to each target voxel as the semantic information of each target voxel , so that the semantic information of the target voxel can reflect the attribute information of the target environment, and realize the effect of integrating the information of the real environment into the semantic information.
  • the present disclosure also discloses a computer program, including computer readable codes.
  • the processor in the electronic device executes the information management method or augmented reality of any of the above target environments. The steps of the display method embodiment.
  • the present disclosure also discloses a computer program product, including computer program instructions.
  • the computer program instructions When the computer program instructions are executed by a computer, the steps of any of the above-mentioned target environment information management methods or augmented reality display method embodiments are realized.
  • This disclosure relates to the field of augmented reality.
  • acquiring the image information of the target object in the real environment and then using various visual correlation algorithms to detect or identify the relevant features, states and attributes of the target object, and thus obtain the image information that matches the specific application.
  • AR effect combining virtual and reality.
  • the target object may involve faces, limbs, gestures, actions, etc. related to the human body, or markers and markers related to objects, or sand tables, display areas or display items related to venues or places.
  • Vision-related algorithms can involve visual positioning, SLAM, 3D reconstruction, image registration, background segmentation, object key point extraction and tracking, object pose or depth detection, etc.
  • Specific applications can not only involve interactive scenes such as guided tours, navigation, explanations, reconstructions, virtual effect overlays and display related to real scenes or objects, but also special effects processing related to people, such as makeup beautification, body beautification, special effect display, virtual Interactive scenarios such as model display.
  • the relevant features, states and attributes of the target object can be detected or identified through the convolutional neural network.
  • the above-mentioned convolutional neural network is a network model obtained by performing model training based on a deep learning framework.
  • the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the methods described in the method embodiments above, and its specific implementation can refer to the description of the method embodiments above. For brevity, here No longer.
  • the disclosed methods and devices may be implemented in other ways.
  • the device implementations described above are only illustrative.
  • the division of modules or units is only a logical function division. In actual implementation, there may be other division methods.
  • units or components can be combined or integrated. to another system, or some features may be ignored, or not implemented.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.
  • a unit described as a separate component may or may not be physically separated, and a component shown as a unit may or may not be a physical unit, that is, it may be located in one place, or may also be distributed to network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.
  • the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.
  • the technical solution of the present disclosure is essentially or part of the contribution to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) execute all or part of the steps of the methods in various embodiments of the present disclosure.
  • the aforementioned storage medium includes: a tangible device capable of holding and storing instructions used by the instruction execution device, which may be a volatile storage medium or a non-volatile storage medium.
  • a computer readable storage medium may be, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.
  • Computer-readable storage media include: portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or flash memory), static random access memory (SRAM), compact disc read only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanically encoded device, such as a printer with instructions stored thereon A hole card or a raised structure in a groove, and any suitable combination of the above.
  • RAM random access memory
  • ROM read-only memory
  • EPROM erasable programmable read-only memory
  • flash memory static random access memory
  • SRAM static random access memory
  • CD-ROM compact disc read only memory
  • DVD digital versatile disc
  • memory stick floppy disk
  • mechanically encoded device such as a printer with instructions stored thereon
  • a hole card or a raised structure in a groove and any suitable combination of the above.
  • computer-readable storage media are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., pulses of light through fiber optic cables), or transmitted electrical signals.
  • the target voxels obtained by dividing the three-dimensional information of the target environment into voxels, and then determining the attribute information of the target environment corresponding to each target voxel as the semantics of each target voxel information, so that the semantic information of the target voxel can reflect the attribute information of the target environment, and realize the integration of the information of the real environment into the semantic information, thereby improving the realism of virtual fusion.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • Geometry (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Processing Or Creating Images (AREA)

Abstract

一种目标环境的信息管理方法及相关增强现实的显示方法，目标环境的信息管理方法包括：获取目标环境的三维信息；将三维信息进行体素划分，得到与目标环境对应的多个目标体素；根据与每个目标体素对应的目标环境的属性信息，确定每个目标体素的语义信息。通过该方法，可以将真实环境的信息融入到语义信息，提高虚实融合的真实感。

Description

目标环境的信息管理方法及相关增强现实的显示方法、电子设备、存储介质、计算机程序和计算机程序产品
相关申请的交叉引用
本公开基于申请号为202111056910.8、申请日为2021年9月9日、申请名称为“目标环境的信息管理方法及相关增强现实的显示方法”的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此引入本公开作为参考。
技术领域
本公开涉及增强现实技术领域，特别是涉及一种目标环境的信息管理方法及相关增强现实的显示方法、电子设备、存储介质、计算机程序和计算机程序产品。
背景技术
增强现实(Augmented Reality，AR)技术是一种将虚拟信息与真实世界巧妙融合的技术，广泛运用了多媒体、三维建模、实时跟踪及注册、智能交互、传感等多种技术手段，通过将计算机生成的文字、图像、三维模型、音乐、视频等虚拟信息模拟仿真后，应用到真实世界中，两种信息互为补充，从而实现对真实世界的“增强”。
然而，由于对语义信息的数字化不足，目前各种增强现实的虚实融合通常真实感较低。
发明内容
本公开提供一种目标环境的管理方法及相关增强现实的显示方法。
本公开第一方面提供了一种目标环境的信息管理方法，方法包括：获取目标环境的三维信息；将三维信息进行体素划分，得到与目标环境对应的多个目标体素；根据与每个目标体素对应的目标环境的属性信息，确定每个目标体素的语义信息。
因此，通过获取将目标环境的三维信息进行体素划分后得到的目标体素，再将与每个目标体素对应的目标环境的属性信息，确定为每个目标体素的语义信息，使得目标体素的语义信息能够反映目标环境的属性信息，实现了将真实环境的信息融入到语义信息之中，提高了虚实融合的真实感。
其中，上述的目标环境的属性信息包括空间属性信息、社会属性信息和物体属性信息中的至少一个；目标体素的语义信息对应目标环境的属性信息，所述目标体素的语义信息包括空间语义信息、社会语义信息和物体语义信息中的至少一个；上述的根据与每个目标体素对应的目标环境的属性信息，确定每个目标体素的语义信息，包括以下至少一种：根据与每个目标体素对应的 目标环境的空间属性信息，确定每个目标体素的空间语义信息上述的根据与每个目标体素对应的目标环境的社会属性信息，确定每个目标体素的社会语义信息；上述的根据与每个目标体素对应的目标环境的物体属性信息，确定每个目标体素的物体语义信息。
因此，通过分别确定目标环境的属性信息中的空间属性信息、社会属性信息和物体属性信息，以此能够对应确定目标体素的空间语义信息、社会语义信息和物体语义信息。
其中，上述的空间属性信息包括现实路径信息和物体阻挡信息；社会属性信息包括区域类别信息，物体属性信息包括物体材质信息；空间语义信息包括路径语义信息和阻挡语义信息，社会语义信息包括类别语义信息，物体语义信息包括材质语义信息；上述的根据与每个目标体素对应的目标环境的空间属性信息，确定每个目标体素的空间语义信息，包括：根据与每个目标体素对应的目标环境的现实路径信息和物体阻挡信息，确定每个目标体素的路径语义信息和阻挡语义信息；上述的根据与每个目标体素对应的目标环境的社会属性信息，确定每个目标体素的社会语义信息，包括：根据与每个目标体素对应的目标环境的区域类别信息，确定每个目标体素的类别语义信息；上述的根据与每个目标体素对应的目标环境的物体属性信息，确定每个目标体素的物体语义信息，包括：根据与每个目标体素对应的目标环境的物体材质信息，确定每个目标体素的物体材质语义信息。
因此，通过确定目标体素对应的目标环境的现实路径信息和物体阻挡信息，可以以此确定目标体素的路径语义信息和阻挡语义信息。此外，通过据与每个目标体素对应的目标环境的区域类别信息，可以相应确定每个目标体素的类别语义信息。另外，通过确定与每个目标体素对应的目标环境的物体材质信息，可以相应确定每个目标体素的物体材质语义信息。
其中，上述的根据与每个目标体素对应的目标环境的现实路径信息，确定每个目标体素的路径语义信息，包括：确定目标环境中不同路径的交汇处的交汇目标体素，基于交汇处不同路径的现实路径信息确定交汇目标体素的路径语义信息。
因此，通过基于交汇处不同路径的现实路径信息确定交汇目标体素的路径语义信息，使得交汇目标体素的路径语义信息能够包含交汇的路径的语义信息，方便后续的路径信息的查找。
其中，上述的三维信息包括点云信息或者是网格信息；将三维信息进行体素划分，得到与目标环境对应的多个目标体素，包括：将点云信息或者是网格信息按照预设划分方法进行体素划分，得到与目标环境对应的多个原始体素。判断每一个原始体素是否满足点云划分要求或者是网格划分要求，得到判断结果；在所述判断结果表征所述每一个原始体素中的多个原始体素满足点云划分要求或者网格划分要求的情况下，将所述多个原始体素作为与目标环境对应的多个目标体素。
因此，通过确定每一个原始体素是否满足点云划分要求或者是网格划分 要求，可以将不满足要求的原始体素排除在外，以此可以减少目标体素的数量，减少存储空间的占用。
其中，上述的将多个原始体素作为与目标环境对应的目标体素，包括：在满足要求的多个原始体素中确定被其他原始体素包围的包围原始体素；将排除包围原始体素的其他原始体素作为与目标环境对应的多个目标体素。
通过将排除包围原始体素的其他原始体素作为与目标环境对应的目标体素，在存储目标体素时，可以减少存储空间的占用。
其中，上述的将满足要求的原始体素作为与目标环境对应的目标体素，包括：将点云信息或者是网格信息按照预设划分方法进行体素划分，得到与目标环境对应的多个原始体素；在原始体素中确定被其他原始体素包围的包围原始体素；将排除包围原始体素的其他原始体素作为与目标环境对应的多个目标体素。
因此，通过不将包围原始体素作为目标体素，使得在存储目标体素时，可以减少存储空间的占用。
其中，在根据与每个目标体素对应的目标环境的属性信息，确定每个目标体素的语义信息之后，目标环境的信息管理方法还包括：获取与目标环境相关的其他语义信息；将其他语义信息存储在与目标环境对应的目标体素中。
因此，通过将其他语义信息存储在与目标环境对应的目标体素中，以此使得目标体素能够存储包含有与整个目标环境关联的其他语义信息。
本公开第二方面提供了一种增强现实的显示方法，方法包括：获取目标环境中目标体素的语义信息，其中，目标体素的语义信息是通过上述第一方面描述的目标环境的信息管理方法得到的；在目标环境的目标体素的对应位置显示语义信息。
因此，通过在目标环境的目标体素的对应位置显示语义信息，可以方便用户通过查看语义信息，快速了解目标环境。
本公开第三方面还提供了一种增强现实的显示方法，方法包括：获取虚拟物体所处环境中的目标体素的语义信息，其中，目标体素的语义信息是通过上述第一方面描述的目标环境的信息管理方法得到的；基于目标体素的语义信息，确定虚拟物体的对应行为显示虚拟物体的对应行为。
因此，通过基于目标体素的语义信息来控制虚拟物体的行为，可以使得虚拟物体的行为更加自然和逼真。
本公开第四方面提供了电子设备，该电子设备包括相互耦接的处理器和存储器，其中，处理器用于执行存储器存储的计算机程序以实现上述第一方面描述目标环境的信息管理方法，或者是上述第二方面、第三方面描述的增强现实的显示方法。
本公开第五方面提供了一种计算机可读存储介质，其上存储有程序指令，程序指令被处理器执行时实现上述第一方面描述目标环境的信息管理方法，或者是上述第二方面、第三方面描述的增强现实的显示方法。
本公开第六方面提供了一种计算机程序，包括计算机可读代码，当所述 计算机可读代码在电子设备中运行时，所述电子设备中的处理器执行时实现上述第一方面描述目标环境的信息管理方法，或者是上述第二方面、第三方面描述的增强现实的显示方法。
本公开第七方面提供了一种计算机程序产品，包括计算机程序指令，所述计算机程序指令被计算机执行时实现上述第一方面描述目标环境的信息管理方法，或者是上述第二方面、第三方面描述的增强现实的显示方法。
上述方案，通过获取将目标环境的三维信息进行体素划分后得到的目标体素，再将与每个目标体素对应的目标环境的属性信息，确定为每个目标体素的语义信息，使得目标体素的语义信息能够反映目标环境的属性信息，实现了将真实环境的信息融入到语义信息之中，提高了虚实融合的真实感。
附图说明
图1是本公开目标环境的信息管理方法一实施例的第一流程示意图；
图2是本公开目标环境的信息管理方法一实施例的第二流程示意图；
图3是本公开目标环境的信息管理方法另一实施例的流程示意图；
图4是本公开增强现实的显示方法一实施例的流程示意图；
图5是本公开增强现实的显示方法另一实施例的流程示意图；
图6是本公开目标环境的信息管理装置一实施例的框架示意图；
图7是本公开电子设备一实施例的框架示意图；
图8为本公开计算机可读存储介质一实施例的框架示意图。
具体实施方式
下面结合说明书附图，对本公开实施例的方案进行详细说明。
以下描述中，为了说明而不是为了限定，提出了诸如特定系统结构、接口、技术之类的具体细节，以便透彻理解本公开。
本文中术语“系统”和“网络”在本文中常被可互换使用。本文中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中字符“/”，一般表示前后关联对象是一种“或”的关系。此外，本文中的“多”表示两个或者多于两个。
请参阅图1，图1是本公开目标环境的信息管理方法一实施例的第一流程示意图。如图1所示，该方法可以包括如下步骤：
步骤S11：获取目标环境的三维信息。
目标环境的三维信息，可以认为是基于目标环境得到，能够表达目标环境情况的三维数据。示例性的，三维信息可以是基于目标环境建立的点云信息，或者网格信息等等。目标环境可以是真实世界中的任一环境。
在本公开实施例中，目标环境的信息管理方法的执行主体可以为目标环境的信息管理装置，例如，目标环境的信息管理方法可以由终端设备、或服务器或其他处理设备执行，其中，终端设备可以为用户设备、移动设备、用 户终端、终端、蜂窝电话、个人数字处理、手持设备、计算设备、车载设备、可穿戴设备等。在一些可能的实现方式中，该目标管径的信息管理方法可以通过处理器调用存储器中存储的计算机刻度指令的方式来实现。下面，以目标环境的信息管理装置设置在电子设备上，将电子设备作为执行主体为例来说明。
在一些可能的实施方式中，电子设备获取目标环境的三维信息可以是对目标环境进行图像信息采集，根据目标环境的图像信息生成高精度地图，然后基于高精度地图得到目标环境的点云信息或者是网格信息等等。
步骤S12：将三维信息进行体素划分，得到与目标环境对应的多个目标体素。
在本公开实施例中，电子设备在得到目标环境的三维信息以后，可以将三维信息进行体素划分，以此得到目标环境对应的多个目标体素。其中，将三维信息进行体素划分的方式可以是对三维信息进行栅格化，也可以是将点云信息栅格化，还可以是将网格信息栅格化，对此，本公开实施例不作限制。
需要说明的是，体素是一个语义信息存储的最小单元。体素可以存储一个网格的信息，也可以用来存储路径、阻挡、材质和音乐等等各类目标环境的属性信息。
示例性的，电子设备可以将网格在重建的稠密网格或者稠密点云进行投影，重新计算网格点的高度，使体素的栅格化网格和重建网格比较贴切，实现物理世界对齐。
步骤S13：根据与每个目标体素对应的目标环境的属性信息，确定每个目标体素的语义信息。
目标环境的属性信息可以认为是目标环境中能够被人所解读的信息。这里，与目标环境相关的，能够为人所知的信息，都可以认为是目标环境的属性信息。在一些可能的实施方式中，目标环境的属性信息，可以包括系统自动生成的属性信息和人为输入而确定的属性信息。
在本公开的一些实施例中，目标环境的属性信息可以包括空间属性信息、社会属性信息和物体属性信息中的至少一个。目标体素的语义信息可以对应于目标环境的属性信息，目标体素的语义信息也可以包括空间语义信息、社会语义信息和物体语义信息中的至少一个。
在本公开实施例中，空间属性信息可以认为是目标环境所处的空间所表征的信息。例如，目标环境是某个广场，则空间属性信息可以是该广场所处的位置、大小、面积，属于哪一个地区，与哪一条道路连接等等空间上的信息。社会属性信息可以认为是属于目标环境中各种人为定义的信息、与人相关的信息等。例如，目标环境是危险区域，是禁止吵闹区域等等。物体属性信息可以认为是目标环境中存在的物体所表征的信息。例如，目标环境中存在由碎石铺就的道路，则物体属性信息可以是碎石。又如，目标环境中存在一个雕塑，则物体属性信息可以是雕塑的材质、形状等等。再如，目标环境中存在一个透明方块，则该方块的属性信息还可以包括透明材质等等。
在本公开实施例中个，因为每一个目标体素是目标环境的一部分，且每一个目标体素在目标环境中也可能会各自区别。因此，每一个目标体素的语义信息，既可以包含目标环境整体的属性信息，还可以包含该目标体素对应的目标环境的特定的属性信息。例如，目标环境为走廊，则与该走廊对应的全部目标体素的语义信息都可以包含走廊的属性信息，对于走廊上的通道对应的目标体素则还可以包括走廊通道的语义信息，对于走廊的墙壁对应的目标体素则还可以包括走廊墙壁的语义信息。
因此，通过获取将目标环境的三维信息进行体素划分后得到的目标体素，再将与每个目标体素对应的目标环境的属性信息，确定为每个目标体素的语义信息，使得目标体素的语义信息能够反映目标环境的属性信息，实现了将真实环境的信息融入到语义信息之中的效果。
参阅图2，图2是本公开目标环境的信息管理方法一实施例的第二流程示意图。在本公开实施例中，三维信息包括目标环境的点云信息或者是网格信息。上述步骤S12提及的“将三维信息进行体素划分，得到与目标环境对应的多个目标体素”的实现，可以包括步骤S121至步骤S123。
步骤S121：将点云信息或者网格信息按照预设划分方法进行体素划分，得到与目标环境对应的多个原始体素。
在本公开实施例中，预设划分方法可以是通用的栅格化方法，本公开实施例不做限制。
示例性的，点云信息可以为稠密点云，网格信息可以为稠密网格，预设划分方法为八叉树划分，基于预先设置好的体素尺度，例如30cm，电子设备可以将稠密点云或稠密网格划分为30cm×30cm×30cm的最小单元，从而得到与目标环境对应的多个原始体素。
步骤S122：判断每一个原始体素是否满足点云划分要求或者是网格划分要求，得到判断结果。
在本公开实施例中，对于得到的每一个原始体素，可能存在部分体素不满足要求，因此，电子设备需要判断每一个原始体素是否满足点云划分要求或者是网格划分要求，将不满足要求的原始体素排除。其中，点云划分要求例如是原始体素中包含的点云信息少于预设阈值，网格划分要求例如是原始体素中任何的面、片。不满足要求的原始体素，可以认为是无法提供足够的信息的体素，因此需要将其排除。
步骤S123：在判断结果表征多个原始体素满足点云划分要求或者网格划分要求的情况下，将多个原始体素作为与目标环境对应的多个目标体素。
在本公开实施例中，满足点云划分要求或者网格划分要求的原始体素，可以用于后续确定的其对应的语义信息，以此可以将满足要求的原始体素作为与目标环境对应的目标体素。
因此，通过确定每一个原始体素是否满足点云划分要求或者是网格划分要求，可以将不满足要求的原始体素排除在外，以此可以减少目标体素的数量，减少存储空间的占用。
在一些可能的实施方式中，上述S123步骤提及的“将多个原始体素作为与目标环境对应的多个目标体素”的实现，可以包括：步骤S1231和步骤S1232。
步骤S1231：在满足要求的原始体素中确定被其他原始体素包围的包围原始体素。
被其他原始体素包围的包围原始体素，也就是体素的六个面都存在其他原始体素与其连接。
步骤S1232：将排除包围原始体素的其他原始体素作为与目标环境对应的目标体素。
由于包围原始体素在后续的增强现实的显示中，可能无法被用户看到，因此可以不存储该原始体素，也即不将包围原始体素作为目标体素。
因此，通过将排除包围原始体素的其他原始体素作为与目标环境对应的目标体素，在存储目标体素时，可以减少存储空间的占用。
在一些可能的实施方式中，若包围原始体素四周的原始体素中的包含的语义信息包括透明材质的语义信息，则最后也可以将该包围原始体素作为目标体素。
在一些可能的实施方式中，在执行上述步骤S123中“判断每一个原始体素是否满足点云划分要求或者是网格划分要求”之前，还可以先在原始体素中确定被其他原始体素包围的包围原始体素，然后再将除包围原始体素的其他原始体素确定为用于判断是否满足点云划分要求或者是网格划分要求的原始体素。也即，在本公开的一些实施例中，包围原始体素不会用于后续的点云划分要求或者是网格划分要求的判断。以此，可以减少需要判断是否点云划分要求或者是网格划分要求的原始体素的数量，有助于提高本公开目标环境的信息管理方法的速度，并且，不将包围原始体素用于后续的判断，意味着包围原始体素不会作为目标体素，以此使得在存储目标体素时，可以减少存储空间的占用。
在本公开的一些实施例中，上述步骤S12提及的“将三维信息进行体素划分，得到与目标环境对应的多个目标体素”的实现，可以包括步骤S124至步骤S126。
步骤S124：将点云信息或者网格信息按照预设划分方法进行体素划分，得到与目标环境对应的多个原始体素。
关于本步骤的详细描述，请参阅上述的步骤S121，此处不再赘述。
步骤S125：在原始体素中确定被其他原始体素包围的包围原始体素。
在本公开实施例中，被其他原始体素包围的包围原始体素，也就是体素的六个面都存在其他原始体素与其连接。
步骤S126：将排除包围原始体素的其他原始体素作为与目标环境对应的多个目标体素。
在本公开实施例中，由于包围原始体素在后续的增强现实的显示中，可能无法被用户看到，因此可以不将包围原始体素作为目标体素，使得在存储目标体素时，可以减少存储空间的占用。
在本公开的一些实施例中，上述步骤S13提及的“根据与每个目标体素对应的目标环境的属性信息，确定每个目标体素的语义信息”的实现，可以包括：根据与每个目标体素对应的目标环境的空间属性信息，确定每个目标体素的空间语义信息。
在本公开实施例中，目标环境的空间属性信息，可以是目标环境所在的空间所表征的信息。例如，目标环境是一道路，马路的一旁有一商铺；目标体素对应的目标环境是该商铺的墙面，则基于该墙面所处的空间，可以确定目标体素的空间语义信息是某道路的某商铺的墙面。
在一些可能的实施方式中，空间属性信息可以包括现实路径信息和物体阻挡信息；对应的，空间语义信息可以包括路径语义信息和阻挡语义信息。
在本公开实施例中，现实路径信息可以认为是目标环境的相关道路、通道信息；则路径语义信息可以认为是能够表征现实环境中道路信息、通道信息的语义信息，路径语义信息例如是一目标体素对应的目标环境所属的道路，该道路与什么道路连通，能够通往的目的地等等。
示例性的，数值1表示路网体素，这样，根据体素为1的数值可以确定路网是如何连接的。电子设备可以将相邻的体素连接起来，例如，可以选择两个路网的交叉点，连接两个交叉点，则与者两个交叉点相交的所有体素都设置相应的路网信息，如1。如此，相对于直接电线连接的方式，在一些搜索示例中，具有更好的引导作用。
在本公开实施例中，物体阻挡信息可以认为是物体经过目标体素对应的目标环境的空间，是否会产生碰撞的信息。例如，某一目标体素对应的目标环境所在的空间，为路边的消防栓，则可以认为经过该空间，会与该消防栓碰撞。则对应的阻挡语义信息可以是，该区域存在阻挡情况。再如，虚拟实体在现实空间中行走的情况下，需要获取自身周围的情况，比如哪里没有阻挡，可以行走，是安全的；或者，那里存在阻挡，不能行走，或者能跳上去等信息。将不同的情况存储在体素中，不同的情况可以设置不同的体素数值，如此，电子设备可以根据体素数值确定阻挡信息。此时，上述步骤S13提及的“根据与每个目标体素对应的目标环境的空间属性信息，确定每个目标体素的空间语义信息”可以包括：根据与每个目标体素对应的目标环境的现实路径信息和物体阻挡信息，确定每个目标体素的路径语义信息和阻挡语义信息。
因此，通过确定目标体素对应的目标环境的现实路径信息和物体阻挡信息，可以以此确定目标体素的路径语义信息和阻挡语义信息。
在一些可能的实施方式中，不同的道路、通道的连接交汇处，可以分别认为是连接的道路、通道中的任一道路的一部分。例如，A路与B路的交汇处，则可以认为是属于A路的一部分，也可以认为是属于B路的一部分。又如，A通道是大厅走廊，B通道是楼梯通道，则A通道与B通道的交汇处，可以认为是属于A通道的一部分，也可以认为是属于B通道的一部分。对于不同路径的交汇处的目标体素，定义为交汇目标体素。对于交汇目标体素， 确定其路径语义信息的步骤，可以包括：确定目标环境中不同路径的交汇处的交汇目标体素，基于交汇处不同路径的现实路径信息确定交汇目标体素的路径语义信息。示例性的，确定交汇目标体素所在的路径交汇处能够属于哪些道路，然后将这些道路的路径语义信息都作为交汇目标体素的路径语义信息。
因此，通过基于交汇处不同路径的现实路径信息确定交汇目标体素的路径语义信息，使得交汇目标体素的路径语义信息能够包含交汇的路径的语义信息，方便后续的路径信息的查找。
在一些可能的实施方式中，上述步骤提及的“根据与每个目标体素对应的目标环境的属性信息，确定每个目标体素的语义信息”的实现，可以包括：根据与每个目标体素对应的目标环境的社会属性信息，确定每个目标体素的社会语义信息。例如，目标体素对应的目标环境为划定的单车停放区域，则目标环境表征的社会属性信息为单车停放区，对应的社会语义信息也为单车停放区域。又如，目标体素对应的目标环境为图书馆的阅览室，则目标环境的社会属性信息为阅览室，对应的社会语义信息也为阅读区域等等。
可以理解的是，通过确定目标体素对应的目标环境的社会属性信息，可以相应确定每个目标体素的社会语义信息。体素划分可以使电子设备编辑的信息可以是基于三维空间的信息，如此，可以更加有空间结构，使虚实融合的真实感更高。
在一些可能的实施方式中，社会属性信息包括区域类别信息。社会语义信息包括类别语义信息。这里，可以根据与每个目标体素对应的目标环境的区域类别信息，确定每个目标体素的类别语义信息。目标环境可以按照不同的标准被分为不同的区域。例如，从是否能够说话的角度分类，可以分为安静区域，说话区域等等。从路人行走安全的角度分类，可以分为小心行走区域、禁止行走区域、自由通行区域等等。分类标准可以预先设定，此处不做限制。区域类别信息即为目标环境所属的类型。类别语义信息也可以是目标环境所属的类型信息。可以理解，当分类标准有多个时，则一个目标体素的类别语义信息也可以是包含多个不同分类标准的类别语义信息。因此，通过根据与每个目标体素对应的目标环境的区域类别信息，可以相应确定每个目标体素的类别语义信息。
在一些可能的实施方式中，上述步骤提及的“根据与每个目标体素对应的目标环境的属性信息，确定每个目标体素的语义信息”的实现，可以包括：根据与每个目标体素对应的目标环境的物体属性信息，确定每个目标体素的物体语义信息。
在本公开实施例中，目标环境的物体属性信息可以是目标环境中存在的目标物体所表征的信息。目标物体可以是目标环境中的任意的物体，目标物体的物体属性信息可以包括与目标物相关的各种信息。例如，目标环境中存在一汽车，则该汽车所表征的信息可以是汽车的大小、品牌、大小等等。对应的物体语义信息也可以是目标物相关的各种信息。因此，通过确定每个目 标体素对应的目标环境的物体属性信息，可以相应确定每个目标体素的物体语义信息。
在一些可能的实施方式中，物体属性信息包括物体材质信息。物体语义信息包括物体材质语义信息。这里，电子设备可以根据与每个目标体素对应的目标环境的物体材质信息，确定每个目标体素的物体材质语义信息。物体材质信息包括物体的组成材质，例如是物体表面的材质等。示例性的，目标物体为一大型绒毛玩偶，其表面是绒毛。则可以确定该绒毛玩偶的物体材质信息可以是表面为绒毛，则对应的物体材质语义信息可以为绒毛表面。又如，目标物体为一沙池，沙池的表面都是沙子，则物体材质信息可为沙子，对应的物体材质语义信息为沙子表面。因此，通过确定与每个目标体素对应的目标环境的物体材质信息，可以相应确定每个目标体素的物体材质语义信息。
需要说明的是，在现实世界中，不同的材质表现出的视觉、听觉、交互信息是不同的，比如，在水泥地面、沙地、河流中行走，展现出来的声音、水花完全不同，这里，通过体素存储物体材质信息，如此，电子设备可以根据体素数值确定物体材质信息和物体语义材质信息。
参阅图3，图3是本公开目标环境的信息管理方法另一实施例的流程示意图。在上述实施例提及的步骤“根据与每个目标体素对应的目标环境的属性信息，确定每个目标体素的语义信息”之后，本公开实施例中的目标环境的信息管理方法还可以包括：步骤S21和步骤S22。
步骤S21：获取与目标环境相关的其他语义信息。
在本公开实施例中，其他语义信息可以是与目标环境整体相关的其他语义信息。例如，目标环境是某一旅游景点的某一区域，则其他语义信息还可以是该旅游景点的其他语义信息。其他语义信息可以是人工确定的，也可以是由系统自动匹配的，确定方式不受限制。
步骤S22：将其他语义信息存储在与目标环境对应的目标体素中。
在本公开实施例中，目标体素是基于目标环境得到的，电子设备可以将其他语义信息存储在与目标环境对应的目标体素中，以此使得目标体素能够存储包含有与整个目标环境关联的其他语义信息。
示例性的，其他语义信息可以包括矢量化信息；对于目标环境，电子设备可以将矢量化信息和目标环境的体素联系在一起；如此，电子设备可以将目标环境相关的路网的矢量化信息一同存储在目标环境的整体结构中。
参阅图4，图4是本公开增强现实的显示方法一实施例的流程示意图。在本实施例中，增强现实的显示方法包括以下步骤：
步骤S31：获取目标环境中目标体素的语义信息。
在本公开实施例中，执行增强现实的显示方法步骤的设备例如是手机，AR眼镜等等设备。
在一些可能的实施方式中，电子设备可以通过获取周围环境的图像，并基于图像识别的技术，将图像中的环境与已经建立的目标环境进行匹配，进而获取目标环境中目标体素的语义信息。目标体素的语义信息可以是通过上 述目标环境的信息管理方法的实施例得到的语义信息。
步骤S32：在目标环境的目标体素的对应位置显示语义信息。
在本公开实施例中，因为目标体素的语义信息能够反映目标体素对应的目标环境的属性信息，因此，电子设备可以在目标环境的目标体素的对应位置显示语义信息。
因此，通过在目标环境的目标体素的对应位置显示语义信息，可以方便用户通过查看语义信息，快速了解目标环境。
参阅图5，图5是本公开增强现实的显示方法另一实施例的流程示意图。在本实施例中，增强现实的显示方法包括以下步骤：
步骤S41：获取虚拟物体所处环境中的目标体素的语义信息。
在本公开实施例中，虚拟物体可以是电子设备模拟生成的虚拟物体，例如是由执行本公开增强现实的显示方法的手机模拟生成的一个虚拟人物。
在本公开实施例中，电子设备在生成虚拟物体以后，可以获取虚拟物体所处环境中的目标体素的语义信息，以此虚拟物体能够通过获得的语义信息，确定其所处的环境。
示例性的，虚拟物体被摆放在空间中，电子设备可以根据虚拟物体的定位位置，获取定位位置的周边体素以及全局体素，由于体素中存储有路网、阻挡、区域、材质等语义信息，电设备可以根据这些语义信息，在虚实融合中进行声音和视觉展现，并实现寻路、避障、空间内区域提醒(如商场卫生间提醒、扶梯安全提醒)等智能行为，提高虚实融合的效果。
步骤S42：基于目标体素的语义信息，确定虚拟物体的对应行为。
在一些可能的实施方式中，基于目标体素的语义信息，控制虚拟物体的行为即是确定目标体素的语义信息与虚拟物体的预设反应的对应关系，由此使得虚拟物体能够针对目标体素的语义信息产生反应，使得虚拟物体能够更加逼真。
步骤S43：显示虚拟物体的对应行为。
在本公开实施例中，因为虚拟物体的对应行为是对目标环境的反应，通过可以显示虚拟物体的对应行为，使得该对应行为能够为用户感知。
示例性的，虚拟物体是虚拟人物，目标环境中包括瓷砖铺就的地面，和沙石地面，在沙石地面上立着一个雕塑，则目标环境中的目标体素的语义信息包括瓷砖地面、沙石地面，雕塑。虚拟人物在获取目标环境中的语义信息以后，当虚拟人物走在沙石路面时，根据沙石路面的语义信息，可以控制虚拟物体的行为其行走有点不自然，因为沙子硌脚。当虚拟人物走到雕塑面前时，可以控制虚拟人物绕开，因为雕塑可以阻挡语义信息。当虚拟人物走到瓷砖地面时，可以控制虚拟人物慢慢行走，因为瓷砖地面路滑。
因此，通过基于目标体素的语义信息来控制虚拟物体的行为，可以使得虚拟物体的行为更加自然和逼真。
请参阅图6，图6是本公开实施例提供的目标环境的信息管理装置一实施例的框架示意图。信息管理装置60包括获取部分61、划分部分62和确定部 分63。获取部分61，被配置为获取目标环境的三维信息；划分部分62，被配置为将三维信息进行体素划分，得到与目标环境对应的多个目标体素；确定模块63，被配置为根据与每个目标体素对应的目标环境的属性信息，确定每个目标体素的语义信息。
其中，上述的目标环境的属性信息包括空间属性信息、社会属性信息和物体属性信息中的至少一个；目标体素的语义信息对应目标环境的属性信息，包括空间语义信息、社会语义信息和物体语义信息中的至少一个；上述的确定部分63，被配置为根据与每个目标体素对应的目标环境的属性信息，确定每个目标体素的语义信息，包括以下至少一种：根据与每个目标体素对应的目标环境的空间属性信息，确定每个目标体素的空间语义信息；根据与每个目标体素对应的目标环境的社会属性信息，确定每个目标体素的社会语义信息；根据与每个目标体素对应的目标环境的物体属性信息，确定每个目标体素的物体语义信息。
其中，上述的空间属性信息包括现实路径信息和物体阻挡信息；社会属性信息包括区域类别信息，物体属性信息包括物体材质信息；空间语义信息包括路径语义信息和阻挡语义信息，社会语义信息包括类别语义信息，物体语义信息包括材质语义信息；上述的确定部分63，被配置为根据与每个目标体素对应的目标环境的空间属性信息，确定每个目标体素的空间语义信息，包括：根据与每个目标体素对应的目标环境的现实路径信息和物体阻挡信息，确定每个目标体素的路径语义信息和阻挡语义信息；上述的确定部分63，被配置为根据与每个目标体素对应的目标环境的社会属性信息，确定每个目标体素的社会语义信息，包括：根据与每个目标体素对应的目标环境的区域类别信息，确定每个目标体素的类别语义信息；上述的确定部分63，被配置为根据与每个目标体素对应的目标环境的物体属性信息，确定每个目标体素的物体语义信息，包括：根据与每个目标体素对应的目标环境的物体材质信息，确定每个目标体素的物体材质语义信息。
其中，上述的确定部分63，被配置为根据与每个目标体素对应的目标环境的现实路径信息，确定每个目标体素的路径语义信息，包括：确定目标环境中不同路径交汇处的交汇目标体素，基于交汇处不同路径的现实路径信息确定交汇目标体素的路径语义信息。
其中，上述的三维信息包括点云信息或者是网格信息；上述的划分部分62，被配置为将三维信息进行体素划分，得到与目标环境对应的多个目标体素，包括：将点云信息或者网格信息按照预设划分方法进行体素划分，得到与目标环境对应的多个原始体素；判断每一个原始体素是否满足点云划分要求或者是网格划分要求，得到判断结果；在所述判断结果表征所述每个原始体素中的多个原始体素满足点云划分要求或者网格划分要求的情况下，将多个原始体素作为与目标环境对应的多个目标体素。
其中，上述的划分部分62，被配置为将满足要求的原始体素作为与目标环境对应的多个目标体素，包括：在满足要求的原始体素中确定被其他原始 体素包围的包围原始体素；将排除包围原始体素的其他原始体素作为与目标环境对应的多个目标体素。
其中，上述的划分部分62，被配置为将三维信息进行体素划分，得到与目标环境对应的多个目标体素，包括：将点云信息或者是网格信息按照预设划分方法进行体素划分，得到与目标环境对应的多个原始体素；在原始体素中确定被其他原始体素包围的包围原始体素；将排除包围原始体素的其他原始体素作为与目标环境对应的多个目标体素。
其中，上述的信息管理装置60还包括语义信息获取部分，在确定部分63被配置为根据与每个目标体素对应的目标环境的属性信息，确定每个目标体素的语义信息之后，语义信息获取部分，被配置为获取与目标环境相关的其他语义信息；将其他语义信息存储在与目标环境对应的目标体素中。
本公开还公开了一增强现实的显示装置，该显示装置包括第一获取部分和第一显示部分。第一获取部分，被配置为获取目标环境中目标体素的语义信息，其中，目标体素的语义信息是通过上述目标环境的信息管理方法实施例得到的。第一显示部分，被配置为在目标环境的目标体素的对应位置显示语义信息。
本公开还公开了另一增强现实的显示装置，该显示装置包括第二获取部分、确定部分和第二显示部分。第二获取部分，被配置为获取虚拟物体所处环境中的目标体素的语义信息，其中，目标体素的语义信息是通过上述目标环境的信息管理方法实施例得到的。确定部分，被配置为基于目标体素的语义信息，确定虚拟物体的对应行为；第二显示部分，被配置为显示虚拟物体的对应行为。
在本公开实施例以及其他的实施例中，“部分”可以是部分电路、部分处理器、部分程序或软件等等，当然也可以是单元，还可以是模块也可以是非模块化的。
请参阅图7，图7是本公开电子设备一实施例的框架示意图。电子设备70包括相互耦接的存储器71和处理器72，处理器72用于执行存储器71中存储的程序指令，以实现上述任一目标环境的信息管理方法或者是增强现实的显示方法实施例的步骤。
在一个具体的实施场景中，电子设备70可以包括但不限于：微型计算机、服务器，此外，电子设备70还可以包括笔记本电脑、平板电脑等移动设备，在此不做限定。
在一些实施例中，处理器72用于控制其自身以及存储器71以实现上述任一目标环境的信息管理方法或者是增强现实的显示方法实施例的步骤。处理器72还可以称为CPU(Central Processing Unit，中央处理单元)。处理器72可能是一种集成电路芯片，具有信号的处理能力。处理器72还可以是通用处理器、数字信号处理器(Digital Signal Processor,DSP)、专用集成电路(Application Specific Integrated Circuit,ASIC)、现场可编程门阵列(Field-Programmable Gate Array,FPGA)或者其他可编程逻辑器件、分立门 或者晶体管逻辑器件、分立硬件组件。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。另外，处理器72可以由集成电路芯片共同实现。
请参阅图8，图8为本公开计算机可读存储介质一实施例的框架示意图。计算机可读存储介质80存储有能够被处理器运行的程序指令81，程序指令81用于实现上述任一目标环境的信息管理方法或者是增强现实的显示方法实施例的步骤。
上述方案，电子设备通过获取将目标环境的三维信息进行体素划分后得到的目标体素，再将与每个目标体素对应的目标环境的属性信息，确定为每个目标体素的语义信息，使得目标体素的语义信息能够反映目标环境的属性信息，实现了将真实环境的信息融入到语义信息之中的效果。
本公开还公开了一种计算机程序，包括计算机可读代码，当计算机可读代码在电子设备中运行时，电子设备中的处理器执行时实现上述任一目标环境的信息管理方法或者是增强现实的显示方法实施例的步骤。
本公开还公开了一种计算机程序产品，包括计算机程序指令，当计算机程序指令被计算机执行时，实现上述任一目标环境的信息管理方法或者是增强现实的显示方法实施例的步骤。
本公开涉及增强现实领域，通过获取现实环境中的目标对象的图像信息，进而借助各类视觉相关算法实现对目标对象的相关特征、状态及属性进行检测或识别处理，从而得到与具体应用匹配的虚拟与现实相结合的AR效果。示例性的，目标对象可涉及与人体相关的脸部、肢体、手势、动作等，或者与物体相关的标识物、标志物，或者与场馆或场所相关的沙盘、展示区域或展示物品等。视觉相关算法可涉及视觉定位、SLAM、三维重建、图像注册、背景分割、对象的关键点提取及跟踪、对象的位姿或深度检测等。具体应用不仅可以涉及跟真实场景或物品相关的导览、导航、讲解、重建、虚拟效果叠加展示等交互场景，还可以涉及与人相关的特效处理，比如妆容美化、肢体美化、特效展示、虚拟模型展示等交互场景。
可通过卷积神经网络，实现对目标对象的相关特征、状态及属性进行检测或识别处理。上述卷积神经网络是基于深度学习框架进行模型训练而得到的网络模型。
在一些实施例中，本公开实施例提供的装置具有的功能或包含的模块可以用于执行上文方法实施例描述的方法，其具体实现可以参照上文方法实施例的描述，为了简洁，这里不再赘述。
上文对各个实施例的描述倾向于强调各个实施例之间的不同之处，其相同或相似之处可以互相参考，为了简洁，本文不再赘述。
在本公开所提供的几个实施例中，应该理解到，所揭露的方法和装置，可以通过其它的方式实现。例如，以上所描述的装置实施方式仅仅是示意性的，例如，模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如单元或组件可以结合或者可以集成到另一个系统， 或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性、机械或其它的形式。
作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施方式方案的目的。
另外，在本公开各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本公开的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)或处理器(processor)执行本公开各个实施方式方法的全部或部分步骤。而前述的存储介质包括：可以保持和存储由指令执行设备使用的指令的有形设备，可为易失性存储介质或非易失性存储介质。计算机可读存储介质例如可以是(但不限于)电存储设备、磁存储设备、光存储设备、电磁存储设备、半导体存储设备或者上述的任意合适的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、静态随机存取存储器(SRAM)、便携式压缩盘只读存储器(CD-ROM)、数字多功能盘(DVD)、记忆棒、软盘、机械编码设备、例如其上存储有指令的打孔卡或凹槽内凸起结构、以及上述的任意合适的组合。这里所使用的计算机可读存储介质不被解释为瞬时信号本身，诸如无线电波或者其他自由传播的电磁波、通过波导或其他传输媒介传播的电磁波(例如，通过光纤电缆的光脉冲)、或者通过电线传输的电信号。
工业实用性
本公开实施例中，通过获取将目标环境的三维信息进行体素划分后得到的目标体素，再将与每个目标体素对应的目标环境的属性信息，确定为每个目标体素的语义信息，使得目标体素的语义信息能够反映目标环境的属性信息，实现了将真实环境的信息融入到语义信息之中，从而提高了虚拟融合的真实感。

Claims (14)

  1. 一种目标环境的信息管理方法，包括：
    获取所述目标环境的三维信息；
    将所述三维信息进行体素划分，得到与所述目标环境对应的多个目标体素；
    根据与每个所述目标体素对应的目标环境的属性信息，确定每个所述目标体素的语义信息。
  2. 根据权利要求1所述的方法，其中，所述目标环境的属性信息包括空间属性信息、社会属性信息和物体属性信息中的至少一个；所述目标体素的语义信息对应所述目标环境的属性信息；所述目标体素的语义信息包括空间语义信息、社会语义信息和物体语义信息中的至少一个；所述根据与每个所述目标体素对应的目标环境的属性信息，确定每个所述目标体素的语义信息，包括以下至少一种：
    根据与每个所述目标体素对应的目标环境的空间属性信息，确定每个所述目标体素的空间语义信息；
    根据与每个所述目标体素对应的目标环境的社会属性信息，确定每个所述目标体素的社会语义信息；
    根据与每个所述目标体素对应的目标环境的物体属性信息，确定每个所述目标体素的物体语义信息。
  3. 根据权利要求2所述的方法，其中，所述空间属性信息包括现实路径信息和物体阻挡信息；所述社会属性信息包括区域类别信息，所述物体属性信息包括物体材质信息；所述空间语义信息包括路径语义信息和阻挡语义信息，所述社会语义信息包括类别语义信息，所述物体语义信息包括材质语义信息；
    所述根据与每个所述目标体素对应的目标环境的空间属性信息，确定每个所述目标体素的空间语义信息，包括：根据与每个所述目标体素对应的目标环境的现实路径信息和物体阻挡信息，确定每个所述目标体素的路径语义信息和阻挡语义信息；
    所述根据与每个所述目标体素对应的目标环境的社会属性信息，确定每个所述目标体素的社会语义信息，包括：根据与每个所述目标体素对应的目标环境的区域类别信息，确定每个所述目标体素的类别语义信息；
    所述根据与每个所述目标体素对应的目标环境的物体属性信息，确定每个所述目标体素的物体语义信息，包括：根据与每个所述目标体素对应的目标环境的物体材质信息，确定每个所述目标体素的物体材质语义信息。
  4. 根据权利要求3所述的方法，其中，所述根据与每个所述目标体素对应的目标环境的现实路径信息，确定每个所述目标体素的路径语义信息，包括：
    确定所述目标环境中不同路径交汇处的交汇目标体素，基于交汇处不同路径的现实路径信息确定所述交汇目标体素的路径语义信息。
  5. 根据权利要求1-4任一项所述的方法，其中，所述三维信息包括点云信息或者是网格信息；所述将所述三维信息进行体素划分，得到与所述目标环境对应的多个目标体素，包括：
    将所述点云信息或者所述网格信息按照预设划分方法进行体素划分，得到与所述目标环境对应的多个原始体素；
    判断每一个所述原始体素是否满足点云划分要求或者是网格划分要求，得到判断结果；
    在所述判断结果表征所述每个原始体素中的多个原始体素满足点云划分要求或者网格划分要求的情况下，将所述多个原始体素作为与所述目标环境对应的多个目标体素。
  6. 根据权利要求5所述的方法，其中，所述将所述多个原始体素作为与所述目标环境对应的多个目标体素，包括：在满足要求的所述多个原始体素中确定被其他原始体素包围的包围原始体素；
    将排除所述包围原始体素的其他原始体素作为与所述目标环境对应的多个目标体素。
  7. 根据权利要求1所述的方法，其中，所述将所述三维信息进行体素划分，得到与所述目标环境对应的多个目标体素，包括：
    将所述点云信息或者是所述网格信息按照预设划分方法进行体素划分，得到与所述目标环境对应的多个原始体素；
    在所述多个原始体素中确定被其他原始体素包围的包围原始体素；
    将排除所述包围原始体素的其他原始体素作为与所述目标环境对应的多个目标体素。
  8. 根据权利要求1-4任一项所述的方法，其中，在根据与每个所述目标体素对应的目标环境的属性信息，确定每个所述目标体素的语义信息之后，所述方法还包括：
    获取与所述目标环境相关的其他语义信息；
    将所述其他语义信息存储在与所述目标环境对应的所述目标体素中。
  9. 一种增强现实的显示方法，包括：
    获取目标环境中目标体素的语义信息，其中，所述目标体素的语义信息是通过上述权利要求1-8任一项所述的方法得到的；
    在所述目标环境的目标体素的对应位置显示所述语义信息。
  10. 一种增强现实的显示方法，包括：
    获取虚拟物体所处环境中的目标体素的语义信息，其中，所述目标体素的语义信息是通过上述权利要求1-8任一项所述的方法得到的；
    基于所述目标体素的语义信息，确定所述虚拟物体的对应行为；
    显示所述虚拟物体的对应行为。
  11. 一种电子设备，包括相互耦接的处理器和存储器，其中，
    所述处理器用于执行所述存储器存储的计算机程序以执行权利要求1至8任一项所述的方法，或者是权利要求8所述的方法，或者是权利要求10所述的方法。
  12. 一种计算机可读存储介质，存储有能够被处理器运行的计算机程序，所述计算机程序用于实现如执行权利要求1至8任一项所述的方法，或者是权利要求9所述的方法，或者是权利要求10所述的方法。
  13. 一种计算机程序，包括计算机可读代码，当所述计算机可读代码在电子设备中运行时，所述电子设备中的处理器执行时实现权利要求1至8中任意一项所述的方法，或者是权利要求9所述的方法，或者是权利要求10所述的方法。
  14. 一种计算机程序产品，包括计算机程序指令，该计算机程序指令使得计算机执行时实现权利要求1至8中任意一项所述的方法，或者是权利要求9所述的方法，或者是权利要求10所述的方法。
PCT/CN2022/074966 2021-09-09 2022-01-29 目标环境的信息管理方法及相关增强现实的显示方法、电子设备、存储介质、计算机程序和计算机程序产品 WO2023035548A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111056910.8 2021-09-09
CN202111056910.8A CN113838209A (zh) 2021-09-09 2021-09-09 目标环境的信息管理方法及相关增强现实的显示方法

Publications (1)

Publication Number Publication Date
WO2023035548A1 true WO2023035548A1 (zh) 2023-03-16

Family

ID=78958836

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/074966 WO2023035548A1 (zh) 2021-09-09 2022-01-29 目标环境的信息管理方法及相关增强现实的显示方法、电子设备、存储介质、计算机程序和计算机程序产品

Country Status (2)

Country Link
CN (1) CN113838209A (zh)
WO (1) WO2023035548A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113838209A (zh) * 2021-09-09 2021-12-24 深圳市慧鲤科技有限公司 目标环境的信息管理方法及相关增强现实的显示方法

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017117675A1 (en) * 2016-01-08 2017-07-13 Sulon Technologies Inc. Head mounted device for augmented reality
CN110827295A (zh) * 2019-10-31 2020-02-21 北京航空航天大学青岛研究院 基于体素模型与颜色信息耦合的三维语义分割方法
CN113139992A (zh) * 2020-01-17 2021-07-20 苹果公司 多分辨率体素网格化
CN113140032A (zh) * 2020-01-17 2021-07-20 苹果公司 基于房间扫描的平面图生成
CN113160411A (zh) * 2021-04-23 2021-07-23 杭州电子科技大学 一种基于rgb-d传感器的室内三维重建方法
CN113838209A (zh) * 2021-09-09 2021-12-24 深圳市慧鲤科技有限公司 目标环境的信息管理方法及相关增强现实的显示方法

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109242963B (zh) * 2018-09-29 2023-08-18 深圳阜时科技有限公司 一种三维场景模拟装置和设备
CN112581629B (zh) * 2020-12-09 2024-05-28 中国科学院深圳先进技术研究院 增强现实显示方法、装置、电子设备及存储介质
CN113018847A (zh) * 2021-03-31 2021-06-25 广州虎牙科技有限公司 体素建筑的生成方法、装置、电子设备及存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017117675A1 (en) * 2016-01-08 2017-07-13 Sulon Technologies Inc. Head mounted device for augmented reality
CN110827295A (zh) * 2019-10-31 2020-02-21 北京航空航天大学青岛研究院 基于体素模型与颜色信息耦合的三维语义分割方法
CN113139992A (zh) * 2020-01-17 2021-07-20 苹果公司 多分辨率体素网格化
CN113140032A (zh) * 2020-01-17 2021-07-20 苹果公司 基于房间扫描的平面图生成
CN113160411A (zh) * 2021-04-23 2021-07-23 杭州电子科技大学 一种基于rgb-d传感器的室内三维重建方法
CN113838209A (zh) * 2021-09-09 2021-12-24 深圳市慧鲤科技有限公司 目标环境的信息管理方法及相关增强现实的显示方法

Also Published As

Publication number Publication date
CN113838209A (zh) 2021-12-24

Similar Documents

Publication Publication Date Title
US11087548B2 (en) Authoring and presenting 3D presentations in augmented reality
EP2973433A2 (en) Mapping augmented reality experience to various environments
JP6050518B2 (ja) 実環境に仮想情報を表現する方法
US10242476B2 (en) Methods and systems for dynamically representing, within a virtual reality data stream being presented to a user, a proxy object that corresponds to an object in the real-world environment of the user
JP5592011B2 (ja) マルチスケール３次元配向
AU2022345532B2 (en) Browser optimized interactive electronic model based determination of attributes of a structure
US11014242B2 (en) Puppeteering in augmented reality
CN112102465B (zh) 基于3d结构引擎的计算平台
CN105637564A (zh) 产生未知对象的扩增现实内容
Montero et al. Designing and implementing interactive and realistic augmented reality experiences
Kasapakis et al. Augmented reality in cultural heritage: Field of view awareness in an archaeological site mobile guide
WO2023035548A1 (zh) 目标环境的信息管理方法及相关增强现实的显示方法、电子设备、存储介质、计算机程序和计算机程序产品
CN116097316A (zh) 用于非模态中心预测的对象识别神经网络
JP7519462B2 (ja) フロアプランを生成するための方法、装置およびプログラム
Trapp et al. Strategies for visualising 3D points-of-interest on mobile devices
Paliou Visual perception in past built environments: Theoretical and procedural issues in the archaeological application of three-dimensional visibility analysis
Soares et al. Designing a highly immersive interactive environment: The virtual mine
Kiourt et al. Realistic simulation of cultural heritage
WO2023155394A1 (zh) 虚拟空间融合方法及相关装置、电子设备、介质及程序
Liu et al. Game engine-based point cloud visualization and perception for situation awareness of crisis indoor environments
US12002165B1 (en) Light probe placement for displaying objects in 3D environments on electronic devices
KR101267570B1 (ko) 2ｄ 텍스쳐 기반의 유도장을 이용한 가상캐릭터 조정행동 표시방법 및 장치, 이를 사용한 가상 생태공원 가시화 시스템
KR20140078083A (ko) 증강 현실이 구현된 만화책
Turan Virtual reality implementation for University Presentation
Gupta Quantum space time travel with the implementation of augmented reality and artificial intelligence

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22866028

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22866028

Country of ref document: EP

Kind code of ref document: A1