WO2023202349A1

WO2023202349A1 - Interactive presentation method and apparatus for three-dimensional label, and device, medium and program product

Info

Publication number: WO2023202349A1
Application number: PCT/CN2023/085213
Authority: WO
Inventors: 王怡丁
Original assignee: 如你所视(北京)科技有限公司
Priority date: 2022-04-22
Filing date: 2023-03-30
Publication date: 2023-10-26
Also published as: CN114842175A; CN114842175B

Abstract

Disclosed in the embodiments of the present disclosure are an interactive presentation method and apparatus for a three-dimensional label, and an electronic device, a storage medium and a computer program product. The method comprises: on the basis of a point location and an angle of view when a user browses a three-dimensional scene, determining at least one target three-dimensional model from among three-dimensional models visible to the user; determining a reference point, which corresponds to the target three-dimensional model and is closest to the point location, to be a mark point corresponding to the target three-dimensional model, and determining a reference plane, in which the mark point is located, to be a presentation plane corresponding to the target three-dimensional model, so as to obtain the mark point and the presentation plane that correspond to the target three-dimensional model; on the basis of the mark point and the presentation plane that correspond to the target three-dimensional model, determining a spatial pose, in the three-dimensional scene, of a three-dimensional label corresponding to the target three-dimensional model; and on the basis of the spatial pose, in the three-dimensional scene, of the three-dimensional label corresponding to the target three-dimensional model, presenting, in the three-dimensional scene, the three-dimensional label corresponding to the three-dimensional model.

Description

Interactive presentation methods, devices, equipment, media and program products for three-dimensional labels

This disclosure requires the priority of a Chinese patent application submitted to the China Patent Office on April 22, 2022, with the application number CN202210427645.8 and the invention title "Interactive presentation method, device, equipment, medium and program product of three-dimensional labels", The entire contents of which are incorporated into this disclosure by reference.

Technical field

The present disclosure relates to the field of computer technology, and in particular, to an interactive presentation method, device, electronic device, storage medium and computer program product for three-dimensional labels.

Background technique

With the development of computer technology, especially the rapid progress of virtual reality (VR), three-dimensional scenes are increasingly used in various fields. Three-dimensional scenes can immerse users into the environment through interactive three-dimensional dynamic views that integrate multi-source information, allowing users to watch different content from different perspectives.

In related technologies, in order to facilitate users to intuitively obtain information about display objects in a three-dimensional scene, the information about the display objects can be presented in the three-dimensional scene in the form of labels.

Contents of the invention

Embodiments of the present disclosure provide an interactive presentation method, device, electronic device, storage medium, and computer program product for three-dimensional labels, which are used to improve the display effect of three-dimensional labels in a three-dimensional scene.

According to one aspect of an embodiment of the present disclosure, a method for interactive presentation of three-dimensional labels is provided. The method includes:

Based on the user's point and perspective when browsing the three-dimensional scene, at least one target three-dimensional model is determined from the three-dimensional models visible to the user, and the target three-dimensional model corresponds to one three-dimensional label to be marked and a plurality of three-dimensional labels for marking three-dimensional labels. reference points and the reference plane on which each said reference point lies;

The reference point corresponding to the target three-dimensional model and closest to the point is determined as the mark point corresponding to the target three-dimensional model, and the reference plane where the mark point is located is determined as the display plane corresponding to the target three-dimensional model. , obtain the mark points and display plane corresponding to the target three-dimensional model;

Based on the marker points and display planes corresponding to the target three-dimensional model, determine the spatial pose of the three-dimensional label corresponding to the target three-dimensional model in the three-dimensional scene;

Based on the spatial pose of the three-dimensional label corresponding to the target three-dimensional model in the three-dimensional scene, the three-dimensional label corresponding to the target three-dimensional model is presented in the three-dimensional scene.

According to another aspect of an embodiment of the present disclosure, an interactive presentation device for three-dimensional labels is provided. The device includes a first determination unit configured to determine the position and perspective of the user based on the user's point position and perspective when browsing the three-dimensional scene. At least one target three-dimensional model is determined in the three-dimensional model, and the target three-dimensional model corresponds to a three-dimensional label to be marked, a plurality of reference points for marking the three-dimensional labels, and a reference plane where each of the reference points is located;

The second determination unit is configured to determine the reference point corresponding to the target three-dimensional model and closest to the point as the mark point corresponding to the target three-dimensional model, and determine the reference plane where the mark point is located as the reference point. Determine the display plane corresponding to the target three-dimensional model, and obtain the mark points and display plane corresponding to the target three-dimensional model;

The pose determination unit is configured to determine the spatial pose of the three-dimensional label corresponding to the target three-dimensional model in the three-dimensional scene based on the marker point corresponding to the target three-dimensional model and the display plane;

The label presentation unit is configured to present the three-dimensional label corresponding to the three-dimensional model in the three-dimensional scene based on the spatial pose of the three-dimensional label corresponding to the target three-dimensional model in the three-dimensional scene.

According to yet another aspect of an embodiment of the present disclosure, an electronic device is provided, including: a memory for storing a computer program product;

A processor, configured to execute a computer program product stored in the memory, and when the computer program product is executed, implement the interactive presentation method of a three-dimensional label provided in any one of the above embodiments of the present disclosure.

According to yet another aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, with program code stored thereon, and the program code can be called and executed by a processor to implement any of the methods provided by any of the above embodiments of the present disclosure. An interactive presentation method for three-dimensional labels.

According to another aspect of the embodiments of the present disclosure, a computer program product is provided, including computer program instructions. When the computer program instructions are executed by a processor, the interactive presentation of the three-dimensional label provided in any of the above embodiments of the present disclosure is realized. method.

In the solution provided by the embodiment of the present disclosure, the target 3D model visible to the user can be determined from the 3D models in the 3D scene based on the user's point position and perspective when browsing the 3D scene, and the reference point in the target 3D model that is closest to the point can be determined point as a marker point, determine the reference plane where the marker point is located as the display plane; then determine the spatial pose of the three-dimensional label based on the marker point and the display plane, and finally present at least one target three-dimensional object in the three-dimensional scene according to the spatial pose of the three-dimensional label The 3D label corresponding to the model. It can ensure the matching degree of the spatial pose and point position of the 3D label, allowing users to obtain the information of the 3D model more intuitively and conveniently, thereby improving the display effect of the 3D label in the 3D scene.

The technical solution of the present disclosure will be described in further detail below through the accompanying drawings and examples.

Description of the drawings

The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the disclosure and, together with the description, serve to explain principles of the disclosure.

The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings. Obviously, the drawings in the following description are only some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without exerting any creative effort:

Figure 1 is a schematic flow chart of one embodiment of the interactive presentation method of three-dimensional labels of the present disclosure;

Figure 2 is a schematic diagram of a presentation method of three-dimensional labels in one embodiment of the interactive presentation method of three-dimensional labels of the present disclosure;

Figure 3 is a schematic scene diagram of one embodiment of the interactive presentation method of three-dimensional labels of the present disclosure;

Figure 4 is a schematic diagram of the marking position of the three-dimensional label in one embodiment of the interactive presentation method of the three-dimensional label of the present disclosure;

Figure 5 is a schematic flowchart of another embodiment of the interactive presentation method of three-dimensional labels of the present disclosure;

Figure 6 is a schematic flowchart of another embodiment of the interactive presentation method of three-dimensional labels of the present disclosure;

Figure 7 is a schematic structural diagram of an embodiment of an interactive presentation device for three-dimensional labels of the present disclosure;

FIG. 8 is a schematic structural diagram of an application embodiment of the electronic device of the present disclosure.

Detailed ways

Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions, and numerical values set forth in these examples do not limit the scope of the disclosure unless otherwise stated.

It should also be understood that in the embodiments of the present disclosure, "plurality" may refer to two or more than two, and "at least one" may refer to one, two, or more than two.

Those skilled in the art can understand that terms such as "first" and "second" in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, etc., and do not represent any specific technical meaning, nor do they represent the differences between them. necessary logical sequence.

It should also be understood that any component, data or structure mentioned in the embodiments of the present disclosure shall not be expressly limited or When the context gives contrary revelation, it can generally be understood as one or more.

It should also be understood that the description of various embodiments in this disclosure focuses on the differences between the various embodiments, and the similarities or similarities between the embodiments can be referred to each other. For the sake of brevity, they will not be repeated one by one.

The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the disclosure, its application or uses.

Techniques, methods and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, such techniques, methods and devices should be considered a part of the specification.

It should be noted that similar reference numerals and letters refer to similar items in the following figures, so that once an item is defined in one figure, it does not need further discussion in subsequent figures.

In addition, the term "and/or" in the disclosure is only an association relationship describing related objects, indicating that there can be three relationships. For example, A and/or B can mean: A alone exists, and A and B exist simultaneously. There are three cases of B alone. In addition, the character "/" in this disclosure generally indicates that the related objects are in an "or" relationship.

Embodiments of the present disclosure may be applied to electronic devices such as terminal devices, computer systems, servers, etc., which may operate with numerous other general or special purpose computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments and/or configurations suitable for use with terminal devices, computer systems, servers and other electronic devices include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients Computers, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, networked personal computers, small computer systems, mainframe computer systems and distributed cloud computing technology environments including any of the above systems, etc.

Electronic devices such as terminal devices, computer systems, servers, etc. may be described in the general context of computer system executable instructions (such as program modules) being executed by the computer system. Generally, program modules may include routines, programs, object programs, components, logic, data structures, etc., that perform specific tasks or implement specific abstract data types. The computer system/server may be implemented in a distributed cloud computing environment where tasks are performed by remote processing devices linked through a communications network. In a distributed cloud computing environment, program modules may be located on local or remote computing system storage media including storage devices.

In order to make the technical solutions and advantages in the embodiments of the present disclosure more clear, the exemplary embodiments of the present disclosure are further described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some of the embodiments of the present disclosure, not all of them. Exhaustive list of examples. It should be noted that, as long as there is no conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

The interactive presentation method of three-dimensional labels of the present disclosure will be exemplified below with reference to FIG. 1 . Figure 1 shows a flow chart of one embodiment of the interactive presentation method of three-dimensional labels of the present disclosure. As shown in Figure 1, the process includes the following steps:

Step 110: Based on the user's point and perspective when browsing the three-dimensional scene, determine at least one target three-dimensional model from the three-dimensional models visible to the user.

Among them, the target three-dimensional model corresponds to a three-dimensional label to be marked, a plurality of reference points for marking the three-dimensional labels, and the reference plane where each reference point is located.

In this implementation, the target three-dimensional model represents a user-visible three-dimensional model to be marked. When constructing a 3D scene, you can select some or all 3D models in the 3D scene as 3D models to be labeled according to requirements, and generate 3D labels for each 3D model to be labeled. When the user browses the three-dimensional scene, the user's visible area in the three-dimensional scene can be determined based on the point position and perspective. The three-dimensional model to be marked located within the visible area is the target three-dimensional model.

The three-dimensional label to be marked represents a three-dimensional label that has not yet been presented in the three-dimensional scene. It is a label with three-dimensional characteristics (such as conforming to the rules of perspective) generated in advance based on the description information of the three-dimensional model. It is used to display the description information of the three-dimensional model. For example, it can include The name, size, material, price, etc. of the 3D model.

The reference point corresponding to the target 3D model and the reference plane where it is located can be determined in advance based on the spatial shape of the target 3D model and the pose information of the target 3D model in the 3D scene. The reference point represents the position where the 3D label can be marked. Reference The plane is used to constrain the spatial posture of the three-dimensional label. As an example, the spatial coordinates in the three-dimensional scene can be used to represent the reference point, and the spatial coordinates and parameters of the four corner points of the reference plane in the three-dimensional scene can be used. The normal vector of the reference plane represents the reference plane.

In an optional implementation of this embodiment, the reference point may be located on the surface of the target three-dimensional model, and the reference plane is perpendicular or parallel to the plane constituting the bounding box of the target three-dimensional model. For example, each three-dimensional model may include 6 reference planes, each reference plane being parallel to the two planes constituting the bounding box.

In practice, service providers can use three-dimensional scenes (such as VR scenes) to simulate real scenes and display multiple objects (such as furniture and other items) to users in the form of scenes. Usually, when building a three-dimensional scene, you can set multiple points according to your needs, and then build a view frustum based on the points and camera parameters, and use the view frustum to determine the user's field of view when browsing the three-dimensional scene. Afterwards, the 3D model constructed based on the appearance and attributes of the display object can be placed in the 3D scene according to the preset pose, and the completed 3D scene can be formed into a Json file. Users can obtain the Json file of the three-dimensional scene from the service provider through an electronic device (such as a terminal computer or a smartphone), and parse the Json file through the pre-loaded three-dimensional application software in the electronic device to present the three-dimensional scene on the electronic device.

In a specific example, the execution subject may be an electronic device used by the user to browse the three-dimensional scene, such as a terminal computer or a smartphone. When a user uses an electronic device to browse a three-dimensional scene, the execution subject can determine the visible area in the three-dimensional scene based on the user's point and perspective, and then determine the three-dimensional model to be marked within the visible area as the target three-dimensional model.

In an optional example, step 110 may be performed by the processor calling corresponding instructions stored in the memory, or may be performed by the first determination unit run by the processor.

Step 120: Determine the reference point corresponding to the target 3D model and closest to the point position as the mark point corresponding to the target 3D model, and determine the reference plane where the mark point is located as the display plane corresponding to the target 3D model to obtain the target. Marking points and display planes corresponding to the 3D model.

In this embodiment, the marking point represents the marking position of the three-dimensional label, and the display plane is used to constrain the spatial posture of the three-dimensional label when it is marked to the marking point.

As an example, the execution subject can traverse at least one reference point corresponding to the target three-dimensional model, determine the distance between the at least one reference point and the point based on the spatial coordinates of the reference point and the point, and then determine the distance between the at least one reference point and the point. The reference point with the smallest distance among the points is determined as the mark point corresponding to the target 3D model, and the plane where the mark point is located is used as the display plane. By performing the above operations on at least one target three-dimensional model, the mark points and display planes corresponding to each target three-dimensional model are determined.

In an optional example, step 120 may be performed by the processor calling corresponding instructions stored in the memory, or may be performed by a second determination unit run by the processor.

Step 130: Determine the spatial pose of the three-dimensional label corresponding to the target three-dimensional model in the three-dimensional scene based on the mark points and display planes corresponding to the target three-dimensional model.

As an example, the execution subject may first determine the marking position of the three-dimensional label based on the marking point, for example, aligning or coinciding the attachment point of the three-dimensional label with the marking point in a specific direction; and then determine the spatial posture of the three-dimensional label based on the display plane, for example Make the length direction of the three-dimensional label parallel to the length direction or width direction of the display plane, make the plane on which the three-dimensional label displays label information coincide with the display plane, and determine the plane on which the three-dimensional label displays label information based on the normal direction of the display plane. Orientation to inherit the spatial characteristics of the display plane in the three-dimensional scene, for example, it can be the perspective characteristics of near, far, and small. In this way, the spatial pose of the three-dimensional label in the three-dimensional scene can be determined.

In an optional example, step 130 may be executed by the processor calling corresponding instructions stored in the memory, or may be executed by the posture determination unit run by the processor.

Step 140: Based on the spatial pose of the three-dimensional label corresponding to the target three-dimensional model in the three-dimensional scene, present the three-dimensional label corresponding to the three-dimensional model in the three-dimensional scene.

As an example, the execution subject can use the transform application in CSS (Cascading Style Sheet) to determine the three-dimensional matrix corresponding to the target three-dimensional model based on the mark points corresponding to the target three-dimensional model and the spatial position of the display plane; and then call one of the following or Multiple functions: Rotate3d(), translate3d(), perspective(), etc., use a three-dimensional matrix to perform 3D rotation, translation or perspective on the three-dimensional label corresponding to the target three-dimensional model. Through a visual and equal combination operation, the three-dimensional label is marked at the mark point with the spatial posture determined in step 130, thereby realizing the presentation of the three-dimensional label in the three-dimensional scene.

The marking method of the three-dimensional label may be, for example, that the attachment point of the three-dimensional label coincides with the marking point, or the three-dimensional label may be aligned with the marking point along a preset direction and then linked to the target three-dimensional model through a connection line. Different 3D models in the same 3D scene can be marked in a variety of ways. The attachment point represents a point in the three-dimensional label used to locate the mark position of the three-dimensional label. For example, it may be a corner point or center point of the three-dimensional label or other representative key points.

In an optional example, step 140 may be performed by the processor calling corresponding instructions stored in the memory, or may be performed by a label presentation unit run by the processor.

The marking method of the three-dimensional label will be further exemplified with reference to FIG. 2 . Figure 2 shows the presentation effect in one embodiment of the interactive presentation method of three-dimensional labels of the present disclosure. As shown in Figure 2, three-dimensional labels 111, 121, and 131 are used to represent the description information of three-dimensional models 110, 120, and 130 respectively. , where the three-dimensional labels 111 and 121 are located above the marking points 112 and 122 respectively, and are linked to the corresponding three-dimensional model through lines, and the three-dimensional label 131 coincides with the marking point 132.

The interactive presentation method of three-dimensional labels in this embodiment will be further exemplified below with reference to FIG. 3 . Figure 3 shows a schematic diagram of an application scenario of the interactive presentation method of three-dimensional tags of the present disclosure. The three-dimensional scene shown in Figure 3(a) includes a three-dimensional model 310 and a three-dimensional model 320. Among them, the three-dimensional model 310 corresponds to four Reference points (respectively 311, 312, 313, 314) and 4 reference planes (respectively 315, 316, 317, 318), the three-dimensional model 320 corresponds to 4 reference points (respectively 321, 322, 323, 324) and 4 reference planes (325, 326, 327, 328 respectively).

When the user uses point 300 to browse the three-dimensional scene, the three-dimensional model 310 (for example, it can be a three-dimensional model representing a sofa) is located within the user's perspective and is therefore visible to the user, while the three-dimensional model 320 is invisible, so the execution subject ( For example, the user's smartphone) may determine the three-dimensional model 310 as the target three-dimensional model. The execution subject can first determine the distances between the four reference points and the point 300, and then determine the reference point 311 with the smallest distance as the mark point. Correspondingly, the reference plane 315 is the display plane. Afterwards, the execution subject can determine the spatial pose of the three-dimensional label based on the marker point 311 and the display plane 315, and present it in the three-dimensional scene. The marked three-dimensional model 310 is shown in Figure 3(b), in which the three-dimensional label 330 coincides with the display plane 315, and the corner point of the three-dimensional label 330 coincides with the marked point 311.

Similarly, when the user uses point 340 to browse the three-dimensional scene, the three-dimensional model 320 is the target three-dimensional model. After the same processing process, the reference point 321 and the reference plane 325 can be determined as the mark point and the display plane respectively, thereby determining The spatial pose of the three-dimensional label corresponding to the three-dimensional model 320 is presented in the three-dimensional scene.

The interactive presentation method of 3D tags provided by this embodiment can determine at least one target 3D model from the 3D models visible to the user based on the user's point and perspective when browsing the 3D scene, and select the target 3D model that is closest to the point. The reference point is used as a marker point, and the reference plane where the marker point is located is determined as the display plane; then the spatial pose of the three-dimensional label is determined based on the marker point and the display plane, and finally at least one target is presented in the three-dimensional scene based on the spatial pose of the three-dimensional label The 3D label corresponding to the 3D model. It can ensure the matching degree of the spatial pose and point position of the 3D label, allowing users to obtain the information of the 3D model more intuitively and conveniently, thereby improving the display effect of the 3D label in the 3D scene.

In some optional implementations of this embodiment, when determining the marker point through step 120, if there are two or more reference points in the reference points corresponding to the target three-dimensional model that are closest to the point, two or more reference points are determined respectively. The reference plane where two or more reference points are located is projected within the viewing angle, and the reference point included in the reference plane with the largest projected area is determined as the mark point.

In this embodiment, the projection of the reference plane within the viewing angle can represent the presentation area of the reference plane within the viewing angle. The larger the presentation area, the easier it is for the user to check the information in the reference plane. According to the perspective principle, when the angle between the normal vector of the reference plane and the user's line of sight is small or large, the projection area of the reference plane within the viewing angle will be smaller, that is, the reference plane's presentation area within the viewing angle will be smaller. For example, when the angle between the normal vector of the reference plane and the user's line of sight is 0° or At 180°, the projection area of the reference plane within the viewing angle is 0. If the plane on which the 3D label displays information is located in the reference plane, the user cannot view the information in the 3D label.

On the contrary, when the normal vector of the reference plane is perpendicular to the user's line of sight, the reference plane faces the user. At this time, the projected area of the reference plane within the viewing angle reaches a maximum value. If the plane on which the three-dimensional label displays information is located in the reference plane, the user You can directly view the information in the three-dimensional label.

In this implementation, when there are two or more reference points closest to the point in the reference points corresponding to the target three-dimensional model at the same time, the reference point with better presentation effect can be selected based on the projected area of the reference plane within the viewing angle. Used as marking points to improve the display effect of three-dimensional labels.

In the process of realizing the present disclosure, the inventor found that when the height of the three-dimensional label is too high, the degree of spatial perspective will be greater. At this time, the visibility of the three-dimensional label and the readability of the text therein will decrease.

Considering this situation, in some optional implementations of this embodiment, if the height of the marker point corresponding to the target 3D model in the 3D scene is greater than the preset height, the display plane where the marker point is located is reduced along the height. Translate the three-dimensional label corresponding to the target three-dimensional model in a small direction to reduce the height of the three-dimensional label corresponding to the target three-dimensional model in the three-dimensional scene.

The following is an exemplary description of the presentation method of the three-dimensional label in this embodiment with reference to Figure 4. Figure 4(a) and Figure 4(b) show two presentation methods respectively. Figure 4(a) shows the three-dimensional label before adjusting the height. Label schematic diagram, Figure 4(b) is a schematic diagram of the three-dimensional label after adjusting the height. When the mark point 411 corresponding to the three-dimensional model 410 is higher than the preset height (for example, it can be 1.6m), using the presentation method of Figure 4(a) will result in a greater degree of spatial perspective of the three-dimensional label 420, and its visibility and information The readability is correspondingly lower. Using the presentation method of FIG. 4(b) , the height of the three-dimensional label 420 in the three-dimensional scene can be reduced, correspondingly reducing its degree of spatial perspective, thereby achieving better visibility and readability.

In this implementation, the height of the mark point corresponding to the target 3D model in the 3D scene is greater than the preset height. By reducing the height of the 3D label, the information displayed by the 3D label is located in an area that is easier for the user's sight to observe, thereby obtaining better results. Good visibility and readability.

Referring next to Figure 5, Figure 5 shows a flow chart of yet another embodiment of the interactive presentation method of three-dimensional labels of the present disclosure. As shown in Figure 5, after the above step 140, the method may also include the following steps:

Step 510: Determine the visible area in the three-dimensional scene based on the point position and perspective.

In this embodiment, the visible area represents the area that can be observed by the user when browsing the three-dimensional scene from the current point of view at the current point, that is, the area presented to the user device.

As an example, the execution subject can construct a viewing cone of the three-dimensional scene based on the point position and perspective. The area within the viewing cone space is the visible area of the three-dimensional scene.

In an optional example, step 510 may be performed by the processor calling corresponding instructions stored in the memory, or may be performed by a region determination unit run by the processor.

Step 520: Obtain the projected image of the visible area in the user device.

Among them, the user device is an electronic device used by the user to browse the three-dimensional scene.

In this embodiment, the projected image represents an image displayed in the screen of the user device. It is a projection image formed by projecting a three-dimensional scene onto the screen of the user device by an execution subject (for example, it can be a user device) combined with the display parameters of the user device (such as screen resolution).

In an optional example, step 520 may be performed by the processor calling corresponding instructions stored in the memory, or may be performed by an image acquisition determination unit run by the processor.

Step 530: Determine the pixel distance between corresponding mark points of at least one target three-dimensional model in the projection image.

As an example, the marker points corresponding to the target three-dimensional model 1 and the target three-dimensional model 2 are marker point a and marker point b respectively. The projections of marker point a and marker point b in the projection image are pixel point A and pixel point B respectively, then the length of line segment AB is the pixel distance between marker point a and marker point b.

In an optional example, step 530 may be performed by the processor calling corresponding instructions stored in the memory, or may be performed by a distance determination unit run by the processor.

Step 540: If the pixel distance between the mark points corresponding to the two target 3D models is less than the preset distance, hide the 3D label corresponding to one of the target 3D models.

For example, when the pixel distance between the mark points corresponding to the two target 3D models is less than the preset distance, the execution subject can randomly hide the 3D label corresponding to one of the target 3D models. Alternatively, the execution subject can hide marked points further away from the point.

In an optional example, step 540 may be performed by the processor calling corresponding instructions stored in the memory, or may be performed by a tag hiding unit run by the processor.

The embodiment shown in Figure 5 embodies the following: when the pixel distance between the mark points corresponding to the two target 3D models in the projected image of the user device is less than the preset distance, the 3D label corresponding to one of the target 3D models is hidden. It can avoid occlusion or overlap caused by three-dimensional labels that are too close, which helps to further improve the presentation effect of three-dimensional labels.

Next, refer to Figure 6, which shows a flow chart of yet another embodiment of the interactive presentation method of three-dimensional labels of the present disclosure. As shown in Figure 6, based on the process shown in Figure 2 or Figure 5, step 140 The method may then further include the following steps:

Step 610: When the user changes the point and/or perspective while browsing the three-dimensional scene, determine a new target three-dimensional model based on the new point and/or new perspective.

Generally, when users browse a three-dimensional scene, in order to view the three-dimensional scene more comprehensively, they can change points and perspectives to obtain information in the three-dimensional scene from multiple positions and angles. When the user changes the point and/or perspective, the visible area in the 3D scene and the visible 3D model usually change accordingly.

In this embodiment, the new target three-dimensional model represents the three-dimensional model to be marked that can be observed by the user after changing the point and/or perspective.

It can be understood that the new target three-dimensional model may include the old target three-dimensional model before changing the point position and/or perspective.

In an optional example, step 610 may be performed by the processor calling corresponding instructions stored in the memory, or may be performed by an update unit run by the processor.

Step 620: For the new target 3D model, perform the operation of determining the marker point and the display plane and the operation of determining the spatial position of the 3D label in the 3D scene again to obtain the spatial position of the 3D label corresponding to the new target 3D model in the 3D scene. posture.

For example, the execution subject can execute the above steps 120 to 130 again for the new target 3D model, as well as the optional implementation methods corresponding to each step, to determine the spatial pose of the 3D tag corresponding to the travel target 3D model in the 3D scene. .

In an optional example, step 620 may be performed by the processor calling corresponding instructions stored in the memory, or may be performed by an iterative unit run by the processor.

Step 630: Based on the spatial pose of the three-dimensional label corresponding to the new target three-dimensional model in the three-dimensional scene, present the three-dimensional label corresponding to the new target three-dimensional model in the three-dimensional scene.

For example, the execution subject can perform the above step 150 again to present the three-dimensional label corresponding to the new target three-dimensional model in the three-dimensional scene.

For another example, after the execution body performs step 150 again, it may also perform the above-mentioned steps 510 to 540 again to further improve the display effect of the three-dimensional label.

Further illustrating with reference to Figure 3, when the user changes from point 300 to point 340 during the browsing process, the execution subject can determine the three-dimensional model 320 as the new target three-dimensional model, and execute the above steps 120 to 140 again, The three-dimensional label corresponding to the three-dimensional model 320 can be presented in the three-dimensional scene.

In an optional example, step 630 may be performed by the processor calling corresponding instructions stored in the memory, or may be performed by a presentation unit run by the processor.

In the embodiment shown in Figure 6, when the user changes the point and/or the angle of view, the user can change the point and/or angle of view according to the new point and/or angle of view. Synchronously updating the 3D labels presented in the 3D scene can further improve the interaction intelligence and display effect between the 3D scene and the user.

In some optional implementations of this embodiment, before step 630, when the user changes points and/or perspectives while browsing the three-dimensional scene, the three-dimensional labels corresponding to at least one target three-dimensional model are hidden.

By hiding the old 3D labels, on the one hand, the memory consumption when browsing the 3D scene can be reduced, and on the other hand, the conflict between the old 3D labels and the new 3D labels can be avoided.

Any interactive presentation method of three-dimensional labels provided by the embodiments of the present disclosure can be executed by any appropriate device with data processing capabilities, including but not limited to: terminal devices and servers. Alternatively, any of the interactive presentation methods of three-dimensional labels provided in the embodiments of the present disclosure can be executed by the processor. For example, the processor executes the interactive presentation method of the three-dimensional labels mentioned in the embodiments of the present disclosure by calling corresponding instructions stored in the memory. No further details will be given below.

Those of ordinary skill in the art can understand that all or part of the steps to implement the above method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, It includes the steps of the above method embodiment; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other various media that can store program codes.

Referring now to FIG. 7 , FIG. 7 shows a schematic structural diagram of an embodiment of a three-dimensional label interactive presentation device of the present disclosure. The device of this embodiment can be used to implement the above method embodiments of the present disclosure. As shown in Figure 7, the device includes: a first determining unit 710 configured to determine at least one target three-dimensional model from the three-dimensional models visible to the user based on the user's point and perspective when browsing the three-dimensional scene. The target three-dimensional scene The three-dimensional model corresponds to a three-dimensional label to be marked, a plurality of reference points for marking the three-dimensional label, and the reference plane where each reference point is located; the second determination unit 720 is configured to determine the nearest point in the target three-dimensional model. The reference point is determined as the mark point corresponding to the target three-dimensional model, and the reference plane where the mark point is located is determined as the display plane corresponding to the target three-dimensional model, and the mark point and display plane corresponding to the target three-dimensional model are obtained; pose determination unit 730 , is configured to determine the spatial pose of the three-dimensional label corresponding to the target three-dimensional model in the three-dimensional scene based on the mark points and display planes corresponding to the target three-dimensional model; the label presentation unit 740 is configured to determine the spatial pose of the three-dimensional label corresponding to the target three-dimensional model based on the three-dimensional label corresponding to the target three-dimensional model. The spatial pose in the three-dimensional scene presents the three-dimensional label corresponding to the three-dimensional model in the three-dimensional scene.

In one of the embodiments, the device further includes: an area determination unit configured to determine the visible area in the three-dimensional scene based on the point and perspective; an image acquisition unit configured to acquire the image of the visible area in the user equipment. Projected image, the user equipment is an electronic device used by the user to browse the three-dimensional scene; the distance determination unit is configured to determine the pixel distance between the mark points respectively corresponding to at least one target three-dimensional model in the projected image; the label hiding unit is configured If the pixel distance between the mark points corresponding to the two target 3D models is less than the preset distance, the 3D label corresponding to one of the target 3D models is hidden.

In one of the embodiments, the label presentation unit 740 also includes an adjustment module configured to reduce the height along the height in the display plane where the marker point is located if the height of the marker point corresponding to the target three-dimensional model is greater than the preset height in the three-dimensional scene. Translate the three-dimensional label corresponding to the target three-dimensional model in a small direction to reduce the height of the three-dimensional label corresponding to the target three-dimensional model in the three-dimensional scene.

In one of the embodiments, the first determination unit 710 also includes a screening module configured to determine if there are two or more reference points closest to the point position in the reference points corresponding to the target three-dimensional model. The projection of the reference plane where more than one reference point is located within the viewing angle, and the reference point included in the reference plane with the largest projected area is determined as the mark point.

In one of the embodiments, the device further includes: an update unit configured to determine a new target 3D target based on the new point and/or new perspective when the user changes the point and/or perspective while browsing the three-dimensional scene. model; the iteration unit is configured to once again perform the operation of determining the marker point and display plane and the operation of determining the spatial pose of the three-dimensional label in the three-dimensional scene for the new target three-dimensional model, and obtain the three-dimensional label corresponding to the new target three-dimensional model in the three-dimensional scene. The spatial pose in the scene; the presentation unit is configured to present the three-dimensional label corresponding to the new target three-dimensional model in the three-dimensional scene based on the spatial pose of the three-dimensional label corresponding to the new target three-dimensional model in the three-dimensional scene.

In one of the embodiments, the device further includes a hiding unit configured to hide the three-dimensional labels corresponding to at least one target three-dimensional model when the user changes points and/or perspectives while browsing the three-dimensional scene.

In addition, embodiments of the present disclosure also provide an electronic device, including:

Memory, used to store computer programs;

A processor, configured to execute a computer program stored in the memory, and when the computer program is executed, implement the interactive presentation method of a three-dimensional label described in any of the above embodiments of the present disclosure.

In addition, embodiments of the present disclosure also provide a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the interactive presentation method of a three-dimensional label according to any of the above embodiments can be implemented.

In addition, an embodiment of the present disclosure also provides a computer program product, which includes computer program instructions. When the computer program instructions are executed by a processor, the interactive presentation method of a three-dimensional label according to any of the above embodiments of the present disclosure can be implemented.

FIG. 8 is a schematic structural diagram of an application embodiment of the electronic device of the present disclosure. Next, an electronic device according to an embodiment of the present disclosure is described with reference to FIG. 8 . As shown in Figure 8, the electronic device includes one or more processors and memory.

The processor may be a central processing unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

Memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and/or cache memory (cache). The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor may execute the program instructions to implement the interactive presentation method of three-dimensional labels and/or the various embodiments of the present disclosure described above. or other desired functionality.

In one example, the electronic device may further include an input device and an output device, and these components are interconnected through a bus system and/or other forms of connection mechanisms (not shown).

In addition, the input device may also include, for example, a keyboard, a mouse, and the like.

The output device can output various information to the outside, including determined distance information, direction information, etc. The output devices may include, for example, displays, speakers, printers, and communication networks and remote output devices to which they are connected, among others.

Of course, for simplicity, only some of the components in the electronic device related to the present disclosure are shown in FIG. 8 , and components such as buses, input/output interfaces, etc. are omitted. In addition to this, the electronic device may include any other suitable components depending on the specific application.

In addition to the above methods and devices, embodiments of the present disclosure may also be a computer program product, which includes computer program instructions that, when executed by a processor, cause the processor to perform the steps described in the above part of this specification. Steps in the interactive presentation method of three-dimensional labels according to various embodiments of the present disclosure.

The computer program product may be written with program code for performing operations of embodiments of the present disclosure in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc. , also includes conventional procedural programming languages, such as the "C" language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server execute on.

In addition, embodiments of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon. The computer program instructions, when executed by a processor, cause the processor to perform the steps described in the above part of this specification according to the present invention. Steps in the interactive presentation method of three-dimensional labels of various embodiments are disclosed.

The computer-readable storage medium may be any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: electrical connection with one or more wires, portable disk, hard disk, random access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or the above any suitable combination.

The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, it should be pointed out that the advantages, advantages, effects, etc. mentioned in the present disclosure are only examples and not limitations. These advantages, advantages, effects, etc. cannot be considered to be Each embodiment of the present disclosure must have. In addition, the specific details disclosed above are only for the purpose of illustration and to facilitate understanding, and are not limiting. The above details do not limit the present disclosure to be implemented by using the above specific details.

Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on its differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple. For relevant details, please refer to the partial description of the method embodiment.

The block diagrams of the devices, devices, equipment, and systems involved in the present disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, devices, equipment, and systems may be connected, arranged, and configured in any manner. Words such as "includes," "includes," "having," etc. are open-ended terms that mean "including, but not limited to," and may be used interchangeably therewith. As used herein, the words "or" and "and" refer to the words "and/or" and are used interchangeably therewith unless the context clearly dictates otherwise. As used herein, the word "such as" refers to the phrase "such as, but not limited to," and may be used interchangeably therewith.

The methods and apparatus of the present disclosure may be implemented in many ways. For example, the methods and devices of the present disclosure may be implemented through software, hardware, firmware, or any combination of software, hardware, and firmware. The above order for the steps of the methods is for illustration only, and the steps of the methods of the present disclosure are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, the present disclosure may also be implemented as programs recorded in recording media, and these programs include machine-readable instructions for implementing methods according to the present disclosure. Thus, the present disclosure also covers recording media storing programs for executing methods according to the present disclosure.

It should also be noted that in the devices, equipment and methods of the present disclosure, each component or each step can be decomposed and/or recombined. These decompositions and/or recombinations should be considered equivalent versions of the present disclosure.

The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The foregoing description has been presented for the purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of the present disclosure to the form disclosed herein. Although various example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

An interactive presentation method of three-dimensional labels, which is characterized by including:

Based on the user's point and perspective when browsing the three-dimensional scene, at least one target three-dimensional model is determined from the three-dimensional models visible to the user, and the target three-dimensional model corresponds to one three-dimensional label to be marked and a plurality of three-dimensional labels for marking three-dimensional labels. reference points and the reference plane on which each said reference point lies;

The reference point corresponding to the target three-dimensional model and closest to the point is determined as the mark point corresponding to the target three-dimensional model, and the reference plane where the mark point is located is determined as the display plane corresponding to the target three-dimensional model. , obtain the mark points and display plane corresponding to the target three-dimensional model;

Based on the marker points and display planes corresponding to the target three-dimensional model, determine the spatial pose of the three-dimensional label corresponding to the target three-dimensional model in the three-dimensional scene;

Based on the spatial pose of the three-dimensional label corresponding to the target three-dimensional model in the three-dimensional scene, the three-dimensional label corresponding to the target three-dimensional model is presented in the three-dimensional scene.
The method according to claim 1, characterized in that, based on the spatial pose of the three-dimensional label corresponding to the target three-dimensional model in the three-dimensional scene, the three-dimensional label corresponding to the target three-dimensional model is presented in the three-dimensional scene. Afterwards, the method further includes:

Based on the point position and the viewing angle, determine the visible area in the three-dimensional scene;

Obtain the projected image of the visible area in a user device, where the user device is an electronic device used by the user to browse the three-dimensional scene;

Determine the pixel distance between the corresponding mark points of at least one of the target three-dimensional models in the projection image;

If the pixel distance between the corresponding mark points of the two target three-dimensional models is less than the preset distance, then the three-dimensional label corresponding to one of the target three-dimensional models is hidden.
The method according to claim 1, characterized in that, based on the spatial pose of the three-dimensional label corresponding to the target three-dimensional model in the three-dimensional scene, the three-dimensional label corresponding to the target three-dimensional model is presented in the three-dimensional scene. ,include:

If the height of the marker point corresponding to the target three-dimensional model in the three-dimensional scene is greater than the preset height, translate the three-dimensional label corresponding to the target three-dimensional model in the direction of decreasing height in the display plane where the marker point is located, so as to Reduce the height of the three-dimensional label corresponding to the target three-dimensional model in the three-dimensional scene.
The method according to claim 1, characterized in that determining the reference point corresponding to the target three-dimensional model and closest to the point position as the mark point corresponding to the target three-dimensional model includes:

If there are two or more reference points closest to the point in the reference points corresponding to the target three-dimensional model, respectively determine the reference plane where the two or more reference points are located within the viewing angle. Project, and determine the reference point included in the reference plane with the largest projected area as the mark point.
The method according to any one of claims 1 to 4, characterized in that, based on the spatial pose of the three-dimensional label corresponding to the target three-dimensional model in the three-dimensional scene, the target three-dimensional model is presented in the three-dimensional scene. After corresponding three-dimensional labels, the method also includes:

When the user changes points and/or perspectives while browsing the three-dimensional scene, determine a new target three-dimensional model based on the new points and/or new perspectives;

For the new target 3D model, the operation of determining the marker point and the display plane and the operation of determining the spatial pose of the 3D label in the 3D scene are performed again to obtain the 3D label corresponding to the new target 3D model. The spatial pose in the three-dimensional scene;

Based on the spatial pose of the three-dimensional label corresponding to the new target three-dimensional model in the three-dimensional scene, the three-dimensional label corresponding to the new target three-dimensional model is presented in the three-dimensional scene.
The method according to claim 5, characterized in that, based on the spatial pose of the three-dimensional label corresponding to the new target three-dimensional model in the three-dimensional scene, the three-dimensional label corresponding to the new three-dimensional model is presented in the three-dimensional scene. Before labeling, the method also includes:

When the user changes points and/or perspectives while browsing the three-dimensional scene, three-dimensional labels corresponding to at least one of the target three-dimensional models are hidden.
An interactive presentation device for three-dimensional labels, which is characterized by including:

The first determining unit is configured to determine at least one target three-dimensional model from the three-dimensional models visible to the user based on the user's point position and perspective when the user browses the three-dimensional scene. The target three-dimensional model corresponds to a three-dimensional label to be marked and A plurality of reference points for marking three-dimensional labels and a reference plane on which each of said reference points is located;

The second determination unit is configured to determine the reference point corresponding to the target three-dimensional model and closest to the point as the mark point corresponding to the target three-dimensional model, and determine the reference plane where the mark point is located as the reference point. Determine the display plane corresponding to the target three-dimensional model, and obtain the mark points and display plane corresponding to the target three-dimensional model;

The pose determination unit is configured to determine the spatial pose of the three-dimensional label corresponding to the target three-dimensional model in the three-dimensional scene based on the marker point corresponding to the target three-dimensional model and the display plane;

The label presentation unit is configured to present the three-dimensional label corresponding to the three-dimensional model in the three-dimensional scene based on the spatial pose of the three-dimensional label corresponding to the target three-dimensional model in the three-dimensional scene.
The device of claim 7, further comprising:

an area determination unit configured to determine the visible area in the three-dimensional scene based on the point and the viewing angle;

An image acquisition unit configured to acquire a projected image of the visible area in a user device, where the user device is an electronic device used by the user to browse the three-dimensional scene;

a distance determination unit configured to determine the pixel distance between mark points respectively corresponding to at least one of the target three-dimensional models in the projection image;

The label hiding unit is configured to hide the three-dimensional label corresponding to one of the target three-dimensional models if the pixel distance between the corresponding mark points of the two target three-dimensional models is less than a preset distance.
The device according to claim 7, wherein the label presentation unit further includes an adjustment module configured to: if the height of the marker point corresponding to the target three-dimensional model in the three-dimensional scene is greater than a preset height, The three-dimensional label corresponding to the target three-dimensional model is translated in the display plane where the marker point is located in the direction of height reduction, so as to reduce the height of the three-dimensional label corresponding to the target three-dimensional model in the three-dimensional scene.
The device according to claim 7, wherein the first determination unit further includes a screening module configured to: if there are two or more reference point distances in the reference points corresponding to the target three-dimensional model at the same time, If the point is the closest, the projection of the reference plane where the two or more reference points are located in the viewing angle is determined respectively, and the reference point included in the reference plane with the largest projection area is determined as the mark point.
The device according to any one of claims 7 to 10, characterized in that the device further includes: an update unit configured to: when the user changes points and/or perspectives while browsing the three-dimensional scene, Based on the new point and/or new perspective, a new target three-dimensional model is determined; for the new target three-dimensional model, the operation of determining the marker point and the display plane and the determining the space of the three-dimensional label in the three-dimensional scene are performed again. Through the operation of pose, the new target three is obtained. The spatial pose of the three-dimensional label corresponding to the three-dimensional model in the three-dimensional scene; based on the spatial pose of the three-dimensional label corresponding to the three-dimensional model of the new target in the three-dimensional scene, the new target is presented in the three-dimensional scene The 3D label corresponding to the 3D model.
The device according to claim 11, characterized in that the device further includes a hiding unit configured to: when the user changes points and/or perspectives while browsing the three-dimensional scene, hide at least one of the The three-dimensional labels corresponding to the target three-dimensional models.
An electronic device, characterized by including:

Memory for storing computer program products;

A processor, configured to execute a computer program product stored in the memory, and when the computer program product is executed, implement the method described in any one of claims 1 to 6 above.
A computer-readable storage medium on which computer program instructions are stored, characterized in that when the computer program instructions are executed by a processor, the method described in any one of the above claims 1-6 is implemented.
A computer program product, including computer program instructions, characterized in that when the computer program instructions are executed by a processor, the method described in one of the above claims 1-6 is implemented.