CN110826375B - Display method, display device, terminal equipment and storage medium - Google Patents

Display method, display device, terminal equipment and storage medium Download PDF

Info

Publication number
CN110826375B
CN110826375B CN201810910950.6A CN201810910950A CN110826375B CN 110826375 B CN110826375 B CN 110826375B CN 201810910950 A CN201810910950 A CN 201810910950A CN 110826375 B CN110826375 B CN 110826375B
Authority
CN
China
Prior art keywords
markers
marker
spatial
virtual object
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810910950.6A
Other languages
Chinese (zh)
Other versions
CN110826375A (en
Inventor
尹元庆
戴景文
贺杰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Virtual Reality Technology Co Ltd
Original Assignee
Guangdong Virtual Reality Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Virtual Reality Technology Co Ltd filed Critical Guangdong Virtual Reality Technology Co Ltd
Priority to CN201810910950.6A priority Critical patent/CN110826375B/en
Priority to PCT/CN2019/097128 priority patent/WO2020020102A1/en
Priority to US16/666,429 priority patent/US11049324B2/en
Publication of CN110826375A publication Critical patent/CN110826375A/en
Application granted granted Critical
Publication of CN110826375B publication Critical patent/CN110826375B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1407—General aspects irrespective of display type, e.g. determination of decimal point position, display with fixed or driving decimal point, suppression of non-significant zeros
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • Software Systems (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Processing Or Creating Images (AREA)

Abstract

The embodiment of the application discloses a display method, a display device, terminal equipment and a storage medium, and relates to the technical field of display. The display method is applied to terminal equipment, and comprises the following steps: acquiring an image containing a target marker, wherein the target marker is at least one marker in a plurality of markers within the visual field range of the terminal equipment, and the plurality of markers are separately arranged at different positions; acquiring a target space position of the target marker relative to the terminal equipment according to the image; reading a stored first spatial position relationship among the plurality of markers, and acquiring a second spatial position relationship of the virtual object relative to the plurality of markers; determining a display position of the virtual object based on the first spatial positional relationship, the second spatial positional relationship, and the target spatial position; and displaying the virtual object according to the display position.

Description

Display method, display device, terminal equipment and storage medium
Technical Field
The present application relates to the field of display technologies, and in particular, to a display method, an apparatus, a terminal device, and a storage medium.
Background
In recent years, with the progress of science and technology, technologies such as Augmented Reality (AR) have become hot spots of research at home and abroad, and Augmented Reality is a technology for increasing the perception of a user to the real world through information provided by a computer system, in which a virtual object generated by a computer, a scene, or a content object such as system prompt information is superimposed on a real scene to enhance or modify the perception of the real world environment or data representing the real world environment.
In most cases, when the virtual content is displayed by using the terminal device, the virtual content is displayed according to the identification result of the marker after the marker acquired image is identified. However, the identification and tracking of the marker may be affected by the visual field of the terminal device, so that the terminal device may easily lose the tracking of the marker, and the display of the terminal device may be affected.
Disclosure of Invention
The embodiment of the application provides a display method, a display device, terminal equipment and a storage medium, which can enhance the display stability of the terminal equipment.
In a first aspect, an embodiment of the present application provides a display method, which is applied to a terminal device, and the method includes: acquiring an image containing a target marker, wherein the target marker is at least one marker in a plurality of markers within the visual field range of the terminal equipment, and the plurality of markers are separately arranged at different positions; acquiring a target space position of the target marker relative to the terminal equipment according to the image; reading a stored first spatial position relationship among the plurality of markers, and acquiring a second spatial position relationship of the virtual object relative to the plurality of markers; determining a display position of the virtual object based on the first spatial positional relationship, the second spatial positional relationship, and the target spatial position; and displaying the virtual object according to the display position.
In a second aspect, an embodiment of the present application provides a display apparatus, which is applied to a terminal device, and the apparatus includes: the device comprises an image acquisition module, a first position acquisition module, a position relation reading module, a second position acquisition module and a display execution module. The image acquisition module is used for acquiring an image containing a target marker, wherein the target marker is at least one marker in a plurality of markers within the visual field range of the terminal equipment, and the markers are separately arranged at different positions; the first position acquisition module is used for acquiring a target space position of the target marker relative to the terminal equipment according to the image; the position relation reading module is used for reading a stored first space position relation among the plurality of markers and acquiring a second space position relation of the virtual object relative to the plurality of markers; the second position obtaining module is configured to determine a display position of the virtual object based on the first spatial position relationship, the second spatial position relationship, and the target spatial position; the display execution module is used for displaying the virtual object according to the display position.
In a third aspect, an embodiment of the present application provides a terminal device, including: one or more processors; a memory; one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to perform the display method provided by the first aspect above.
In a fourth aspect, an embodiment of the present application provides a storage medium, where a program code is stored in the computer-readable storage medium, and the program code may be called by a processor to execute the display method provided in the first aspect.
According to the scheme provided by the application, the terminal equipment acquires an image containing a target object, the target object is at least one of the plurality of markers in the visual field range of the terminal equipment, the plurality of markers are separately arranged at different positions, then the target space position of the target marker relative to the terminal equipment is acquired according to the image, the stored first space position relation among the plurality of markers is read, the second space position relation of the virtual object relative to the plurality of markers is acquired, the display position of the virtual object is determined based on the first space position relation, the second space position relation and the target space position among the plurality of markers, finally the virtual object is displayed according to the display position, the terminal equipment identifies one marker in the plurality of markers in a large range, and the display of the virtual object can be realized, the display stability of the virtual object corresponding to the terminal equipment is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 shows a schematic diagram of an application scenario suitable for use in an embodiment of the present application.
FIG. 2 shows a flow diagram of a display method according to one embodiment of the present application.
FIG. 3 shows a flow diagram of a display method according to another embodiment of the present application.
Fig. 4 shows a schematic usage scenario diagram of an implementation manner provided by an embodiment of the present application.
Fig. 5 is a schematic view illustrating a usage scenario of another implementation manner provided in an embodiment of the present application.
Fig. 6 is a schematic view illustrating another usage scenario of another implementation manner provided in the embodiment of the present application.
Fig. 7 is a schematic view illustrating another usage scenario of another implementation manner provided in the embodiment of the present application.
Fig. 8 is a schematic view illustrating another usage scenario of another implementation manner provided in the embodiment of the present application.
Fig. 9 is a schematic view illustrating a usage scenario in an alternative implementation manner provided in an embodiment of the present application.
Fig. 10 is a schematic view illustrating another usage scenario of an alternative implementation manner in another implementation manner provided by an embodiment of the present application.
Fig. 11 shows a flowchart of step S240 in the display method according to the embodiment of the present application.
Fig. 12 shows a flowchart of step S260 in the display method according to the embodiment of the present application.
FIG. 13 shows a block diagram of a display device according to an embodiment of the present application.
Fig. 14 illustrates a block diagram of a second position acquisition module in a display device according to an embodiment of the present application.
FIG. 15 shows another block diagram of a display device according to an embodiment of the present application.
Fig. 16 is a block diagram of a terminal device for executing a display method according to an embodiment of the present application.
Fig. 17 is a storage unit according to an embodiment of the present application, configured to store or carry program code for implementing a display method according to an embodiment of the present application.
Detailed Description
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application.
In most application scenarios of the augmented reality technology, a marker is set in the application scenario, and the marker is used for displaying virtual content. When the terminal equipment displays the virtual content, the camera and other image acquisition devices of the terminal equipment are used for imaging the marker, then the information of the marker is identified according to the imaging to obtain an identification result, and the virtual content is displayed according to the identification result.
Moreover, a marker is arranged in most application scenes, and the terminal equipment superimposes the virtual object on the real scene by identifying the marker. However, since the field of view of the terminal device is limited, when the marker in the scene is lost in the field of view of the terminal device, the information of the marker in the scene cannot be recognized, and the terminal device cannot display the virtual content. For example, when the application scene is a tablecloth of a desktop, the marker is arranged at a position of the desktop, the terminal device displays the virtual tablecloth on the desktop according to the marker information after recognizing the information of the marker, but when the position of the terminal device changes and the marker is lost in the visual field range of the terminal device, the information of the marker cannot be recognized, so that the virtual tablecloth cannot be displayed on the desktop.
In view of the above problems, the inventors have studied and proposed the display method, apparatus, terminal device and storage medium provided in the embodiments of the present application for a long time, so as to improve the stability of the display of the terminal device.
An application scenario of the display method provided in the embodiment of the present application is described below.
Referring to fig. 1, a schematic diagram of an application scenario of the display method provided in the embodiment of the present application is shown, where the application scenario includes a display system 10. The display system 10 includes: a terminal device 100 and a plurality of markers 200.
Wherein, the plurality of markers 200 can be separately arranged at different positions in the scene. For example, the markers 200 may be separately installed on the tabletop or separately installed in a region of the ground, and of course, the actual installation scenarios of the markers are not limited in the embodiments of the present application.
In the embodiment of the present application, the terminal device 100 may be a head-mounted display device, or may be a mobile device such as a mobile phone and a tablet. When the terminal device 100 is a head-mounted display device, the head-mounted display device may be an integrated head-mounted display device. The terminal device 100 may also be an intelligent terminal such as a mobile phone connected to an external head-mounted display device, that is, the terminal device 100 may be inserted or connected to the external head-mounted display device as a processing and storage device of the head-mounted display device, and perform a display function on a virtual object in the head-mounted display device.
In the embodiment of the present application, the images of the plurality of markers 200 described above are stored in the terminal device 100. Each of the plurality of markers 200 may include at least one sub-marker having one or more feature points. When any one or more of the plurality of markers 200 is within the visual field of the terminal device 100, the terminal device 100 may use one of the plurality of markers 200 that is within the visual field as a target marker and acquire an image including the target marker. When the image containing the target marker is acquired, the acquired image of the target marker can be identified, and spatial position information such as the position and the orientation of the target marker relative to the terminal equipment and identification results such as identity information of the target marker can be obtained. The terminal device may display the corresponding virtual object based on the spatial position relationship between the plurality of markers 200, the spatial position of the target marker with respect to the terminal device, and the like. It is to be understood that the specific markers are not limited in the embodiments of the present application, and only need to be identified and tracked by the terminal device.
Referring to fig. 2, an embodiment of the present application provides a display method, which is applicable to a terminal device, and the method may include:
step S110: and acquiring an image containing a target marker, wherein the target marker is at least one marker in a plurality of markers within the visual field range of the terminal equipment, and the plurality of markers are separately arranged at different positions.
In the embodiment of the present application, the plurality of markers are separately disposed at different positions in an actual scene, the plurality of markers are at least two markers, and the positions at which the plurality of markers are disposed may be fixed or may be changed at any time according to a requirement. When the terminal device needs to display the virtual object, image acquisition can be performed on a target marker existing in the visual field range of the terminal device, wherein the target marker is at least one marker existing in the visual field range of the terminal device in the plurality of markers. The field of view of the terminal device refers to the field of view of the image capturing device of the terminal device, and the field of view of the image capturing device may be determined by the size of the field of view.
In an embodiment of the present application, the marker may include at least one sub-marker, and the sub-marker may be a pattern having a certain shape. In one embodiment, each sub-marker may have one or more feature points, wherein the shape of the feature points is not limited, and may be a dot, a ring, a triangle, or other shapes. In addition, the distribution rules of the sub-markers within different markers are different, and thus, each marker may have different identity information. The terminal device may acquire identity information corresponding to the marker by recognizing the sub-marker included in the marker, and the identity information may be information that can be used to uniquely identify the marker, such as a code, but is not limited thereto.
In one embodiment, the outline of the marker may be rectangular, but the shape of the marker may be other shapes, and the rectangular region and the plurality of sub-markers in the region constitute one marker. Of course, the marker may also be an object which is composed of a light spot and can emit light, the light spot marker may emit light of different wavelength bands or different colors, and the terminal device acquires the identity information corresponding to the marker by identifying information such as the wavelength band or the color of the light emitted by the light spot marker. Of course, the specific marker is not limited in the embodiment of the present application, and the marker only needs to be recognized by the terminal device.
Thus, an image containing the above-mentioned target marker can be obtained for subsequent determination of the position of the virtual object.
Step S120: and acquiring the target space position of the target marker relative to the terminal equipment according to the image.
After the terminal device obtains the image containing the target marker, the image containing the target marker can be identified to obtain an identification result of the target marker. The identification result of the target marker may include a target spatial position of the target marker relative to the terminal device, identity information of the target marker, and the like. The target spatial position may include a position of the target marker relative to the terminal device, posture information, and the like, where the posture information is an orientation and a rotation angle of the target marker relative to the terminal device.
Thus, the target spatial position of the target marker relative to the terminal device can be obtained.
Step S130: and reading the stored first spatial position relation among the plurality of markers, and acquiring a second spatial position relation of the virtual object relative to the plurality of markers.
In this embodiment of the application, before the displaying of the virtual object is performed this time, the terminal device may acquire one or more images, where the one or more images may include multiple markers separately set at different positions, identify the multiple markers according to the one or more images, establish a first spatial position relationship between the multiple markers, and store the first spatial position relationship in the terminal device, and based on the first spatial position relationship, may obtain relative spatial positions between the different markers.
Of course, in the case that the positions of the plurality of markers are fixed, after the first implementation of the display method, the first spatial position relationships of the plurality of markers are stored in the terminal device, and it is not necessary to obtain the first spatial position relationships between the plurality of markers when the display method is implemented subsequently, that is, the first spatial position relationships read when the display method is implemented subsequently are the first spatial position relationships between the plurality of markers obtained when the display method is implemented firstly. For example, in an indoor scene such as a museum, a plurality of markers may be respectively and fixedly set at different positions, the set positions may be pre-stored in the terminal device, and when a user enters the museum, an image of any one of the markers is acquired, that is, information such as positions and postures of other markers may be obtained according to a pre-stored first spatial position relationship between the plurality of markers, and a corresponding virtual object is displayed.
In addition, the terminal device may obtain a second spatial position relationship of the virtual object with respect to the plurality of markers, that is, a position where the virtual object needs to be displayed and a position relationship between the plurality of markers, where the second spatial position relationship may be set according to actual requirements, for example, the second spatial position relationship may be a position where the virtual object is located at a center of an area formed by the plurality of markers, or the second spatial position relationship may be where the virtual object is located on one of the markers, but is not limited thereto. The second spatial position relationship may be stored in the terminal device in advance, or may be selected by the user in real time.
Step S140: and determining the display position of the virtual object based on the first spatial position relation, the second spatial position relation and the target spatial position.
After the target spatial position of the target marker relative to the terminal device is obtained, the positional relationship of the virtual object relative to the target marker may be determined according to the first spatial positional relationship between the plurality of markers and the positional relationship of the virtual object relative to the plurality of markers.
In a traditional mode, when a terminal device displays a virtual object, a display position of the virtual object can be accurately obtained only by ensuring that a plurality of markers exist in a visual field range at the same time, so that the identification and tracking range of the markers is limited. In the embodiment of the application, the terminal device may obtain the position relationship of the virtual object relative to the target marker according to the second spatial position relationship of the virtual object relative to the plurality of markers and the first spatial position relationship between the plurality of markers. Based on the position relationship of the virtual object relative to the target marker and the target spatial position of the target marker relative to the terminal device, the spatial position of the virtual object relative to the terminal device can be obtained by taking the target marker as a reference.
The terminal device can perform coordinate conversion on the spatial position of the virtual object relative to the terminal device to obtain the display position of the virtual object in the display space of the terminal device, so that the virtual object can be displayed subsequently.
Step S150: and displaying the virtual object according to the display position.
After the display position of the virtual object is obtained, the virtual object can be displayed. Specifically, the terminal device may display the virtual object at the display position based on the data of the virtual object. The data of the virtual object may be model data of the virtual object.
In the embodiment of the application, the virtual object can be observed through the terminal device, the user of the terminal device can move the position, and the virtual object can also change along with the movement of the user position. For example, when the terminal device is far away from the position of the virtual object in the real scene, the virtual object becomes smaller, whereas when the terminal device is close to the position of the virtual object in the real scene, the virtual object becomes larger, and the like. In addition, the recording of the display content can also be carried out, so that the observation and the analysis after the observation are convenient.
Therefore, according to the display method provided by the embodiment of the application, an image including a target marker is acquired, the target marker is at least one of the multiple markers within a visual field range of the terminal device, then a target spatial position of the target marker relative to the terminal device is acquired according to the image, a first spatial position relationship between the stored multiple markers is read, a second spatial position relationship of the virtual object relative to the multiple markers is acquired, a display position of the virtual object is determined based on the first spatial position relationship between the multiple markers, the target spatial position and the second spatial position relationship of the virtual object relative to the multiple markers, and finally the virtual object is displayed according to the display position. The terminal equipment is used for identifying the target marker in the visual field range in the plurality of markers, and the virtual content can be displayed. And the plurality of markers are separately arranged at a plurality of different positions, and the terminal equipment identifies any one marker, so that the positions of other markers which are not in the visual field range can be determined, the markers in a large range can be identified and tracked, and the stability of the terminal equipment for displaying the virtual object is improved.
Referring to fig. 3, another embodiment of the present application provides a display method, which can be applied to a terminal device, and the method can include:
step S210: a first spatial positional relationship between the plurality of markers is acquired.
In the embodiment of the present application, a first spatial position relationship between a plurality of markers may be acquired in advance, so as to determine a display position of a virtual object later.
As an embodiment, acquiring the first spatial positional relationship of the plurality of markers may include:
carrying out image acquisition on a plurality of markers which are separately arranged to obtain an image containing the markers; based on an image including a plurality of markers, a first spatial positional relationship between the plurality of markers is obtained.
It is understood that, when the terminal device is located at some positions, all of the plurality of markers are within the field of view of the image capturing device of the terminal device, and at this time, the terminal device may perform image capturing on the plurality of markers to obtain an image including the plurality of markers, where the image including the plurality of markers should include all of the markers in the plurality of markers.
After obtaining the image including the plurality of markers, each of the plurality of markers is identified, and thus, the identification result such as the spatial position and the identification information of each of the plurality of markers with respect to the terminal device can be obtained.
Then, based on the spatial positional relationship of each of the plurality of markers with respect to the terminal device and the identification result such as the identification information, the spatial positional relationship between the plurality of markers can be established.
The spatial position relationship between each marker and other markers can be sequentially obtained according to the spatial position of each marker relative to the terminal device and the identity information of each marker, so as to obtain the spatial position relationship between the markers. For example, as shown in fig. 4, the plurality of markers include a marker a, a marker B, a marker C, a marker D, and a marker E, the terminal device may acquire an image including the marker a, the marker B, the marker C, the marker D, and the marker E at the same time, and the spatial position of the marker a relative to the terminal device, the spatial position of the marker B relative to the terminal device, the spatial position of the marker C relative to the terminal device, the spatial position of the marker D relative to the terminal device, and the spatial position of the marker E relative to the terminal device may be obtained from the image. According to the spatial positions of the marker A, the marker B, the marker C, the marker D and the marker E relative to the terminal equipment, the spatial position relations between the marker A and the marker B, between the marker A and the marker C, between the marker A and the marker D, between the marker A and the marker E, between the marker B and the marker C, between the marker B and the marker D, between the marker B and the marker E, between the marker C and the marker D, between the marker C and the marker E, and between the marker D and the marker E can be obtained, and the spatial position relation between the marker D and the marker E can be obtained, so that the first spatial position relation between the marker A, the marker B, the marker C, the marker D and the marker E can be obtained.
The spatial positions of all the markers adjacent to each other in position can be sequentially obtained according to the spatial position of each marker in the plurality of markers relative to the terminal device and the identity information of each marker, so that the spatial position relationship among the plurality of markers can be obtained. For example, when the marker includes a marker a, a marker B, a marker C, a marker D, and a marker E, the marker a is adjacent to the marker B, the marker B is adjacent to the marker C, the marker C is adjacent to the marker D, and the marker D is adjacent to the marker E, spatial positional relationships between the marker a and the marker B, between the marker B and the marker C, between the marker C and the marker D, and between the marker D and the marker E can be obtained, respectively, to obtain positional relationships between the plurality of markers. In the above example, if there are other adjacent relationships among the plurality of markers, the spatial positional relationship among the plurality of markers may be obtained by obtaining the spatial positional relationship among the other adjacent markers, and it is sufficient that at least the spatial positional relationship among the four markers in the adjacent relationship is obtained, and there is no island in the obtained spatial positional relationship among the markers in the adjacent relationship, that is, there is no separate spatial positional relationship among the obtained spatial positional relationships among the markers in the adjacent relationship, and the separate spatial positional relationship means that there is no identical marker between the adjacent marker corresponding to the separate spatial positional relationship and the adjacent marker corresponding to any one of the other spatial positional relationships.
As another embodiment, acquiring the first spatial positional relationship of the plurality of markers may also include: acquiring a plurality of images obtained by carrying out a plurality of times of image acquisition on a plurality of markers which are separately arranged, wherein each image at least comprises two markers, and the images acquired by two adjacent images comprise at least one same marker; obtaining a positional relationship between at least two markers included in each image based on a plurality of images acquired by a plurality of times of image acquisition; based on the positional relationship between the markers in each image, a first spatial positional relationship between the plurality of markers is obtained.
It is to be understood that, in addition to the above-described embodiment in which the first spatial positional relationship between the plurality of markers is obtained by acquiring the images of the plurality of markers at one time, it is also possible to divide the image acquisition of the markers a plurality of times and then obtain the first spatial positional relationship between the plurality of markers from the images of the markers acquired a plurality of times.
A plurality of markers separately disposed at different positions may be acquired in multiple times, and each acquired image of a marker should include at least two markers. In addition, at least one same marker should be present in the marker images acquired multiple times, so as to ensure that no island exists between the multiple spatial position relationships obtained according to the marker images acquired multiple times, that is, no independent spatial position relationship exists in the multiple spatial position relationships, and the single spatial position relationship means that the same marker does not exist between the marker in the image corresponding to the single spatial position relationship and the marker in the image corresponding to any one of the other spatial position relationships. As one way, at least one same marker may be included in the images of two adjacent markers, for example, referring to fig. 7, 8, 9 and 10, when the markers include marker a, marker B, marker C, marker D and marker E. As shown in fig. 7, an image containing marker a and marker B may be acquired for the first time. As shown in fig. 8, a second image may be acquired containing marker B as well as marker C. As shown in fig. 9, an image containing marker C as well as marker D may be acquired a third time. As shown in fig. 10, the fourth time may be the acquisition of the image including the marker D and the marker E, so that the first acquired image and the second acquired image include the same marker B, the second acquired image and the third acquired image include the same marker C, and the third acquired image and the fourth acquired image include the same marker D.
After obtaining a plurality of images of the markers acquired for a plurality of times, each image may be identified to obtain a spatial positional relationship between the markers in each image, and then based on the spatial positional relationship between the markers in each image, the same markers included in the images are selected as a reference, thereby obtaining a first spatial positional relationship between the plurality of markers. For example, in the above example, the image including the marker a and the marker B acquired for the first time is identified, so as to obtain the spatial position of the marker a relative to the terminal device and the spatial position of the marker B relative to the terminal device, and then the spatial position relationship between the marker a and the marker B is obtained according to the spatial positions of the marker a and the marker B relative to the terminal device; identifying the image which is acquired for the second time and contains the marker B and the marker C, obtaining the spatial position of the marker B relative to the terminal equipment and the spatial position of the marker C relative to the terminal equipment, and then obtaining the spatial position relation between the marker B and the marker C according to the spatial positions of the marker B and the marker C relative to the terminal equipment; identifying the image which is acquired for the third time and contains the marker C and the marker D, obtaining the spatial position of the marker C relative to the terminal equipment and the spatial position of the marker D relative to the terminal equipment, and then obtaining the spatial position relation between the marker C and the marker D according to the spatial positions of the marker C and the marker D relative to the terminal equipment; identifying the image which is acquired for the fourth time and contains the marker D and the marker E, obtaining the spatial position of the marker D relative to the terminal equipment and the spatial position of the marker E relative to the terminal equipment, and then obtaining the spatial position relation between the marker D and the marker E according to the spatial positions of the marker D and the marker E relative to the terminal equipment; then, according to the spatial position relationship between the marker a and the marker B, the spatial position relationship between the marker B and the marker C, the spatial position relationship between the marker C and the marker D, and the spatial position relationship between the marker D and the marker E, the spatial position relationship among the marker a, the marker B, the marker C, the marker D, and the marker E, that is, the first spatial position relationship among the plurality of markers, can be established, so that according to the spatial position of any one of the marker a, the marker B, the marker C, the marker D, and the marker E relative to the terminal device, the spatial position of each of the other markers relative to the terminal device can be obtained by using the first spatial position relationship.
In the above-described embodiment in which the first spatial positional relationship between the plurality of markers is obtained by acquiring the images of the markers a plurality of times, the first spatial positional relationship between the plurality of markers may be obtained by acquiring the images of the markers twice. Specifically, a first part of markers in a plurality of markers separately arranged at different positions may be subjected to image acquisition to obtain a marker image including the first part of markers, where the first part of markers includes at least two markers; acquiring a spatial position relationship between the first partial markers based on a marker image including the first partial markers; acquiring an image of a second part of the markers in the plurality of markers to obtain a marker image containing the second part of the markers, wherein the second part of the markers comprises at least one marker in the first part of the markers and other markers except the first part of the markers in the plurality of markers; acquiring a spatial position relationship between the second partial markers based on a marker image including the second partial markers; and obtaining a first spatial position relation among the plurality of markers according to the spatial position relation among the first part of markers and the spatial position relation among the second part of markers.
It can be understood that when the plurality of markers are separately disposed at different positions, the distribution range of the plurality of markers may be relatively large, and since the field of view of the image acquisition device of the terminal device is limited, the image acquisition device of the terminal device cannot acquire images including all the markers at one time, the acquisition of the images of the plurality of markers may be divided into two times, and then the spatial position relationship between the plurality of markers is established according to the images of the markers acquired at two times.
The plurality of markers may include a first portion of markers including at least two markers and a second portion of markers including at least one of the first portion of markers and the plurality of markers other than the first portion of markers.
After the image of the first part of the markers is acquired, the image including the first part of the markers can be identified, the spatial position and the identity information of each marker in the first part of the markers relative to the terminal device are obtained, and then the spatial position relationship among the first part of the markers, namely the spatial position relationship among all the markers in the first part of the markers, is obtained according to the spatial position and the identity information of each marker in the first part of the markers relative to the terminal device.
After the image of the second part of the markers is acquired, the image containing the second part of the markers can be identified, the spatial position and the identity information of each marker in the second part of the markers relative to the terminal device are obtained, and then the spatial position relationship among the second part of the markers, namely the spatial position relationship among all the markers in the second part of the markers, is obtained according to the spatial position and the identity information of each marker in the second part of the markers relative to the terminal device.
Since the second partial marker includes at least one marker in the first partial marker, the first spatial positional relationship between the plurality of markers can be obtained from the spatial positional relationship between the first partial markers and the spatial positional relationship between the second partial markers with reference to the same marker in the first partial marker and the second partial marker. And obtaining the spatial positions of all other markers relative to the terminal equipment according to the obtained first spatial position relationship and the spatial position of one marker relative to the terminal equipment.
For example, referring to fig. 5 and fig. 6, when the plurality of markers include the marker a, the marker B, the marker C, the marker D, and the marker E, the images of the first part of the markers of the marker a, the marker B, and the marker C may be collected first, and then the spatial position relationship among the first part of the markers, that is, the spatial position relationship among the marker a, the marker B, and the marker C, may be obtained according to the image of the first part of the markers including the marker a, the marker B, and the marker C; then, images of a second part of the markers including the marker B, the marker D and the marker E are collected, and then the spatial position relationship among the second part of the markers, namely the spatial position relationship among the marker B, the marker D and the marker E is obtained according to the images of the second part of the markers including the marker B, the marker D and the marker E; and finally, according to the spatial position relationship among the first part of markers and the spatial position relationship among the second part of markers, by taking the marker B as a reference, the spatial position relationship among the markers A, the marker B, the marker C, the marker D and the marker E, namely the first spatial position relationship among the multiple markers, can be obtained. The spatial position relationship between the first part of markers is obtained according to the image of the first part of markers, and the spatial position relationship between the second part of markers is obtained according to the image of the second part of markers, and the spatial position relationship between the markers in the image can be obtained by referring to the scheme in the previous embodiment, which is not described in detail herein.
As still another embodiment, the terminal device may perform image acquisition on only one marker at a time, and obtain the spatial position relationship between the markers according to the posture change information of the terminal device. Specifically, the image including the marker Xn is acquired for the nth time, the image including the marker Xn is identified to obtain the position of the marker Xn relative to the terminal device, the image including the marker Xn +1 is acquired for the n +1 th time, the image including the marker Xn +1 is identified to obtain the position of the marker Xn +1 relative to the terminal device, and the 6DOF (degree of freedom) information of the terminal device is obtained in real time through the VIO (Visual-Inertial odometer), wherein the 6DOF information can include information such as the rotation and the orientation of the terminal device, so that the posture change information of the terminal device relative to the image acquisition for the nth time when the image is acquired for the n +1 th time can be obtained, and according to the posture change information of the terminal device relative to the image acquisition for the nth time when the image is acquired for the n +1 th time, the position of the Xn relative to the terminal device and the position of the Xn +1 relative to the terminal device, the spatial position relationship between the marker Xn and the marker Xn +1 can be obtained, and the first spatial position relationship among the multiple markers can be obtained by performing the operation on all the marker images.
In the embodiment of the present application, it should be understood that no matter what kind of implementation manner is used to obtain the first spatial position relationship among the multiple markers, the obtained first spatial position relationship should satisfy that no island exists among all the spatial position relationships, that is, no independent spatial position relationship exists, that is, according to the spatial position of any one of the multiple markers relative to the terminal device, the spatial position of each of the other markers relative to the terminal device can be obtained by using the first spatial position relationship.
After obtaining the first spatial positional relationship among the plurality of markers, the first spatial positional relationship may be stored in the terminal device, so that the terminal device may determine the display position of the virtual object when displaying the virtual object.
Step S220: and acquiring an image containing a target marker, wherein the target marker is at least one marker in a plurality of markers within the visual field range of the terminal equipment, and the plurality of markers are separately arranged at different positions.
When the terminal device needs to display the virtual object, an image of at least one target marker within the field of view of the terminal device can be acquired for subsequent determination of the display position of the virtual object.
It is understood that when only one of the plurality of markers is within the visual field of the terminal device, the marker is used as a target marker, and the terminal device can acquire an image of the marker to obtain an image containing the target marker. When two or more markers in the plurality of markers are within the visual field of the terminal device, one of the two or more markers can be selected as a target marker, and image acquisition is performed to obtain an image containing the target marker.
Step S230: and acquiring the target space position of the target marker relative to the terminal equipment according to the image.
After obtaining the image containing the target marker, a target spatial position of the target marker relative to the terminal device may be obtained from the image containing the target marker.
Step S240: and reading the stored first spatial position relation among the plurality of markers, and acquiring a second spatial position relation of the virtual object relative to the plurality of markers.
In an embodiment of the present application, the second spatial position relationship of the virtual object with respect to the plurality of markers includes at least one of: a spatial positional relationship between the virtual object and a spatial region constituted by a plurality of markers; a spatial positional relationship between the virtual object and a center position of a region constituted by a plurality of markers; a spatial positional relationship of the virtual object and a preset marker of the plurality of markers; the spatial position relationship of the virtual object and the target marker.
It is understood that, from the second spatial positional relationship of the virtual object with respect to the plurality of markers, the spatial positional relationship between the position of the virtual object and the positions of the plurality of markers can be obtained. The virtual object may have a spatial positional relationship with a spatial region constituted by a plurality of markers, that is, the position of the virtual object may be related to a spatial region constituted by a plurality of markers, may be one region in a spatial region constituted by a plurality of markers, may overlap a spatial region constituted by a plurality of markers, may cover a spatial region constituted by a plurality of markers, or the like. For example, when the virtual object overlaps with a spatial region formed by a plurality of markers and the virtual object is a virtual chessboard, the virtual chessboard may be superimposed and displayed on the region formed by the plurality of markers.
The virtual object may have a spatial positional relationship with the center position of the region constituted by the plurality of markers, that is, the virtual object may overlap the center position of the region constituted by the plurality of markers, or the virtual object may be at a specified position corresponding to the center position. For example, when the virtual object is superimposed on the center position of the area constituted by the plurality of markers and the virtual object is a virtual decoration that needs to be displayed at the center position on a dining table, the virtual decoration may be displayed at the center position.
The virtual object may also have a spatial position relationship with a preset marker in the plurality of markers, where the preset marker is a set marker in the plurality of markers, that is, the virtual object may coincide with the preset marker or may be located at a designated position corresponding to the preset marker. The virtual object may have a spatial positional relationship with the target marker, that is, the virtual object may overlap with the target marker or may be located at a predetermined position corresponding to the target marker.
The region formed by the plurality of markers may include the largest region surrounded by the plurality of markers.
Step S250: and determining the display position of the virtual object based on the first spatial position relation, the second spatial position relation and the target spatial position.
When the second spatial position relationship of the virtual object relative to the plurality of markers is the spatial position relationship of the virtual object and the target marker, and the virtual object is located in the target marker, the display position of the virtual object may be directly determined by the target spatial position of the target marker relative to the terminal device. That is, the display position of the virtual object in the display space can be obtained by coordinate conversion of the target marker with respect to the target spatial position of the terminal device. For example, when the virtual object is an ornament on a desktop, the virtual ornament may be directly displayed at the position of the target marker recognized by the terminal device.
When the second spatial position relationship of the virtual object relative to the plurality of markers is a spatial position relationship between the virtual object and a preset marker in the plurality of markers, referring to fig. 11, step S250 may include:
step S251: the center position of the region constituted by the plurality of markers is acquired based on the first spatial positional relationship.
In the embodiment of the present application, a marker near the center of the region formed by the plurality of markers may be used as the preset marker. First, the center position of the region constituted by the plurality of markers may be acquired based on the first spatial positional relationship among the plurality of markers to determine a preset marker close to the center position.
Step S252: and acquiring a preset marker from the plurality of markers based on the central position, wherein the distance from the central position of the preset marker is smaller than the distance from the central position of other markers.
After the center position of the region of the plurality of markers is obtained, the distance of each of the plurality of markers from the center position can be calculated. Then, the marker whose distance from the center position is the shortest is acquired as the preset marker, that is, the distance from the center position of the obtained preset marker should be smaller than the distance from the center position of any one of the markers other than the preset marker among the plurality of markers.
Step S253: and determining the display position of the virtual object according to the target space position, the second space position relation and the space position relation between the preset marker and the target marker.
After obtaining the preset marker closest to the center position, the spatial position relationship between the preset marker and the target marker may be obtained according to the first spatial position relationship between the multiple markers. According to the second spatial position relationship, that is, the spatial position relationship between the virtual object and the preset marker, and the spatial position relationship between the preset marker and the target marker, the spatial position relationship between the virtual object and the target marker can be obtained. And obtaining the spatial position of the virtual object relative to the terminal equipment by taking the target marker as a reference according to the spatial position relation between the virtual object and the target marker and the target spatial position of the target marker relative to the terminal equipment, and then performing coordinate conversion to obtain the display position of the virtual object in the display space.
Step S260: and displaying the virtual object according to the display position.
After the display position of the virtual object is obtained, the virtual object may be displayed. Specifically, the terminal device may display the virtual object at the display position based on the data of the virtual object. The data of the virtual object may be model data of the virtual object.
Therefore, by identifying one of the plurality of markers within the visual field of the terminal device, the virtual object may be displayed in the region formed by the plurality of markers, the virtual object may be displayed at the center position of the region formed by the plurality of markers, the virtual object may be displayed at the position of a preset marker among the plurality of markers, or the virtual object may be displayed directly at the position of the target marker.
In this embodiment of the application, in an application scenario, the virtual object may be a plurality of virtual objects, and the second spatial position relationship of the virtual object with respect to the plurality of markers is that the plurality of virtual objects and the plurality of markers are in one-to-one correspondence and overlap. Therefore, when the display position of the virtual object is obtained, the spatial positions of all the markers relative to the terminal device can be obtained according to the target spatial positions of the target markers relative to the terminal device, and then the spatial positions of each marker relative to the terminal device are subjected to coordinate conversion, so that the display position of each virtual object can be obtained.
For example, when the display method is applied to a dining table of a restaurant, a plurality of markers can be used as placing positions of dishes, each marker can correspond to one type of virtual dishes, the terminal device can identify one of the markers in the visual field, and then all the virtual dishes can be displayed on the corresponding marker according to the spatial position of the marker. When the user orders, a plurality of dishes to be ordered are combined with the plurality of markers, and the terminal equipment only needs to identify one of the markers, so that the plurality of dishes can be displayed on the marker, and convenience is brought to the user for ordering.
Step S270: based on the position updating instruction, the first spatial position relationship among the plurality of markers is acquired again, and the acquired first spatial position relationship is stored.
In the embodiment of the present application, when the first spatial positional relationship between the plurality of markers changes, for example, when the markers are artificially moved, the first spatial positional relationship between the plurality of markers may be updated. Specifically, the location update instruction may be obtained according to an operation of a user. Then, the first spatial positional relationship among the plurality of markers is retrieved, and the retrieved first spatial positional relationship is stored.
In an embodiment of the present application, please refer to fig. 12, the above re-acquiring the first spatial position relationship among the plurality of markers and storing the re-acquired first spatial position relationship may include:
step S271: and acquiring a partial marker image obtained by image acquisition of at least one of the position-changed marker and other markers, wherein the other markers are markers except the position-changed marker in the plurality of markers.
It is understood that when the position of the marker is changed, since there are only a few markers that may be changed, the positional relationship of the marker whose position is changed with respect to the other markers may be determined, and the first spatial positional relationship may be updated.
Therefore, the terminal device can acquire images of at least one of the markers with the changed acquisition positions and other markers under the operation of a user to obtain partial marker images so as to determine the spatial position relationship between the markers with the changed acquisition positions and the markers with the unchanged acquisition positions.
Step S272: and obtaining a first spatial position relation among the plurality of current markers based on the partial marker images and the first spatial position relation among the plurality of previous markers.
After obtaining the partial marker image, the partial marker image may be identified to obtain the spatial position of the marker with the position variation relative to the terminal device and the spatial position of at least one of the other markers. And then obtaining the spatial position relationship between the marker with the changed position and at least one marker in other markers according to the spatial position of the marker with the changed position relative to the terminal equipment and the spatial position of at least one marker in other markers. And then, according to the spatial position relationship and the first spatial position relationship among the plurality of previous markers, because at least one of the other markers is a marker with unchanged position, the first spatial position relationship among the plurality of current markers can be obtained by taking at least one of the other markers as a reference. After obtaining the current first spatial positional relationship, the first spatial positional relationship may be stored in the terminal device.
Of course, in this embodiment of the application, the terminal device may also perform image acquisition on the multiple markers for multiple times to detect whether the positions of the markers change, and when the positions of the markers change, obtain the first spatial position relationship among the multiple markers again.
In addition, the above-mentioned re-acquiring the first spatial position relationship among the plurality of markers may also be performed by referring to the method of step S210 in the embodiment of the present application.
The display method provided by the embodiment of the application acquires spatial position relations among a plurality of markers in advance, provides various implementation modes for acquiring the spatial position relations among the plurality of markers, acquires the display position of the virtual object according to the identification result, the spatial position relations of the virtual object relative to the plurality of markers and the spatial position relations among the plurality of markers after identifying one of the plurality of markers in the visual field of the terminal device, and finally displays the virtual object according to the display position. The display of the virtual object in the area formed by the plurality of markers, or the display of the virtual object in the center position of the area formed by the plurality of markers, or the display of the virtual object in the preset markers in the plurality of markers, or the display of the virtual object in the target marker is realized.
Referring to fig. 13, a block diagram of a display device 400 according to an embodiment of the present disclosure is shown, where the display device 400 is applied to a terminal device. The display device 400 may include: an image acquisition module 410, a first position acquisition module 420, a position relation reading module 430, a second position acquisition module 440, and a display execution module 450. The image acquisition module 410 is configured to acquire an image including a target marker, where the target marker is one of multiple markers located in a field of view of the terminal device, and the multiple markers are separately disposed at different positions; the first position acquisition module 420 is configured to acquire a target spatial position of the target marker relative to the terminal device according to the image; the position relation reading module 430 is configured to read a first stored spatial position relation between the plurality of markers, and obtain a second spatial position relation of the virtual object with respect to the plurality of markers; the second position obtaining module 440 is configured to determine a display position of the virtual object based on the first spatial position relationship, the second spatial position relationship, and the target spatial position; the display execution module 450 is configured to display the virtual object according to the display position.
In an embodiment of the present application, the second spatial position relationship of the virtual object with respect to the plurality of markers includes at least one of: the second spatial positional relationship of the virtual object relative to the plurality of markers comprises at least one of: the virtual object is in a region formed by a plurality of markers; the virtual object is positioned at the center of the area formed by the plurality of markers; the virtual object is in a preset marker of the plurality of markers; the virtual object is at the target marker.
In an embodiment of the present application, when the second spatial position relationship of the virtual object relative to the plurality of markers is that the virtual object overlaps with a preset marker in the plurality of markers, referring to fig. 14, the second position obtaining module 440 may include: a center position acquisition unit 441, a preset marker acquisition unit 442, and a position determination unit 443. The central position obtaining unit 441 may be configured to obtain a central position of an area formed by a plurality of markers based on the first spatial position relationship; the preset marker acquiring unit 442 may be configured to acquire a preset marker from the plurality of markers based on the center position, the distance from the center position of the preset marker being smaller than the distance from the center positions of the other markers; the position determination unit 443 may be configured to determine the display position of the virtual object according to the spatial position relationship between the preset marker and the target spatial position.
In the embodiment of the present application, please refer to fig. 15, the display apparatus 400 may further include a position relationship obtaining module 460. The position relation acquisition module is used for acquiring a first space position relation among the plurality of markers.
As an implementation manner, the location relation obtaining module 460 may specifically be configured to: acquiring images of a plurality of markers which are separately arranged at different positions to obtain an image containing the markers; based on an image including a plurality of markers, a first spatial positional relationship between the plurality of markers is obtained.
As another embodiment, the position relation obtaining module 460 may specifically be configured to: acquiring an image of a first part of markers in a plurality of markers which are separately arranged at different positions to obtain a marker image containing the first part of markers, wherein the first part of markers comprises at least two markers; acquiring a spatial position relationship between the first partial markers based on a marker image including the first partial markers; acquiring an image of a second part of the markers in the plurality of markers to obtain a marker image containing the second part of the markers, wherein the second part of the markers comprises at least one marker in the first part of the markers and other markers except the first part of the markers in the plurality of markers; acquiring a spatial position relationship between the second partial markers based on a marker image including the second partial markers; and obtaining a first spatial position relation among the plurality of markers according to the spatial position relation among the first part of markers and the spatial position relation among the second part of markers.
As another embodiment, the position relationship obtaining module 460 may specifically be configured to: acquiring a plurality of images obtained by acquiring a plurality of images of a plurality of markers which are separately arranged at different positions for a plurality of times, wherein each image at least comprises two markers, and the images acquired by two adjacent images comprise at least one same marker; obtaining a positional relationship between markers in each image based on a plurality of images acquired by a plurality of times of image acquisition; based on the positional relationship between the markers in each image, a first spatial positional relationship between the plurality of markers is obtained.
In the embodiment of the present application, please refer to fig. 15, the display apparatus may further include a relationship re-obtaining module 470. The relationship reacquisition module may be configured to reacquire the first spatial position relationship among the plurality of markers based on the position update instruction, and store the reacquired first spatial position relationship.
Further, the relationship reacquisition module may be specifically configured to: acquiring a partial marker image obtained by image acquisition of at least one of the position-changed marker and other markers, wherein the other markers are markers other than the position-changed marker in the plurality of markers; and obtaining a first spatial position relation among a plurality of current markers based on the partial marker images and the first spatial position relation among a plurality of previous markers, and storing the first spatial position relation.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses and modules may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, the coupling between the modules may be electrical, mechanical or other type of coupling.
In addition, functional modules in the embodiments of the present application may be integrated into one processing module, or each of the modules may exist alone physically, or two or more modules are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode.
Referring to fig. 16, a block diagram of a terminal device according to an embodiment of the present application is shown. The terminal device 100 may be a terminal device capable of running an application, such as a smart phone, a tablet computer, an electronic book, or the like. The terminal device 100 in the present application may include one or more of the following components: a processor 110, a memory 120, an image acquisition apparatus 130, and one or more applications, wherein the one or more applications may be stored in the memory 120 and configured to be executed by the one or more processors 110, the one or more programs configured to perform a method as described in the aforementioned method embodiments.
Processor 110 may include one or more processing cores. The processor 110 connects various parts within the entire terminal device 100 using various interfaces and lines, and performs various functions of the terminal device 100 and processes data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 120 and calling data stored in the memory 120. Alternatively, the processor 110 may be implemented in hardware using at least one of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 110 may integrate one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a modem, and the like. The CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for rendering and drawing display content; the modem is used to handle wireless communications. It is understood that the modem may not be integrated into the processor 110, but may be implemented by a communication chip.
The Memory 120 may include a Random Access Memory (RAM) or a Read-Only Memory (Read-Only Memory). The memory 120 may be used to store instructions, programs, code sets, or instruction sets. The memory 120 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing various method embodiments described below, and the like. The storage data area may also store data created by the terminal 100 in use, and the like.
In the embodiment of the present application, the image capturing device 130 is used to capture an image of a marker. The image capturing device 130 may be an infrared camera or a color camera, and the specific type of the camera is not limited in the embodiment of the present application.
Referring to fig. 17, a block diagram of a computer-readable storage medium according to an embodiment of the present application is shown. The computer readable medium 800 has stored therein a program code that can be called by a processor to execute the method described in the above method embodiments.
The computer-readable storage medium 800 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read only memory), an EPROM, a hard disk, or a ROM. Alternatively, the computer-readable storage medium 800 includes a non-volatile computer-readable storage medium. The computer readable storage medium 800 has storage space for program code 810 to perform any of the method steps of the method described above. The program code can be read from or written to one or more computer program products. The program code 810 may be compressed, for example, in a suitable form.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not necessarily depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.

Claims (8)

1. A display method is applied to a terminal device, and comprises the following steps:
acquiring an image containing a target marker, wherein the target marker is at least one marker in a plurality of markers within the visual field range of the terminal equipment, and the plurality of markers are separately arranged at different positions;
acquiring a target space position of the target marker relative to the terminal equipment according to the image;
reading a stored first spatial position relationship between the plurality of markers, and acquiring a second spatial position relationship of the virtual object relative to the plurality of markers, wherein the second spatial position relationship is a spatial position relationship between the virtual object and a preset marker in the plurality of markers;
acquiring the central position of an area formed by the plurality of markers based on the first spatial position relation;
acquiring a preset marker from the plurality of markers based on the central position, wherein the distance between the preset marker and the central position is smaller than the distances between other markers and the central position;
determining the display position of the virtual object according to the target space position, the second space position relation and the space position relation between the preset marker and the target marker;
and displaying the virtual object according to the display position.
2. The method of claim 1, wherein prior to said acquiring an image containing a target marker, the method further comprises:
acquiring images of a plurality of markers which are separately arranged to obtain an image containing the markers;
obtaining a first spatial positional relationship between the plurality of markers based on the image including the plurality of markers.
3. The method of claim 1, wherein prior to said acquiring an image containing a target marker, the method further comprises:
acquiring a plurality of images obtained by carrying out a plurality of times of image acquisition on a plurality of markers which are separately arranged, wherein each image at least comprises two markers, and the images acquired by two adjacent images comprise at least one same marker;
obtaining a positional relationship between markers in each image based on a plurality of images of the plurality of image acquisitions;
obtaining a first spatial position relationship among the plurality of markers based on the position relationship among the markers in each image.
4. The method according to any one of claims 1-3, further comprising:
based on the position updating instruction, the first spatial position relation among the plurality of markers is obtained again, and the obtained first spatial position relation is stored.
5. The method according to claim 4, wherein said re-acquiring the first spatial positional relationship between the plurality of markers and storing the re-acquired first spatial positional relationship comprises:
acquiring a partial marker image obtained by image acquisition of at least one of a marker with a position change and other markers, wherein the other markers are markers except for the marker with the position change in the plurality of markers;
and obtaining a first spatial position relation among the plurality of current markers based on the partial marker images and the first spatial position relation among the plurality of previous markers, and storing the first spatial position relation.
6. A display device is applied to a terminal device, and the device comprises: the system comprises an image acquisition module, a first position acquisition module, a position relation reading module, a second position acquisition module and a display execution module, wherein the second position acquisition module comprises a central position acquisition unit, a preset marker acquisition unit and a position determination unit,
the image acquisition module is used for acquiring an image containing a target marker, wherein the target marker is at least one marker in a plurality of markers within the visual field range of the terminal equipment, and the markers are separately arranged at different positions;
the first position acquisition module is used for acquiring a target space position of the target marker relative to the terminal equipment according to the image;
the position relation reading module is used for reading a stored first space position relation among the plurality of markers and acquiring a second space position relation of the virtual object relative to the plurality of markers, wherein the second space position relation is a space position relation of the virtual object and a preset marker in the plurality of markers;
the central position acquisition unit is used for acquiring the central position of an area formed by the plurality of markers based on the first spatial position relation;
the preset marker acquisition unit is used for acquiring a preset marker from the plurality of markers based on the central position, and the distance between the preset marker and the central position is smaller than the distance between other markers and the central position; (ii) a
The position determining unit is used for determining the display position of the virtual object according to the target space position, the second space position relation and the space position relation between the preset marker and the target marker;
the display execution module is used for displaying the virtual object according to the display position.
7. A terminal device, comprising:
one or more processors;
a memory;
one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to perform the method of any of claims 1-5.
8. A computer-readable storage medium, having stored thereon program code that can be invoked by a processor to perform the method according to any one of claims 1 to 5.
CN201810910950.6A 2018-07-23 2018-08-10 Display method, display device, terminal equipment and storage medium Active CN110826375B (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CN201810910950.6A CN110826375B (en) 2018-08-10 2018-08-10 Display method, display device, terminal equipment and storage medium
PCT/CN2019/097128 WO2020020102A1 (en) 2018-07-23 2019-07-22 Method for generating virtual content, terminal device, and storage medium
US16/666,429 US11049324B2 (en) 2018-07-23 2019-10-29 Method of displaying virtual content based on markers

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810910950.6A CN110826375B (en) 2018-08-10 2018-08-10 Display method, display device, terminal equipment and storage medium

Publications (2)

Publication Number Publication Date
CN110826375A CN110826375A (en) 2020-02-21
CN110826375B true CN110826375B (en) 2022-08-12

Family

ID=69541350

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810910950.6A Active CN110826375B (en) 2018-07-23 2018-08-10 Display method, display device, terminal equipment and storage medium

Country Status (1)

Country Link
CN (1) CN110826375B (en)

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3238755B2 (en) * 1992-08-21 2001-12-17 富士通株式会社 Hologram creation and stereoscopic display method and stereoscopic display device
US9524580B2 (en) * 2014-01-06 2016-12-20 Oculus Vr, Llc Calibration of virtual reality systems
JP6459421B2 (en) * 2014-11-17 2019-01-30 セイコーエプソン株式会社 Head-mounted display device, method for controlling head-mounted display device, computer program
CN104571532B (en) * 2015-02-04 2018-01-30 网易有道信息技术（北京）有限公司 A kind of method and device for realizing augmented reality or virtual reality
US20170154468A1 (en) * 2015-12-01 2017-06-01 Le Holdings (Beijing) Co., Ltd. Method and electronic apparatus for constructing virtual reality scene model
CN105608746B (en) * 2016-03-16 2019-10-11 成都电锯互动科技有限公司 A method of reality is subjected to Virtual Realization
CN107390875B (en) * 2017-07-28 2020-01-31 腾讯科技（上海）有限公司 Information processing method, device, terminal equipment and computer readable storage medium
CN107977977B (en) * 2017-10-20 2020-08-11 深圳华侨城卡乐技术有限公司 Indoor positioning method and device for VR game and storage medium
CN107797665B (en) * 2017-11-15 2021-02-02 王思颖 Three-dimensional digital sand table deduction method and system based on augmented reality

Also Published As

Publication number Publication date
CN110826375A (en) 2020-02-21

Similar Documents

Publication Publication Date Title
US11263469B2 (en) Electronic device for processing image and method for controlling the same
EP3039655B1 (en) System and method for determining the extent of a plane in an augmented reality environment
CN111078003B (en) Data processing method and device, electronic equipment and storage medium
CN110794955B (en) Positioning tracking method, device, terminal equipment and computer readable storage medium
US11087545B2 (en) Augmented reality method for displaying virtual object and terminal device therefor
US9628706B2 (en) Method for capturing and displaying preview image and electronic device thereof
EP3039647A1 (en) Expanding a digital representation of a physical plane
KR20170053198A (en) Markerless augmented reality data processing device using depth images and augmented reality data processing method using a depth image
JP2020009447A (en) Method and device for augmenting reality
CN111766936A (en) Virtual content control method and device, terminal equipment and storage medium
CN111913564B (en) Virtual content control method, device, system, terminal equipment and storage medium
CN111913674A (en) Virtual content display method, device, system, terminal equipment and storage medium
CN111813214A (en) Virtual content processing method and device, terminal equipment and storage medium
CN110737326A (en) Virtual object display method and device, terminal equipment and storage medium
CN110826375B (en) Display method, display device, terminal equipment and storage medium
CN111913639B (en) Virtual content interaction method, device, system, terminal equipment and storage medium
CN111913560A (en) Virtual content display method, device, system, terminal equipment and storage medium
CN115885314A (en) Image processing method, device, equipment and storage medium
CN110659587A (en) Marker, marker identification method, marker identification device, terminal device and storage medium
KR20210029526A (en) Electronic device for image synthetic and operating thereof
CN111913565B (en) Virtual content control method, device, system, terminal device and storage medium
CN111399630B (en) Virtual content interaction method and device, terminal equipment and storage medium
CN115690363A (en) Virtual object display method and device and head-mounted display device
CN111258520B (en) Display method, display device, terminal equipment and storage medium
US10325406B2 (en) Image synthesis method and image synthesis device for virtual object

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant