WO2021184388A1

WO2021184388A1 - Image display method and apparatus, and portable electronic device

Info

Publication number: WO2021184388A1
Application number: PCT/CN2020/080518
Authority: WO
Inventors: 邵帅
Original assignee: Oppo广东移动通信有限公司
Priority date: 2020-03-20
Filing date: 2020-03-20
Publication date: 2021-09-23
Also published as: WO2021184442A1; CN114830067A

Abstract

Disclosed are an image display method and apparatus, and a portable electronic device. The portable electronic device comprises: an image collection module for acquiring image data of a real scenario; a signal transceiving module for communicating with a reader of a first electronic tag or the first electronic tag; a processing module for acquiring the relative position of the portable electronic device and the first electronic tag on the basis of a signal received by the signal transceiving module, and fusing position information corresponding to the relative position with the image data of the real scenario so as to obtain a first image; and a display module for displaying the first image. The present application solves the technical problem in the relevant art of the user experience being affected since the presentation of a tracking result of an electronic tag tracking system does not take the real environment where a user is located into consideration.

Description

Image display method and device, and portable electronic equipment

Technical field

This application relates to the field of electronic tag tracking, and in particular to an image display method and device, and portable electronic equipment.

Background technique

Electronic tag tracking system is mainly used for item tracking. In the electronic tag tracking system, the electronic tag is placed on the tracked item, and the user uses the reader to locate the location of the electronic tag to complete the item tracking. The final result will be presented to the user in the form of pictures, lights, or sounds. However, in related technologies, the electronic tag tracking system is not intuitive enough to present the tracking process and results, that is, the presentation of the tracking results of the electronic tag tracking system does not consider the real environment the user is in, which affects the user experience.

In view of the above-mentioned problems, no effective solutions have yet been proposed.

Summary of the invention

The embodiments of the present application provide an image display method and device, and a portable electronic device to at least solve the related technology, because the presentation of the tracking result of the electronic tag tracking system does not consider the user's real environment, which affects the user experience. problem.

According to one aspect of the embodiments of the present application, there is provided a portable electronic device, including: an image acquisition module for acquiring image data of a real scene; a signal transceiving module for communicating with a reader or a first electronic tag The electronic tag communicates; the processing module is used to obtain the relative position of the portable electronic device and the first electronic tag based on the signal received by the signal transceiver module; fuse the position information corresponding to the relative position with the image data of the real scene to obtain the first image; The display module is used to display the first image.

Optionally, the signal transceiver module includes: one or more second electronic tags, used to send the first radio frequency signal to the reader; wireless communication circuit, used to receive the reader based on the first radio frequency signal and the second radio frequency signal determined And send the relative position to the processing module, where the second radio frequency signal is the radio frequency signal sent by the first electronic tag to the reader.

Optionally, each of the one or more second electronic tags has a tag identifier, wherein each tag identifier corresponds to an installation position of the second electronic tag on the portable electronic device.

Optionally, the portable electronic device includes: augmented reality (Augmented Reality, AR for short) glasses; the augmented reality glasses include at least: two temple structures, two lens ring structures, and a nose bridge structure for connecting the two lens ring structures There are at least three second electronic tags; among them, at least one second electronic tag of the at least three second electronic tags is arranged in the nose bridge structure of the augmented reality glasses, and the remaining second electronic tags are arranged in the two temple structures.

Optionally, the signal transceiver module includes: a wireless carrier chip for sending a third radio frequency signal to the first electronic tag; receiving the reflected signal returned by the first electronic tag after receiving the third radio frequency signal; the processing module is also used for The relative position is determined based on the third radio frequency signal and the reflected signal.

Optionally, the portable electronic device includes: AR glasses, the AR glasses at least include: two temple structures, two lens ring structures, and a nose bridge structure for connecting the two lens ring structures; the wireless carrier chip includes: a microprocessor And at least three wireless carrier antennas; among them, at least one of the at least three wireless carrier antennas is arranged in the nose bridge structure, and the remaining wireless carrier antennas are arranged in the two mirror leg structures; the microprocessor is arranged in the two mirrors Any one of the leg structures in the temple structure.

Optionally, the portable electronic device includes: a smart mobile terminal; the wireless carrier chip includes: a microprocessor and at least one wireless carrier antenna; at least one wireless carrier antenna is set in the back shell of the smart mobile terminal, and the microprocessor is set in the smart In the main circuit board of the mobile terminal.

Optionally, the relative position includes at least one of the following: a relative angle and a relative distance between the first electronic tag and the portable electronic device.

Optionally, the first electronic tag is an electronic tag set on the target tracking device.

According to another aspect of the embodiments of the present application, an electronic tag tracking system is provided, including: a portable electronic device, a reader, and a first electronic tag; wherein the portable electronic device includes the above portable electronic device; and the reader is used for Collect the label information of the first electronic label and the label information of the second electronic label set on the portable electronic device; determine the difference between the portable electronic device and the first electronic label based on the label information of the first electronic label and the label information of the second electronic label Relative position, and send the relative position to the portable electronic device.

Optionally, the reader is configured to determine the first relative position of the reader and the portable electronic device based on the tag information of the second electronic tag; determine the second relative position of the reader and the first electronic tag based on the tag information of the first electronic tag Position; the relative position of the portable electronic device and the first electronic tag is determined based on the first relative position and the second relative position.

According to another aspect of the embodiments of the present application, there is provided an electronic tag tracking system, including: a portable electronic device and a first electronic tag; wherein, the portable electronic device includes the above portable electronic device; the portable electronic device is used to collect the first electronic device; Tag information of an electronic tag; the relative position of the portable electronic device and the first electronic tag is determined based on the tag information of the first electronic tag.

According to another aspect of the embodiments of the present application, there is provided an image display method, including: acquiring the relative position of the portable electronic device and the first electronic tag; The data is fused to obtain the first image; the first image is displayed.

Optionally, acquiring the relative position of the portable electronic device and the first electronic tag includes: sending a first radio frequency signal to the reader corresponding to the first electronic tag; receiving the relative position determined by the reader based on the first radio frequency signal and the second radio frequency signal. Location, where the second radio frequency signal is the radio frequency signal sent by the first electronic tag to the reader;

Or, send a third radio frequency signal to the first electronic tag; receive a reflected signal returned by the first electronic tag after receiving the third radio frequency signal; determine the relative position based on at least one of the third radio frequency signal and the reflected signal.

Optionally, one or more second electronic tags are provided in the portable electronic device; sending the first radio frequency signal to the reader corresponding to the first electronic tag includes: sending to the reader through one or more second electronic tags One or more first radio frequency signals.

Optionally, before acquiring the relative position of the portable electronic device and the first electronic tag, the above method further includes: presenting a list of electronic tags to the target object; receiving a selection instruction of the target object; and in response to the selection instruction, selecting one from the electronic tag list Or multiple target electronic tags, and the selected one or more target electronic tags are used as the first electronic tags.

According to another aspect of the embodiments of the present application, there is provided an image display device, including: an acquisition module for acquiring the relative position of a portable electronic device and a first electronic tag; and a fusion module for combining position information corresponding to the relative position The first image is obtained by fusion with the image data of the real scene acquired by the portable electronic device; the first display module is used to display the first image.

Optionally, the acquisition module includes: a first transceiver unit, configured to send a first radio frequency signal to a reader corresponding to the first electronic tag; and receive a relative position determined by the reader based on the first radio frequency signal and the second radio frequency signal, wherein , The second radio frequency signal is the radio frequency signal sent by the first electronic tag to the reader; the second transceiver unit sends the third radio frequency signal to the first electronic tag; receiving the reflection returned by the first electronic tag after receiving the third radio frequency signal Signal; the relative position is determined based on at least one of the third radio frequency signal and the reflected signal.

Optionally, one or more second electronic tags are provided in the portable electronic device; the first transceiver unit is configured to send one or more first radio frequency signals to the reader through the one or more second electronic tags.

Optionally, the device further includes: a second display module for displaying a list of electronic tags to the target object; a receiving module for receiving a selection instruction of the target object; and a selection module for selecting from the electronic tag list in response to the selection instruction Select one or more target electronic tags, and use the selected one or more target electronic tags as the first electronic tags.

According to another aspect of the embodiments of the present application, a non-volatile storage medium is provided, the non-volatile storage medium includes a stored program, and the device where the storage medium is located is controlled to execute the above-mentioned image display when the program is running. method.

In the embodiment of the present application, since the position information of the relative position of the portable electronic device and the first electronic tag is fused with the image data of the real scene, the relative position and the real environment can be combined, so that the electronic tag The presentation of the tracking results of the system is more intuitive, which improves the user experience, and thus solves the technical problems that affect the user experience because the presentation of the tracking results of the electronic tag tracking system does not consider the user's real environment in related technologies.

Description of the drawings

The drawings described here are used to provide a further understanding of the application and constitute a part of the application. The exemplary embodiments and descriptions of the application are used to explain the application, and do not constitute an improper limitation of the application. In the attached picture:

Fig. 1 is a schematic structural diagram of an electronic tag tracking system according to an embodiment of the present application;

Fig. 2 is a schematic structural diagram of another electronic tag tracking system according to an embodiment of the present application;

Fig. 3 is a schematic flowchart of an image display method according to an embodiment of the present application;

Fig. 4(a) is a schematic diagram of the principle of tag location tracking based on an AR device according to an embodiment of the present application;

Figure 4(b) is a schematic diagram of another process of tracking the position of tags based on AR devices according to an embodiment of the present application;

Fig. 5 is a schematic diagram of an AR device integrated with an RFID tag according to an embodiment of the present application;

6 is a schematic diagram of the operation flow of an electronic tag tracking system according to an embodiment of the present application;

FIG. 7 is a schematic diagram of an AR device integrating Ultra Wide Band (UWB) tags according to an embodiment of the present application;

Fig. 8 is a schematic diagram of a smart terminal integrated with a UWB system according to an embodiment of the present application;

FIG. 9 is a schematic structural diagram of a portable electronic device according to an embodiment of the present application;

FIG. 10 is a schematic structural diagram of a signal receiving and sending device according to an embodiment of the present application;

FIG. 11 is a schematic structural diagram of a portable electronic device according to an embodiment of the present application;

Fig. 12 is a schematic structural diagram of a portable electronic device according to an embodiment of the present application;

FIG. 13 is a schematic structural diagram of a wireless carrier chip according to an embodiment of the present application;

Fig. 14 is a schematic structural diagram of an image display device according to an embodiment of the present application;

Fig. 15 is a schematic structural diagram of another image display device according to an embodiment of the present application.

Detailed ways

In order to enable those skilled in the art to better understand the solutions of the application, the technical solutions in the embodiments of the application will be clearly and completely described below in conjunction with the drawings in the embodiments of the application. Obviously, the described embodiments are only These are a part of the embodiments of this application, but not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work should fall within the protection scope of this application.

It should be noted that the terms "first" and "second" in the specification and claims of the application and the above-mentioned drawings are used to distinguish similar objects, and not necessarily used to describe a specific sequence or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations of them are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those clearly listed. Those steps or units may include other steps or units that are not clearly listed or are inherent to these processes, methods, products, or equipment.

AR technology refers to combined imaging technology, which is a technology that uses computer image algorithms to combine and interact with the virtual world on the screen and the real scene. The biggest difference between AR and virtual reality (Virtual reality, VR for short) is that VR only presents the user's computer simulated images, while AR combines the real environment with the simulated images, so wearing AR devices will not affect the user's normal life behavior .

The electronic tag tracking system is an application of wireless communication technology independent of AR technology. An electronic tag tracking system usually includes a reader and the electronic tag being tracked. The reader can be independent, that is, the reader has independent calculation and storage functions. The reader can also be connected to the server to obtain instructions to complete data transmission. Electronic tags usually contain tag processing chips and tag antennas. The structure of electronic tags using different wireless transmission protocols will also be different. For example, an electronic tag using passive video identification (Radio Frequency Identification, referred to as RFID) technology does not require a power supply, so the electronic tag only consists of a tag chip and a tag antenna.

Electronic tag tracking system is mainly used for item tracking. In this system, the electronic tag is placed on the tracked item, and the user uses the reader to locate the position of the electronic tag to complete the item tracking. The final result will be presented to the user in the form of pictures, lights, or sounds.

In related technologies, some electronic tags use Bluetooth technology to communicate with readers. At this time, the readers include but are not limited to smart mobile terminals. In this solution, the tracking of items is achieved by means of sound waves, and the location of items is determined by the volume of the electronic tag. At the same time, the location of the items can also be displayed on the mobile phone in the form of a map. For example, some RFID readers can locate electronic devices. The method of the tag is to read the Received Signal Strength Indicator (RSSI) transmitted by the tag, and determine the location of the tag according to the strength of the RSSI signal. In this process, the user needs to hold the RFID reader and change its physical location according to the RSSI feedback instructions.

In addition, the visible light of the human eye is an electromagnetic wave with a frequency of 405-790 THz. The electromagnetic wave spectrum used in wireless communication is 3KHz-300GHz, which is invisible to the naked eye. The frequencies of the wireless technologies commonly used in electronic tags are Bluetooth (Blue Tooth, BT)-2.4 GHz, Wireless Fidelity (Wireless Fidelity, WIFI)-2.4/5 GHz, RFID-100KHz to 2.4 GHz, and UWB-3 to 10 GHz. Although these radio frequency (RF) signals are not visible to the naked eye, they can effectively penetrate non-conductive materials such as paper, textiles, and plastics. If the electronic tag is covered by a paper object, the electronic tag cannot be seen with the naked eye, but its electromagnetic waves can penetrate the paper product and be captured by the reader. In summary, related technologies use sound waves in the positioning process, and the map presents the location of the tracked item or the relative position of the item and the reader. This method cannot most intuitively combine the label position with the real physical environment.

In order to solve the above technical problems, in the embodiments of the present application, the advantages of the AR device that combines the real environment and the analog image are fully utilized to visually present the position of the electronic tag to the user, for example, the position corresponding to the electronic tag The graphic objects are superimposed on the shooting interface for shooting real-world images, so that the user can "see" the label position in the real world, thereby enhancing the user experience. It will be described in detail below in conjunction with specific embodiments.

The embodiment of this application provides two system architectures of Figures 1 and 2 according to whether there is a reader in the electronic tag tracking system. Among them, the portable electronic device in the electronic tag tracking system of Figure 1 can communicate with the reader directly from The reader receives the above-mentioned relative position, or the portable electronic device in the electronic tag tracking system of Figure 1 directly communicates with the first electronic tag. After acquiring the signal of the first electronic tag, the processing module is based on the signal extracted from the signal. The label information determines the above-mentioned relative position. specifically:

Fig. 1 is a schematic structural diagram of an electronic tag tracking system according to an embodiment of the present application. As shown in Fig. 1, the system includes: a portable electronic device 10, a reader 12 and a first electronic tag 14, in which:

The portable electronic device 10 is used to collect real-world images and provide a shooting interface for displaying the real world; the portable electronic device 10 includes: an image acquisition module for acquiring image data of a real scene; a signal transceiver module for communicating with The reader of the first electronic tag or the first electronic tag communicates; the processing module is configured to obtain the relative position of the portable electronic device and the first electronic tag based on the signal received by the signal transceiving module; and correspond the relative position The location information of the first electronic tag is fused with the image data of the real scene to obtain the first image; the display module is used to display the first image; the reader 12 is used to collect the tag information of the first electronic tag and the tag information set on the portable electronic device The tag information of the second electronic tag; the relative position of the portable electronic device and the first electronic tag is determined based on the tag information of the first electronic tag and the tag information of the second electronic tag, and the relative position is sent to the portable electronic device.

Wherein, the reader 12 is also used to determine the first relative position of the reader and the portable electronic device based on the tag information of the second electronic tag; determine the second relative position of the reader and the first electronic tag based on the tag information of the first electronic tag Position; the relative position of the portable electronic device and the first electronic tag is determined based on the first relative position and the second relative position.

Fig. 2 is a schematic structural diagram of another electronic tag tracking system according to an embodiment of the present application. As shown in Figure 3, the system includes:

The portable electronic device 20 and the first electronic tag 22, wherein the portable electronic device 20, in addition to the function described in the embodiment of FIG. 1, is also used to collect the tag information of the first electronic tag, and based on the The tag information of an electronic tag determines the relative position of the portable electronic device and the first electronic tag.

Under the above-mentioned system architecture, the embodiments of the present application provide an embodiment of an image display method. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions And, although the logical sequence is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than here.

FIG. 3 is a schematic flowchart of an image display method according to an embodiment of the present application. As shown in FIG. 3, the method includes the following steps:

In step S300, the portable electronic device directly interacts with the first electronic tag, so that the portable electronic device determines the relative position between the portable electronic device and the first electronic tag according to the interactive data. The specific method will be described in detail below. No longer.

S300', data interaction between the portable electronic device and the reader, and data interaction between the reader and the first electronic tag, so that the reader determines the relative position of the portable electronic device and the first electronic tag according to the data of the two interactions, The specific method will be described in detail below, and will not be repeated here.

It should be noted that step S300 and step S300' are two parallel steps, that is, there is no sequence between them, and step S300 and step S300' are two optional steps, that is, in practical applications, If the portable electronic device has already acquired the relevant data, it may not need to perform step S300 and step S300'.

Step S302: Obtain the relative position of the portable electronic device and the first electronic tag.

Specifically, portable electronic devices include, but are not limited to, AR devices, such as AR glasses or smart mobile terminals with AR functions.

When acquiring the relative position of the AR device and the first electronic tag, the following two solutions can be adopted:

Solution 1, which corresponds to step S300: As shown in Figure 4(a), the AR device first sends a first radio frequency signal to the reader corresponding to the first electronic tag, and then receives the reader based on the first radio frequency signal The relative position determined with a second radio frequency signal, wherein the second radio frequency signal is a radio frequency signal sent by the first electronic tag to the reader.

Among them, the reader can determine the distance between the AR device and the reader and the distance between the reader and the first electronic tag based on the signal strength of the first radio frequency signal and the second radio frequency signal. For example, the reader can maintain a mapping between signal strength and distance. The relationship, that is, the reader can determine the above two distances (the distance between the AR device and the reader, and the distance between the reader and the first electronic tag) based on the mapping relationship. Alternatively, the above-mentioned first radio frequency signal may also be an echo signal (that is, a reflection signal) fed back after the reader sends a trigger signal to the electronic tag in the portable electronic device. At this time, the reader may be based on the transmission time of the trigger signal and the feedback signal. The receiving time determines the distance between the two, for example, based on the time difference between the transmitting time and the above receiving time and the signal transmission speed. The calculation method of the distance between the reader and the first electronic tag is the same as the calculation method of the distance between the reader and the portable electronic device, and will not be repeated here.

In some embodiments of the present application, one or more second electronic tags are provided in the portable electronic device, so that the portable electronic device can send one or more second electronic tags to the reader through the one or more second electronic tags. A first radio frequency signal.

Wherein, each second electronic tag of the one or more second electronic tags has a tag identification, wherein each tag identification corresponds to an installation position of a second electronic tag in the portable electronic device, for example, The tag identifier of the second electronic tag is 1, indicating that the second electronic tag is located on the left leg of the AR glasses, and the tag identifier of the second electronic tag is 2, indicating that the second electronic tag is located on the right leg of the AR glasses, etc. In this way, the position of the second electronic tag can be determined more quickly, and the distance between the second electronic tag and the reader can be quickly determined.

In addition, when multiple second electronic tags are provided in the portable electronic device, a relative position can be determined based on each second electronic tag; then the portable electronic device and the first electronic tag can be determined based on the multiple relative positions determined by the multiple second electronic tags. For the final relative position between an electronic tag, for example, the average value of the above multiple relative positions is calculated, and the average value is used as the final relative position.

It should be noted that the relative positions between different objects involved in the embodiments of the present application include but are not limited to relative coordinate information, and the relative coordinate information may be determined based on the coordinates of different objects in the world coordinate system. It can be understood that the above-mentioned relative coordinate information may also be determined based on the coordinates of different objects in other types of coordinate systems, which is not limited in the embodiment of the present application.

Solution two, that is, the solution corresponding to step S300': as shown in Figure 4(b), the AR device sends a third radio frequency signal to the first electronic tag; after receiving the first electronic tag, the third radio frequency signal is received The returned reflected signal; the relative position is determined based on the third radio frequency signal and the reflected signal.

For example, it is determined based on the transmission time of the third radio frequency signal and the reception time of the reflected signal, or is determined based on the signal strength of the reflected signal.

Step S304, fusing the position information corresponding to the relative position with the image data of the real scene acquired by the portable electronic device to obtain a first image. In some embodiments, the location information includes but not location coordinates, location objects, etc., and its manifestation may be graphical objects for display. There are many manifestations of graphic objects, including but not limited to bar objects, sphere objects, and cylinders. The graphic object can also be an indicator, for example, it can be a number or a letter, and different indicators correspond to different types of tracked devices.

In some embodiments, step S304 may be represented as the following process, but is not limited to this: superimpose the graphic object corresponding to the relative position to the image displayed on the shooting interface of the portable electronic device to obtain the target display interface, where , The shooting interface is used to display real-world images to be shot.

In step S306, the first image is displayed.

The following combines a specific application scenario to elaborate on Solution One:

Solution one is designed based on RFID technology combined with an augmented reality electronic tag tracking system, which includes passive RFID electronic tags, readers, and AR devices. The RFID communication protocol is adopted between the electronic tag and the reader. AR devices and readers use other wireless communication protocols (such as Bluetooth, WIFI, UWB) for data exchange. In this system, AR devices do not exchange data in any form with electronic tags. The electronic tag tracking system based on augmented reality does not need to know the absolute position of the electronic tag, just obtain the relative position of the electronic tag and the AR device. This technical solution integrates an RFID tag on the AR device to obtain the relative position of the AR device and the tracked RFID tag. As shown in Figure 5, three passive RFID tags are integrated on the AR glasses, which are respectively placed on the left and right legs of the device and the beam used to connect the two lens rings.

Passive RFID tags are composed of a tag antenna and a tag chip. This electronic tag relies on the captured RF energy to drive and therefore does not require an internal power supply. The RFID chip contains a memory that can store a unique identification code-Electronic Product Code (EPC), which is used to track items, such as the identification codes A01, A02, and A03 of the three tag antennas in Figure 5. The reader internally stores the matching table between EPC and the article. As shown in Table 1, this matching table is stored in the RFID reader.

Table 1

EPCEPC	物品名称Item Name
A01A01	AR左腿AR left leg
A02A02	AR右腿AR right leg

A03A03	AR横梁AR beam
B0001B0001	钥匙key
B0002B0002	手机cell phone

The operation process of the system is shown in Figure 6. The AR device sends instructions to the RFID reader, and the RFID reader starts to obtain the information of the RFID tag in the environment. The tag information is the tag’s Electronic Product Code (EPC), RSSI , Phase and time scale. The read tags include the RFID tag on the AR and the tracked tag. The reader calculates the relative position of the tracked tag and the AR device by processing the RSSI and phase information of the RFID tag on the AR and the tracked tag, and transmits this information to the AR device. After obtaining this information, the AR device uses image technology to visually present the location of the tracked tag to the user. What the user sees is the superposition of the real environment and the virtual calculated label position, so as to intuitively know the position of the electronic label.

The specific calculation process of the reader positioning algorithm in this solution is as follows: the RFID reader sends an instruction to obtain the information of the RFID tag including EPC, RSSI and phase. Using the RSSI and phase information of the tag antenna, the reader can calculate the distance r between the tag and the reader. Combining RSSI and phase information can also be used to build a model of the environment, or use signal processing methods to improve the accuracy of distance calculations. Using the Angle of Arrival algorithm (Angle of Arrival, AOA), the angle θ of the tag to the reader can be calculated. For multiple RFID tags on the AR device, the average RSSI and phase method can be used to improve the positioning accuracy. Through the above positioning algorithm, the relative distance Ptag between the target RFID tag and the reader can be calculated:

P _tag (r _tag ,θ _tag )

Among them, r _tag is the distance between the tag and the reader, and θ _tag is the relative angle between the tag and the reader.

_{In the same way, the relative distance P AR} (r _AR ,θ _Ar ) between the AR device and the reader can be calculated, and the distance between the target tag and the AR device can be calculated as:

P _d ＝P _AR -P _tag ＝(r _d ,θ _d )

This coordinate P _{d is} the relative distance between the tag and the AR device.

After calculating the relative distance, the reader feeds this information back to the AR device, and the AR device maps this coordinate to the screen through a visual algorithm. This coordinate coincides with the actual environment observed by the user through the AR device, so as to achieve "See" the effect of RFID tags.

The following combines a specific application scenario to elaborate on the second scheme:

Solution two is designed based on UWB technology combined with an augmented reality electronic tag tracking system. The AR device of the system and UWB tag directly carry out data transmission without a reader, and the AR device contains an ultra-wideband (UWB) antenna and a UWB processor. Chips, and microprocessors. As shown in Figure 7, the UWB antenna is placed in front of the AR glasses, and is placed at three positions on the left, center, and right; the UWB processor chip can transmit and receive UWB signals; the microprocessor performs the calculation of the positioning algorithm. In work, the AR device communicates with the UWB tag through the UWB antenna transmitting signal, and receives the reflected signal, and then calculates the relative position of the UWB electronic tag and the AR device through the microprocessor of the AR device. The AR device then superimposes the position of the UWB electronic tag with the actual environment through the above-mentioned visual processing, and presents it to the user. The aforementioned AR device includes but is not limited to AR glasses.

The positioning algorithm in this scheme is similar to the positioning algorithm in scheme 1. It can be considered that the reader is directly integrated into the AR device, so there is no need to calculate the relative position of the reader, the tag and the AR device, and the UWB tag and AR device can be obtained directly The specific calculation process is as follows: the positioning algorithm is divided into two parts, namely distance and angle. The UWB processor chip and microprocessor integrated in the AR device can calculate the distance between the AR device and the electronic tag through Time of Flight (ToF), and then use the AOA method mentioned above to calculate the electronic tag and The angle between the AR device and the relative position P _d (r _d , θ _d ) of the electronic tag and the AR device are obtained, and then the AR device processes this coordinate in a computer vision method and presents it to the user.

It should be noted that the second solution is not limited to the head-mounted AR device of FIG. 7, and can also be applied to smart terminals with rear cameras and UWB processing chips. As shown in Figure 8, the UWB antenna is integrated into the back shell of the smart terminal. The UWB processing chip is integrated into the circuit board in the smart terminal, and the calculation of the positioning algorithm can use the processor of the smart terminal.

In addition to tracking a single electronic tag, the above technical solution can also be applied to tracking multiple electronic tags. Multi-tag tracking mainly depends on the wireless communication standard used by the electronic tag. As mentioned above, RFID and UWB both support one-to-many communication mode, while Bluetooth, WIFI, and Zigbee technology also support one-to-many communication mode.

In the multi-tag tracking scenario, the user can input multiple tags that need to be tracked, or select all electronic tags in reality. In this scenario, all tags in this scenario will be presented to the user in a visual form. Specifically, before acquiring the relative position of the portable electronic device and the first electronic tag, display an electronic tag list to the target object; receive a selection instruction of the target object; in response to the selection instruction, select from the electronic tag list One or more target electronic tags, and the selected one or more target electronic tags are used as the first electronic tags. Among them, the electronic tag list can be displayed in the above-mentioned shooting interface, or the electronic tag list can be displayed in a display interface dedicated to display the electronic tag list.

With the above solution, since the relative position of the portable electronic device and the first electronic tag is added to the shooting interface of the portable device in the form of a graphic object, the relative position and the real environment can be combined, so that the electronic tag system is The presentation of the tracking result is more intuitive, which improves the user experience, and thus solves the technical problem that in the related technology, because the presentation of the tracking result of the electronic tag tracking system does not consider the real environment of the user, the technical problem that affects the user experience is solved.

Fig. 9 is a schematic structural diagram of a portable electronic device according to an embodiment of the present application. As shown in Figure 9, the equipment includes:

The image acquisition module 902 is used to acquire image data of a real scene, for example, to acquire an image of the real world, and provide a shooting interface for displaying the real world;

The signal transceiving module 904 is used to communicate with the reader of the first electronic tag or the first electronic tag;

Specifically, the first electronic tag is an electronic tag set on the target tracking device.

In some embodiments of the present application, there are two ways to form the signal transceiver module:

(1) As shown in Figure 10, the signal transceiving module 904 includes: one or more second electronic tags 1002 for sending a first radio frequency signal to the reader, wherein each of the one or more second electronic tags The second electronic tags each have a tag identification, where each tag identification corresponds to the installation position of a second electronic tag in the portable electronic device; the wireless communication circuit 1004 is used to receive the reader based on the first radio frequency signal and the second radio frequency signal Determine the relative position and send the relative position to the processing module, where the second radio frequency signal is the radio frequency signal sent by the first electronic tag to the reader. Among them, the wireless communication circuit includes but is not limited to UWB communication circuit, for example, includes UWB antenna.

Specifically, the relative position includes at least one of the following: a relative angle and a relative distance between the first electronic tag and the portable electronic device.

In some embodiments of the present application, as shown in FIG. 11, the above-mentioned portable electronic device includes: AR glasses 11; there are at least three second electronic tags; among them, at least one second electronic tag of the at least three second electronic tags Set in the nose bridge structure of the augmented reality glasses, the remaining second electronic tags are set in the two temple structures, for example, in some optional embodiments, when there are three second electronic tags, as shown in FIG. 11, the first The second electronic tag 1104 is arranged in the nose bridge structure of the AR glasses, the second electronic tag 1102 and the second electronic tag 1106 are arranged in the two temple structures of the AR glasses (that is, they are dispersedly arranged in the two temple structures). This scheme is due to the installation position of the three electronic tags. It should be noted that when the length of the electronic tag is greater than the length of the nose bridge structure, a part of the electronic tag is set in the nose bridge structure, and the rest is set on the top of the lens ring structure. For example, the electronic tag is set on the nose bridge structure as the center. The nose bridge structure and the top side of the lens ring structure connected with the nose bridge structure.

Wherein, as shown in FIGS. 12 and 13, the portable electronic device is a smart mobile terminal 12. As shown in FIG. 13, the wireless carrier chip 13 includes: a microprocessor 1302 and at least one wireless carrier antenna 1304. At least one wireless carrier antenna 1304 is provided in the rear case 1202 of the smart mobile terminal. The microprocessor 1302 is arranged in the main circuit board (not shown in the figure) of the smart mobile terminal, and the main circuit board is a circuit board for realizing the communication function and other functions of the smart mobile terminal.

(2) The signal transceiver module 904 includes, but is not limited to: a wireless carrier chip for sending a third radio frequency signal to the first electronic tag; and receiving the reflected signal returned by the first electronic tag after receiving the third radio frequency signal.

The processing module 906 is configured to obtain the relative position of the portable electronic device and the first electronic tag based on the signal received by the signal transceiving module; fuse the position information corresponding to the relative position with the image data of the real scene to obtain the first image. The processing module 906 includes but is not limited to a processor provided in a portable electronic device.

In addition, the processing module 906 is also used to determine the relative position based on the third radio frequency signal and the reflected signal.

The display module 908 is used to display the first image. In some optional embodiments, the display module 908 includes but is not limited to a display screen. The display screen may be a module independent of the image acquisition module 902, or may be a module integrated in the image acquisition module 902.

It should be noted that, for the preferred implementation of the embodiment shown in FIG. 9, reference may be made to the related description of the embodiment shown in FIG. 3, which will not be repeated here.

Fig. 14 is a schematic structural diagram of an image display device according to an embodiment of the present application. The device is used to implement the method shown in FIG. 3. As shown in FIG. 14, the device includes:

The obtaining module 1402 is used to obtain the relative position of the portable electronic device and the first electronic tag;

In some embodiments of the present application, as shown in FIG. 15, the acquisition module 1402 includes: a first transceiver unit 150, configured to send a first radio frequency signal to a reader corresponding to the first electronic tag; and receive the reader The relative position determined based on the first radio frequency signal and the second radio frequency signal, where the second radio frequency signal is the radio frequency signal sent by the first electronic tag to the reader; the second transceiver unit 152, Send a third radio frequency signal to the first electronic tag; receive the reflected signal returned by the first electronic tag after receiving the third radio frequency signal; based on at least one of the third radio frequency signal and the reflected signal One determines the relative position.

Specifically, the portable electronic device is provided with one or more second electronic tags; the first transceiver unit is configured to send the one or more second electronic tags to the reader through the one or more second electronic tags. Multiple first radio frequency signals.

Wherein, each second electronic tag of the one or more second electronic tags has a tag identification, wherein each tag identification corresponds to an installation position of the one second electronic tag in the portable electronic device.

The fusion module 1404 is configured to fuse the position information corresponding to the relative position with the image data of the real scene to obtain a first image;

The first display module 1406 is used to display the first image.

As shown in FIG. 15, the above-mentioned device further includes the following modules: a second display module 156, used to display a list of electronic tags to a target object; a receiving module 158, used to receive a selection instruction of the target object; a selection module 160, used to In response to the selection instruction, one or more target electronic tags are selected from the electronic tag list, and the selected one or more target electronic tags are used as the first electronic tags.

It should be noted that, for the preferred implementation of the embodiment shown in FIG. 14-15, reference may be made to the related description of the embodiment shown in FIG. 3, which will not be repeated here.

The embodiment of the present application also provides a non-volatile storage medium, the non-volatile storage medium includes a stored program, wherein, when the program is running, the device where the non-volatile storage medium is located is controlled to execute the above The described image display method. For example, when the program is running, the following program instructions are executed: obtain the relative position of the portable electronic device and the first electronic tag; superimpose the graphic object corresponding to the relative position on the shooting interface of the portable electronic device to obtain the target display interface, where , The shooting interface is used to display the real-world image to be shot; the target display interface is displayed.

The serial numbers of the foregoing embodiments of the present application are for description only, and do not represent the superiority or inferiority of the embodiments.

In the above-mentioned embodiments of the present application, the description of each embodiment has its own focus. For a part that is not described in detail in an embodiment, reference may be made to related descriptions of other embodiments.

In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units may be a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components may be combined or may be Integrate into another system, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, units or modules, and may be in electrical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solution of this application essentially or the part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium. , Including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage media include: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program code .

The above are only the preferred embodiments of this application. It should be pointed out that for those of ordinary skill in the art, without departing from the principle of this application, several improvements and modifications can be made, and these improvements and modifications are also Should be regarded as the scope of protection of this application.

Claims

A portable electronic device including:

The image acquisition module is used to acquire the image data of the real scene;

The signal transceiver module is used to communicate with the reader of the first electronic tag or the first electronic tag;

The processing module is configured to obtain the relative position of the portable electronic device and the first electronic tag based on the signal received by the signal transceiving module; fuse the position information corresponding to the relative position with the image data of the real scene to obtain the first An image

The display module is used to display the first image.
The portable electronic device according to claim 1, wherein the signal transceiving module comprises:

One or more second electronic tags for sending the first radio frequency signal to the reader;

The wireless communication circuit is configured to receive the relative position determined by the reader based on the first radio frequency signal and the second radio frequency signal, and send the relative position to the processing module, wherein the second radio frequency The signal is a radio frequency signal sent by the first electronic tag to the reader.
The portable electronic device according to claim 2, wherein each second electronic tag in the one or more second electronic tags has a tag identification, wherein each tag identification corresponds to the one second electronic tag The label is at the installation position of the portable electronic device.
The portable electronic device according to claim 2, wherein the portable electronic device comprises: augmented reality glasses; the augmented reality glasses include at least: two temple structures, two mirror ring structures, and are used to connect the two A nose bridge structure with a lens ring structure; there are at least three second electronic tags;

Wherein, at least one second electronic tag of the at least three second electronic tags is arranged in the nose bridge structure, and the remaining second electronic tags are arranged in the two temple structures.
The portable electronic device according to claim 1, wherein:

The signal transceiver module includes: a wireless carrier chip for sending a third radio frequency signal to the first electronic tag; receiving the reflected signal returned by the first electronic tag after receiving the third radio frequency signal;

The processing module is further configured to determine the relative position based on at least one of the third radio frequency signal and the reflected signal.
The portable electronic device according to claim 5, wherein the portable electronic device comprises: augmented reality glasses, and the augmented reality glasses at least comprise: two temple structures, two lens ring structures, for connecting the two A nose bridge structure with a mirror ring structure; the wireless carrier chip includes: a microprocessor and at least three wireless carrier antennas;

Wherein, at least one of the at least three wireless carrier antennas is arranged in the nose bridge structure, and the remaining wireless carrier antennas are arranged in the two temples structures; the microprocessor is arranged in the two temples. In any one of the temple structures.
The portable electronic device according to claim 5, wherein the portable electronic device comprises: a smart mobile terminal; the wireless carrier chip comprises: a microprocessor and at least one wireless carrier antenna; the at least one wireless carrier antenna is set In the rear shell of the smart mobile terminal, the microprocessor is arranged in the main circuit board of the smart mobile terminal.
The portable electronic device according to any one of claims 1 to 7, wherein the relative position includes at least one of the following: a relative angle and a relative distance between the first electronic tag and the portable electronic device.
8. The portable electronic device according to any one of claims 1 to 7, wherein the first electronic tag is an electronic tag set on a target tracking device.
An electronic tag tracking system, including: a portable electronic device, a reader, and a first electronic tag;

Wherein, the portable electronic device includes the portable electronic device according to any one of claims 1 to 9;

The reader is configured to collect the label information of the first electronic label and the label information of the second electronic label set on the portable electronic device; based on the label information of the first electronic label and the second electronic label The tag information of the electronic tag determines the relative position of the portable electronic device and the first electronic tag, and sends the relative position to the portable electronic device.
The system according to claim 10, wherein the reader is configured to determine the first relative position of the reader and the portable electronic device based on the tag information of the second electronic tag; based on the first The tag information of the electronic tag determines the second relative position of the reader and the first electronic tag; the relative position of the portable electronic device and the first electronic tag is determined based on the first relative position and the second relative position Location.
An electronic tag tracking system, including: a portable electronic device and a first electronic tag;

Wherein, the portable electronic device includes the portable electronic device according to any one of claims 1 to 9;

The portable electronic device is used for collecting tag information of the first electronic tag; and determining the relative position of the portable electronic device and the first electronic tag based on the tag information of the first electronic tag.
An image display method, including:

Acquiring the relative position of the portable electronic device and the first electronic tag;

Fusing the position information corresponding to the relative position with the image data of the real scene acquired by the portable electronic device to obtain a first image;

The first image is displayed.
The method according to claim 13, wherein obtaining the relative position of the portable electronic device and the first electronic tag comprises:

Send a first radio frequency signal to the reader corresponding to the first electronic tag; receive the relative position determined by the reader based on the first radio frequency signal and the second radio frequency signal, wherein the second radio frequency signal Is the radio frequency signal sent by the first electronic tag to the reader; or,

Send a third radio frequency signal to the first electronic tag; receive the reflected signal returned by the first electronic tag after receiving the third radio frequency signal; based on at least one of the third radio frequency signal and the reflected signal One determines the relative position.
The method according to claim 14, wherein:

One or more second electronic tags are provided in the portable electronic device;

Sending the first radio frequency signal to the reader corresponding to the first electronic tag includes: sending the one or more first radio frequency signals to the reader through the one or more second electronic tags.
The method according to claim 15, wherein each second electronic tag in the one or more second electronic tags has a tag identifier, wherein each tag identifier corresponds to the one second electronic tag in The installation location of the portable electronic device.
The method according to any one of claims 13 to 16, wherein before acquiring the relative position of the portable electronic device and the first electronic tag, the method further comprises:

Show the electronic label list to the target audience;

Receiving a selection instruction of the target object;

In response to the selection instruction, one or more target electronic tags are selected from the electronic tag list, and the selected one or more target electronic tags are used as the first electronic tags.
An image display device, including:

An acquiring module for acquiring the relative position of the portable electronic device and the first electronic tag;

A fusion module for fusing the position information corresponding to the relative position with the image data of the real scene acquired by the portable electronic device to obtain a first image;

The first display module is used to display the first image.
The apparatus according to claim 18, wherein the acquisition module comprises:

The first transceiver unit is configured to send a first radio frequency signal to the reader corresponding to the first electronic tag; receive the relative position determined by the reader based on the first radio frequency signal and the second radio frequency signal, wherein , The second radio frequency signal is the radio frequency signal sent by the first electronic tag to the reader;

The second transceiver unit sends a third radio frequency signal to the first electronic tag; receives the reflected signal returned by the first electronic tag after receiving the third radio frequency signal; based on the third radio frequency signal and the At least one of the reflected signals determines the relative position.
The apparatus according to claim 19, wherein one or more second electronic tags are provided in the portable electronic device; The reader sends the one or more first radio frequency signals.
The device according to claim 20, wherein each second electronic tag in the one or more second electronic tags has a tag identification, wherein each tag identification corresponds to the one second electronic tag in The installation location of the portable electronic device.
The device according to any one of claims 18 to 21, wherein the device further comprises:

The second display module is used to display the electronic label list to the target object;

A receiving module for receiving a selection instruction of the target object;

The selection module is configured to select one or more target electronic tags from the electronic tag list in response to the selection instruction, and use the selected one or more target electronic tags as the first electronic tags.
A non-volatile storage medium, the non-volatile storage medium includes a stored program, wherein, when the program is running, the device where the storage medium is located is controlled to execute the method described in any one of claims 13 to 17 Image display method.