WO2020192544A1

WO2020192544A1 - Method for selecting interactive object on display medium of device

Info

Publication number: WO2020192544A1
Application number: PCT/CN2020/080161
Authority: WO
Inventors: 牛旭恒; 方俊; 李江亮
Original assignee: 北京外号信息技术有限公司
Priority date: 2019-03-27
Filing date: 2020-03-19
Publication date: 2020-10-01
Also published as: TW202044007A; TWI766258B; CN111752425A; CN111752425B

Abstract

Provided is a method for selecting an interactive object on a display medium of a device, wherein a position on the display medium is set to be a reference position, one or more interactive objects are displayed on the display medium and can be selected, and the display positions of the one or more interactive objects change along with a change in the position or posture of the device. The method comprises: obtaining the display positions, on a display medium, of one or more interactive objects; and automatically selecting, according to the obtained display positions and a reference position, an interactive object from the one or more interactive objects.

Description

Method for selecting interactive objects on display medium of equipment

Technical field

The present invention relates to an interaction method, and more specifically, to a method for selecting an interaction object on a display medium of a device.

Background technique

The statements in this section are only to provide background information related to the present invention to help understand the present invention, and this background information does not necessarily constitute prior art.

With the advancement of technology, various electronic devices have become more and more popular. Various interactive objects that can be operated by the user can be displayed on the display medium of the electronic device, such as icons, signs, text, graphics, virtual characters, virtual objects, and so on. Before operating these interactive objects, the user usually needs to select the interactive objects. In the prior art, the selection of the interactive object is usually achieved by clicking the interactive object on the display medium with a mouse or a user's finger, which is inconvenient in many cases. Especially when the user holds the electronic device with one hand, he must use the other hand to realize the selection of interactive objects on the display medium of the electronic device.

Therefore, there is a need for a method for quickly and conveniently selecting interactive objects on the display medium of the device.

Summary of the invention

One aspect of the present invention relates to a method for selecting interactive objects on a display medium of a device, wherein a position on the display medium is set as a reference position, and one or more interactive objects are displayed on the On the display medium and can be selected, the display position of the one or more interactive objects on the display medium will change as the position or posture of the device changes, and the method includes: obtaining the one or more The display position of the interactive object on the display medium; and automatically select the interactive object from the one or more interactive objects according to the obtained display position and the reference position.

Optionally, wherein the obtaining the display position of the one or more interactive objects on the display medium includes: determining the one or more interactive objects during and/or after the position or posture of the device changes The display position on the display medium.

Optionally, wherein the obtaining the display position of the one or more interactive objects on the display medium includes: repeatedly obtaining the display position of the one or more interactive objects on the display medium.

Optionally, wherein the automatically selecting an interactive object from the one or more interactive objects according to the obtained display position and the reference position includes: according to the last obtained display position of each interactive object and the The reference position determines the distance between the interactive object and the reference position; and the interactive object is automatically selected according to the determined distance.

Optionally, wherein the automatically selecting an interactive object according to the determined distance includes: if the distance between the interactive object and the reference position is less than a threshold, selecting the interactive object.

Optionally, wherein, if the distance between each of the plurality of interactive objects and the reference position is less than the threshold, the interactive object closest to the reference position is selected.

Optionally, wherein said determining the distance between the interactive object and the reference position according to the last obtained display position of each interactive object and the reference position comprises: repeatedly according to the last obtained display of each interactive object The position and the reference position determine the distance between the interactive object and the reference position.

Optionally, wherein the automatically selecting an interactive object according to the determined distance includes: if the distance between the interactive object and the reference position continues to decrease, and the current distance is less than a threshold, then selecting the interactive object.

Optionally, wherein the automatically selecting an interactive object from the one or more interactive objects according to the obtained display position and the reference position includes: determining according to the display position of each interactive object on the display medium The direction of the reference position relative to the interactive object; determine the moving direction of the interactive object on the display medium according to the change of the display position of each interactive object on the display medium; and determine the direction of movement of the interactive object on the display medium according to the reference position relative to the interactive object The direction and the moving direction of the interactive object on the display medium are used to automatically select the interactive object.

Optionally, wherein the automatically selecting the interactive object according to the direction of the reference position relative to the interactive object and the moving direction of the interactive object on the display medium includes: if the moving direction of the interactive object on the display medium is consistent with the reference If the position is closest to the direction of the interactive object, the interactive object is selected.

Optionally, wherein the automatically selecting an interactive object from the one or more interactive objects according to the obtained display position and the reference position includes: determining according to the display position of each interactive object on the display medium The direction of the reference position relative to the interactive object; determine the moving direction of the interactive object on the display medium according to the change of the display position of each interactive object on the display medium; according to the last obtained display position of each interactive object And the reference position to determine the distance between the interactive object and the reference position; and according to the direction of the reference position relative to the interactive object, the moving direction of the interactive object on the display medium, and the distance between the interactive object and the reference position Distance to automatically select interactive objects.

Optionally, wherein the reference position is a fixed position on the display medium.

Optionally, wherein the reference position remains unchanged before the selection of the interactive object.

Optionally, wherein, a position on the display medium is set as the reference position in the following manner: a certain preset or default position on the display medium is set as the reference position; according to the user's instruction To set the reference position; or set the reference position according to the display position of the currently selected interactive object on the display medium.

Another aspect of the present invention relates to a storage medium in which a computer program is stored, and when the computer program is executed by a processor, it can be used to implement the above method.

Another aspect of the present invention relates to an electronic device, which includes a processor and a memory, and a computer program is stored in the memory. When the computer program is executed by the processor, the computer program can be used to implement the above method.

By adopting the solution of the present invention, quick and convenient selection of interactive objects on the display medium of the device can be realized, the efficiency of the user operating the device is improved, and the user experience is improved.

Description of the drawings

The embodiments of the present invention will be further described below with reference to the drawings, in which:

Figure 1 shows an exemplary optical label;

Figure 2 shows an exemplary optical label network;

Figure 3 shows a screenshot of the display medium of the device;

Fig. 4 schematically shows a display medium of a device according to an embodiment;

FIG. 5 shows a method for selecting interactive objects on the display medium shown in FIG. 4 according to an embodiment;

Figure 6 schematically shows a display medium after changing the position or posture of the device according to an embodiment;

FIG. 7 schematically shows the display medium shown in FIG. 6 with a virtual circle;

Figure 8 schematically shows the display medium after changing the position or posture of the device according to an embodiment;

FIG. 9 schematically shows a change diagram of the display position of the interactive object on the display medium when the position or posture of the device changes according to an embodiment;

FIG. 10 schematically shows a change diagram of the display position of an interactive object on a display medium when the position or posture of the device is changed according to an embodiment;

Figure 11 schematically shows a display medium according to an embodiment;

FIG. 12 schematically shows a change diagram of the display position of the interactive object on the display medium when the position or posture of the device changes according to an embodiment;

Figure 13 schematically shows a display medium according to an embodiment;

Fig. 14 shows a method for selecting interactive objects on a display medium according to an embodiment.

detailed description

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below through specific embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described here are only used to explain the present invention, but not to limit the present invention.

In this article, interactive objects may refer to various objects displayed on the display medium of the device that can be operated by the user. The display position of the interactive objects on the display medium will change with the change of the position or posture of the device. The device may be various computing devices or electronic devices that have a display medium or can communicate with the display medium, such as mobile phones, tablet computers, smart glasses, smart helmets, smart watches, and so on. The display medium can be integrated into the device and moved with the device. The display medium can also be a separate component from the device and can communicate with the device (for example, the display medium can receive information about the scene to be presented from the device). In this case, when the device's position or posture changes, the display The scene presented on the medium will change accordingly, but the display medium itself can keep its position or posture unchanged. The display medium may be, for example, an electronic screen, a curtain, etc., but it is not limited thereto, and it may be various media that can be used to display interactive objects. For example, in some smart glasses, an image containing an interactive object is projected onto a prism, lens, mirror, etc. through a projector, and finally projected into the human eye. These prisms, lenses, mirrors, etc. are also considered to be display media. The interactive objects on the display medium of the device may be associated with actual objects in the real-world scene, but may also be virtual objects displayed on the display medium of the device (for example, virtual objects displayed in virtual reality or augmented reality applications) . The expression form of the interactive object may be, for example, icons, signs, texts, graphics, virtual characters, virtual objects, etc. displayed on the display medium.

The following uses the icon of the optical communication device displayed on the device display medium as the interactive object for illustration.

Optical communication devices are also called optical tags, and these two terms can be used interchangeably in this article. Optical tags can transmit information by emitting different lights. They have the advantages of long recognition distance, relaxed requirements for visible light conditions, and strong directivity, and the information transmitted by optical tags can change over time, which can provide large information capacity and flexibility Configuration capabilities. Compared with the traditional two-dimensional code, the optical label has a longer recognition distance and stronger information interaction capabilities, which can provide users with great convenience. Some types of optical labels are described in PCT patent application PCT/CN2017/099642, Chinese patent application CN201711374915.9, Chinese patent application CN201811119052.5, etc., and the entire contents of the above patent applications are incorporated herein by reference.

The optical tag usually includes a controller and at least one light source, and the controller can drive the light source through different driving modes to transmit different information outward. Fig. 1 shows an exemplary optical label 100, which includes three light sources (respectively a first light source 101, a second light source 102, and a third light source 103). The optical label 100 also includes a controller (not shown in FIG. 1) for selecting a corresponding driving mode for each light source according to the information to be transmitted. For example, in different driving modes, the controller can use different driving signals to control the light emitting mode of the light source, so that when the light label 100 is photographed by a device with imaging function, the image of the light source therein can show a different appearance. (For example, different colors, patterns, brightness, etc.). By analyzing the imaging of the light source in the optical label 100, the driving mode of each light source at the moment can be analyzed, so as to analyze the information transmitted by the optical label 100 at the moment.

In order to provide users with corresponding services based on optical tags, each optical tag can be assigned an identification information (ID), which is used to uniquely identify or identify the optical tag by the manufacturer, manager or user of the optical tag . Generally, the controller in the optical tag can drive the light source to transmit the identification information outward, and the user can use the device to perform image capture on the optical tag to obtain the identification information transmitted by the optical tag, so that the corresponding identification information can be accessed based on the identification information. Services, for example, accessing a webpage associated with the identification information of the optical tag, obtaining other information associated with the identification information (for example, location information of the optical tag corresponding to the identification information), and so on. The device can obtain multiple images containing the optical label through continuous image acquisition of the optical label through the camera on it, and identify the optical label by analyzing the imaging of the optical label (or each light source in the optical label) in each image Information passed.

The identification information (ID) of the optical tag and other information (such as location information) can be stored in the server. In reality, a large number of optical labels can be constructed into an optical label network. Fig. 2 shows an exemplary optical label network. The optical label network includes a plurality of optical labels and at least one server, wherein the information related to each optical label can be stored on the server. For example, the identification information (ID) or other information of each optical label can be saved on the server, such as service information related to the optical label, description information or attributes related to the optical label, such as the location information, physical Size information, physical shape information, orientation information, etc. The device can use the identified identification information of the optical label to query the server to obtain other information related to the optical label. The location information of the optical tag may refer to the actual location of the optical tag in the physical world, which may be indicated by geographic coordinate information. The server may be a software program running on a computing device, a computing device, or a cluster composed of multiple computing devices.

When the user recognizes the optical labels using the device, the icons corresponding to the optical labels can be displayed on the display medium of the device. Fig. 3 shows a screenshot of the display medium of the device. The upper part of the display medium of the device is used to display the icon corresponding to the light label (as an interactive object). As shown in FIG. 3, two circular icons are shown in the upper part of the display medium, which correspond to two light labels in the field of view of the camera of the device. When the position or posture of the device changes, the position of the two light tags in the field of view of the device will change, and accordingly, the display positions of the two icons corresponding to the two light tags will also change.

Fig. 4 schematically shows a display medium of a device according to an embodiment. The four

interactive objects

41, 42, 43, 44 are shown as four circular icons on the display medium. A cross-shaped mark is also displayed on the display medium, which is used to indicate a reference position on the display medium. The reference position is the center position of the display medium. However, it is understandable that the reference position can also be other positions on the display medium. . In one embodiment, the reference position is preset or default, and can always remain unchanged. In an embodiment, the reference position may also be specified by the user of the device. For example, the user may specify a certain position on the display medium as the reference position by clicking on it. Once the reference position is determined and not specified again, it will not change as the position or posture of the device changes. The four

interactive objects

41, 42, 43, 44 can be associated with actual objects (such as light tags) in the real world scene, or virtual objects displayed on the device display medium (such as in virtual reality or augmented reality applications). Virtual objects shown in). The display position of the

interactive objects

41, 42, 43, 44 on the display medium will change as the position or posture of the device changes. For example, when the

interactive objects

41, 42, 43, 44 correspond to actual objects in the real world scene, as the position or posture of the device changes, the field of view of the camera on them will change, so that the actual objects appear on the device display medium. The display position of will change, and accordingly, the display position of the

interactive objects

41, 42, 43, 44 corresponding to these actual objects will also change on the display medium. When the

interactive objects

41, 42, 43, 44 are virtual objects displayed in virtual reality or augmented reality applications, the display positions of these virtual objects will change with the position or posture of the device (such as smart glasses, smart helmets, etc.) change. In one embodiment, the mark for indicating the reference position may not be displayed on the display medium to avoid interference with normal display.

Fig. 5 shows a method for selecting an interactive object on the display medium shown in Fig. 4 according to an embodiment, wherein a position in the display medium is determined as a reference position. In the embodiment shown in FIG. 4, the center position of the display medium of the device is preset as the reference position. The method shown in Figure 5 includes:

Step 501: Obtain the display position of each interactive object on the display medium.

The device can determine the display position of each interactive object on the display medium during or after the change of its position or posture. When the interactive object is associated with an actual object (such as a light tag) in the real world scene, the display position of the interactive object on the display medium can be determined according to the position of the actual object detected by the device in the field of view of the device. When the interactive object is a virtual object displayed on the display medium of the device (for example, a virtual object displayed in virtual reality or augmented reality applications), it can be based on various sensors (such as smart glasses, smart helmets, etc.) built in the device ( For example, accelerometers, gyroscopes, etc.) to track the position or posture changes of the device, so as to obtain the display position of the interactive object on the display medium of the device. The interactive object usually occupies an area (such as a circular area) on the display medium. Therefore, in order to conveniently indicate its display position, in one embodiment, its center position can be selected as its display position. Of course, other methods are also feasible of.

When the user wants to select an interactive object on the display medium, he can change the position or posture of the device by translation, rotation, etc. For example, if the user wants to select the interactive object 42 on the display medium, he can translate or rotate the device so that the display position of the interactive object 42 on the display medium can be closer to the reference position (in this embodiment, the center position of the display medium). ). In the process of shifting or rotating the device, the display positions of the

interactive objects

41, 43, 44 will also change accordingly. Fig. 6 schematically shows the display medium after changing the position or posture of the device according to an embodiment.

In an embodiment, the device may also periodically or repeatedly obtain the display position of each interactive object on the display medium. For example, the device can repeatedly obtain each interactive object in the display medium at a certain time interval (for example, every 0.05 seconds, every 0.1 seconds, every 0.2 seconds, every 0.3 seconds, every 0.5 seconds, every 1 second, etc.). In order to obtain a series of display positions of each interactive object at a series of time points.

When the position or posture of the device changes, some interactive objects may leave the display range of the display medium of the device (that is, leave the field of view of the device). In this case, the display of these leaving interactive objects may no longer be determined Position; and some interactive objects may enter the display range of the display medium of the device (that is, enter the field of view of the device), in this case, the display position of these entered interactive objects can be obtained.

Step 502: Select an interactive object according to the obtained display position and the reference position.

After obtaining the display position of the interactive object on the display medium, the device may select the interactive object according to the display position and the reference position. After determining the display position of the interactive object on the display medium each time, the device may select the interactive object according to the determined display position and the reference position. The device may also periodically or repeatedly select the interaction object according to the determined display position and the reference position. The display position may be, for example, the final display position of the interactive object, or may be one, more, a part, or all of a series of display positions of the interactive object determined within a period of time.

In one embodiment, the distance between the interactive object and the reference position can be determined according to the final display position and the reference position of each interactive object. If the distance between the interactive object and the reference position is less than a certain preset Set the threshold, then select the interactive object.

Fig. 7 schematically shows the display medium shown in Fig. 6 with a virtual circle. The virtual circle 701 takes the reference position as the center and the preset threshold as the radius. The virtual circle 701 may be displayed on the display medium to help the user better realize the selection of interactive objects, or it may not be displayed on the display medium.

In the case of taking the center of

interactive objects

41, 42, 43, 44 as its display position, the distance between the center of each interactive object and the reference position can be determined. If the distance between the center of an interactive object and the reference position is less than the preset Threshold, the center will enter the range of circle 701. As shown in Figure 7, the center of the interactive object 42 is within the range of the circle 701, which means that the distance between the interactive object 42 and the reference position is less than the preset threshold, so the device can automatically select the interactive object 42 for subsequent various Kind of operation. After the device selects the interactive object, the selected interactive object can be specially marked (for example, highlighted) to inform the user of the currently selected interactive object.

In the foregoing embodiment in which the interactive object is selected by judging whether the distance between the interactive object and the reference position is less than a certain threshold, it is possible that two or more interactive objects satisfy the condition at the same time. In one embodiment, the interactive object closest to the reference position can be selected. In one embodiment, these interactive objects can be selected at the same time. In one embodiment, the user of the device can be further instructed to select from these interactive objects that meet the conditions, for example, a prompt box can be displayed to the user, which contains a list of multiple interactive objects that meet the conditions, and the user can list them in it. Select an interactive object.

In one embodiment, in addition to considering the distance between the interactive object and the reference position, the device may further consider the changing trend of the distance. Specifically, the device may periodically or repeatedly determine the display position of each interactive object on the display medium, and repeatedly determine the interactive object and the reference position according to the final display position of each interactive object and the reference position the distance between. If the distance between the interactive object and the reference position continues to decrease, and the current distance is less than the threshold, the interactive object is automatically selected. For example, if the user wants to select the interactive object 41 after selecting the interactive object 42 (as shown in the display medium in FIG. 7), the user can translate or rotate the device so that the display position of the interactive object 41 on the display medium can be Closer to the reference position. In the process of shifting or rotating the device, the display positions of the

interactive objects

42, 43, 44 will also change accordingly. Fig. 8 schematically shows the display medium after changing the position or posture of the device according to an embodiment. Compared with the display medium shown in FIG. 7, the display medium of FIG. 8 shows that the interactive object 41 is gradually approaching the reference position, but the distance from the reference position is still greater than the preset threshold, and the interactive object 42 is gradually moving away from the reference position, but with the reference position The distance is still less than the preset threshold. The

interactive objects

43 and 44 have left the display range of the display medium and are no longer displayed. In this case, although the distance between the interactive object 42 and the reference position is less than the preset threshold value, since it gradually moves away from the reference position, the device does not select the interactive object 42. After the user further translates or rotates the device, the distance between the interactive object 41 and the reference position continues to decrease and is less than the preset threshold, and the device can select the interactive object 41 at this time.

If the distance between two or more interactive objects and the reference position at the same time continues to decrease, and the current distance is less than the threshold, you can select the interactive object closest to the reference position, or select these interactive objects at the same time, or The user of the device is further instructed to select from these interactive objects that meet the conditions.

In one embodiment, the device may select the interactive object according to the moving direction of the interactive object. Specifically, the device can determine the direction of the reference position relative to the interactive object according to the display position of each interactive object on the display medium; determine that the interactive object is displayed according to the change in the display position of each interactive object on the display medium The moving direction on the medium; and, selecting the interactive object according to the direction of the reference position relative to the interactive object and the moving direction of the interactive object on the display medium. When determining the direction of the reference position relative to the interactive object, the device can select any one of a series of display positions of the interactive object on the display medium within a period of time as the display position of the interactive object on the display medium, for example, the beginning The display position of the interactive object on the display medium, the display position of the interactive object on the display medium at the end of a period of time, or the display position of the interactive object on the display medium at the middle of a period of time. The device may also select the average value of a part or all of the above-mentioned series of display positions as the display position of the interactive object on the display medium. The device can select any two or more of a series of display positions of the interactive object on the display medium within a period of time to determine the moving direction of the interactive object. If among one or more interactive objects, the moving direction of an interactive object on the display medium is closest to the direction of the reference position relative to the interactive object, then the interactive object can be selected. FIG. 9 schematically shows a display position change diagram of interactive objects on the display medium when the position or posture of the device is changed according to an embodiment, wherein circles drawn by dotted lines are used to represent the

interactive objects

41, 42 The initial display positions of, 43, and 44 use a circle drawn by a solid line to represent the display position of the

interactive objects

41, 42, 43, 44 after the device changes its position or posture. In addition, in FIG. 9, the moving direction of each interactive object is shown with a solid arrow, and the direction of the reference position relative to the initial display position of each interactive object is shown with a dotted arrow. It can be seen from FIG. 9 that the moving direction of the interactive object 42 is closest to the direction of the reference position relative to it, so the interactive object 42 can be selected to perform corresponding operations on it.

In one embodiment, if the proximity (for example, the angle) between the moving direction of an interactive object on the display medium and the reference position relative to the direction of the interactive object meets a predetermined condition (for example, the angle is less than a certain A preset angle value, such as 10°), can select the interactive object.

Selecting the interactive object by the reference position relative to the direction of the interactive object and the moving direction of the interactive object on the display medium can prevent the device from moving significantly in the process of selecting the interactive object, thereby greatly improving the efficiency of selecting the interactive object. The user’s interactive experience.

In one embodiment, the device may select the interactive object according to the moving direction of the interactive object and the distance from the reference position. Specifically, the device can determine the direction of the reference position relative to the interactive object according to the display position of each interactive object on the display medium; determine that the interactive object is displayed according to the change in the display position of each interactive object on the display medium The direction of movement on the medium; the distance between the interactive object and the reference position is determined according to the final display position of each interactive object and the reference position; and, according to the direction of the reference position relative to the interactive object, where the interactive object is Display the moving direction on the medium and the distance between the interactive object and the reference position to select the interactive object. Fig. 10 schematically shows a change diagram of the display position of the interactive object on the display medium when the position or posture of the device changes according to an embodiment. Compared with FIG. 9, the display medium in FIG. 10 also has an interactive object 45, and the direction of the reference position relative to the interactive object 45 is basically the same as the direction of the reference position relative to the interactive object 42. The moving direction of the object 42 is also basically the same. In this case, it may be difficult to determine whether the interactive object 42 or the interactive object 45 should be selected by considering only the closeness of the moving direction of the interactive object to the reference position relative to it (for example, the closeness of the two directions may be the same or different. Not big). In this case, the distance between the interactive object and the reference position can be further considered. Since the interactive object 42 is closer to the reference position, the interactive object 42 can be selected to perform corresponding operations on it.

In one embodiment, after the device selects the interactive object, the reference position can be set according to the display position of the currently selected interactive object on the display medium. For example, after the interactive object is selected, the reference position can be changed to the display position of the currently selected interactive object on the display medium. The selected interactive object may be specially marked (for example, highlighted) to inform the user that it is the currently selected interactive object and the reference position is currently located at the display position of the interactive object. Afterwards, in the process of selecting the next interactive object, the reference position on the display medium may remain unchanged (for example, it does not change with the change of the position and posture of the device) until the next interactive object is selected. FIG. 11 schematically shows the display medium after the display position of the interactive object shown in FIG. 9 is changed, in which a hexagon can be used to identify the currently selected interactive object 42, and the interactive object 42 or six The current display position of the polygon on the display medium is set as the new reference position. Fig. 12 schematically shows a change in the display position of the interactive object on the display medium when the position or posture of the device is changed according to an embodiment, where the user wants to select the interactive object after selecting the interactive object 42 41. To this end, the user can change the position or posture of the device to make the interactive object 41 move to the reference position on the display medium. According to the proximity between the moving direction of each interactive object on the display medium and the direction of the reference position relative to the interactive object on the display medium, it can be determined that the interactive object 41 should be selected. Fig. 13 schematically shows the display medium after the interactive object 41 is selected. A hexagon can be used to identify the currently selected interactive object 41, and the interactive object 41 or the hexagon can be currently displayed The display position on the medium is set to the new reference position. This is very advantageous, because the user usually gazes at the currently selected interactive object, which allows the user to easily know the current reference position and move a small amount to the desired next interactive object (for example, make the device Make a small rotation in the direction where the next interactive object is located) to select the next interactive object, thereby improving the interaction efficiency and improving the user's interactive experience.

In one embodiment, during the movement of the device, the amount of change in the distance between the display position of each interactive object and the reference position can be determined, and the interactive object with the fastest or most reduced distance can be selected. This method is actually similar to the method of selecting the interactive object according to the direction of the reference position relative to the interactive object and the moving direction of the interactive object on the display medium shown in FIGS. 9 and 12. Generally, during the movement of the device, if the moving direction of an interactive object on the display medium is closest to the reference position relative to the direction of the interactive object, the distance between the interactive object and the reference position is reduced. Small the fastest or reduce the most. In some embodiments, a distance reduction threshold or a distance reduction speed threshold may be set to avoid possible misoperation.

In another embodiment, the reference position on the display medium can also be changed according to the user's instruction. For example, the user can click on the display medium to determine a new reference position.

In some cases, the interactive objects presented on the display medium of the device may be objects with three-dimensional spatial positions, such as some virtual objects arranged in virtual reality or augmented reality scenes. The positions of interactive objects may be determined by three-dimensional coordinates, for example. Said. Since the objects in the three-dimensional space can have different depths or distances relative to the device, it may sometimes be difficult to select interactive objects based on the presentation position of these interactive objects on the device display medium (for example, some interactive objects may be spaced more apart in space). Far, but may have a relatively close presentation position on the device display medium). This situation is especially serious when there are more interactive objects in the space. To this end, in one embodiment, a method for selecting interactive objects presented on the display medium of a device is provided, wherein one or more interactive objects are located in the space around the device and can be displayed in all the interactive objects. On the display medium, and wherein the position of the one or more interactive objects relative to the device will change as the position or posture of the device changes. The method is shown in Figure 14, and can include the following steps:

Step 601: Set a selection area in a space around the device, wherein the position of the selection area relative to the device remains unchanged before being reset.

The selection area may be a three-dimensional structure, such as a sphere, a cylinder, a cube, or other regular or irregular three-dimensional structures. Setting the selection area may include setting the position of the selection area relative to the device (for example, the distance and direction relative to the device), setting the shape of the selection area, setting the size of the selection area, and so on. However, it can be understood that the shape, size, or position relative to the device of the selection area may also be preset or default. The position of the selected area relative to the device remains unchanged before being reset. Therefore, the position of the selected area in space will change as the position or posture of the device changes, as if it is fixedly coupled to the device . In an embodiment, for example, the direction of the selection area relative to the device can be set by selecting a certain position on the display medium of the device, and the distance of the selection area relative to the device can be set by dragging a sliding block, inputting a value, etc. Similarly, the position of the selected area relative to the device can be set according to the position of the currently selected interactive object in space.

Step 602: Obtain the position of the one or more interactive objects relative to the device according to the position and posture of the device and the position of the one or more interactive objects.

During the movement or rotation of the device, the position and/or posture changes of the device can be measured or tracked in various known ways. For example, the built-in acceleration sensor, magnetic sensor, direction sensor, gravity sensor, gyroscope can be used. , Cameras, etc., through methods known in the art (for example, inertial navigation, visual odometry, SLAM, VSLAM, SFM, etc.) to measure or track its position change and/or posture change. According to the position and posture of the device and the position of one or more interactive objects, the position of the one or more interactive objects relative to the device can be obtained.

Step 603: Automatically select an interactive object from the one or more interactive objects according to the location of the selection area relative to the device and the location of the one or more interactive objects relative to the device.

In one embodiment, if it is determined that the interaction object is located in the selection area according to the location of the selection area relative to the device and the location of the interaction object relative to the device, the interaction object may be selected.

In an embodiment of the present invention, the present invention can be implemented in the form of a computer program. The computer program can be stored in various storage media (for example, a hard disk, an optical disk, a flash memory, etc.), and when the computer program is executed by a processor, it can be used to implement the method of the present invention.

In another embodiment of the present invention, the present invention can be implemented in the form of an electronic device. The electronic device includes a processor and a memory, and a computer program is stored in the memory. When the computer program is executed by the processor, it can be used to implement the method of the present invention.

References to "various embodiments", "some embodiments", "one embodiment", or "an embodiment" herein refer to the specific features, structures, or properties described in connection with the embodiments included in In at least one embodiment. Therefore, the appearances of the phrases "in various embodiments", "in some embodiments", "in one embodiment", or "in an embodiment" in various places throughout this document do not necessarily refer to the same implementation example. In addition, specific features, structures, or properties can be combined in any suitable manner in one or more embodiments. Therefore, a specific feature, structure, or property shown or described in one embodiment can be combined in whole or in part with the feature, structure, or property of one or more other embodiments without limitation, as long as the combination is not non-limiting. Logical or not working. Expressions similar to "according to A" or "based on A" appearing in this article mean non-exclusive, that is, "according to A" can cover "only according to A" or "according to A and B" unless specifically The statement or the context clearly knows its meaning as "only based on A". The steps described in a certain order in the method flow do not have to be executed in this order. On the contrary, the execution order of some of the steps can be changed, and some steps can be executed concurrently, as long as it does not affect the realization of the solution. In addition, the various elements in the drawings of the present application are for illustrative purposes only, and are not drawn to scale.

Thus, several aspects of at least one embodiment of the present invention have been described, and it can be understood that various changes, modifications and improvements can be easily made by those skilled in the art. Such changes, modifications and improvements are intended to be within the spirit and scope of the present invention. Although the present invention has been described through preferred embodiments, the present invention is not limited to the embodiments described here, and also includes various changes and changes made without departing from the scope of the present invention.

Claims

A method for selecting interactive objects presented on a display medium of a device, wherein a position on the display medium is set as a reference position, and one or more interactive objects are displayed on the display medium and can When selected, the display position of the one or more interactive objects on the display medium will change as the position or posture of the device changes, and the method includes:

Obtaining the display position of the one or more interactive objects on the display medium; and

The interactive object is automatically selected from the one or more interactive objects according to the obtained display position and the reference position.
The method according to claim 1, wherein said obtaining the display position of the one or more interactive objects on the display medium comprises:

The display position of the one or more interactive objects on the display medium is determined during and/or after the position or posture of the device changes.
The method according to claim 1, wherein after the interactive object is selected, the reference position is set according to the display position of the currently selected interactive object on the display medium.
The method according to any one of claims 1 to 3, wherein the automatically selecting an interactive object from the one or more interactive objects according to the obtained display position and the reference position comprises:

Determine the distance between the interactive object and the reference position according to the last obtained display position of each interactive object and the reference position; and

The interactive object is automatically selected according to the determined distance.
The method according to claim 4, wherein said automatically selecting an interactive object according to the determined distance comprises:

If the distance between the interactive object and the reference position is less than the threshold, the interactive object is selected.
The method according to claim 5, wherein if the distance between each of the plurality of interactive objects and the reference position is less than the threshold value, the interactive object closest to the reference position is selected.
The method according to claim 4, wherein said determining the distance between the interactive object and the reference position according to the last obtained display position of each interactive object and the reference position comprises: repeatedly according to each interactive object The last obtained display position and the reference position determine the distance between the interactive object and the reference position.
The method according to claim 7, wherein the automatically selecting an interactive object according to the determined distance comprises:

If the distance between the interactive object and the reference position continues to decrease, and the current distance is less than the threshold, the interactive object is selected.
The method according to any one of claims 1 to 3, wherein the automatically selecting an interactive object from the one or more interactive objects according to the obtained display position and the reference position comprises:

Determine the direction of the reference position relative to the interactive object according to the display position of each interactive object on the display medium;

Determine the moving direction of each interactive object on the display medium according to the change of the display position of each interactive object on the display medium; and

The interactive object is automatically selected according to the direction of the reference position relative to the interactive object and the moving direction of the interactive object on the display medium.
The method according to claim 9, wherein the automatically selecting the interactive object according to the direction of the reference position relative to the interactive object and the moving direction of the interactive object on the display medium comprises:

Determine the degree of proximity of the moving direction of each interactive object on the display medium to the reference position relative to the direction of the interactive object, and select the interactive object with the highest degree of proximity; or

If the proximity between the moving direction of the interactive object on the display medium and the direction of the reference position relative to the interactive object satisfies a predetermined condition, the interactive object is selected.
The method according to any one of claims 1 to 3, wherein the automatically selecting an interactive object from the one or more interactive objects according to the obtained display position and the reference position comprises:

Determine the distance change between the display position of each interactive object and the reference position; and

Select the interactive object whose distance decreases the fastest or most.
The method according to any one of claims 1 to 3, wherein the automatically selecting an interactive object from the one or more interactive objects according to the obtained display position and the reference position comprises:

Determine the direction of the reference position relative to the interactive object according to the display position of each interactive object on the display medium;

Determine the moving direction of each interactive object on the display medium according to the change of the display position of each interactive object on the display medium;

Determine the distance between the interactive object and the reference position according to the last obtained display position of each interactive object and the reference position; and

The interactive object is automatically selected according to the direction of the reference position relative to the interactive object, the moving direction of the interactive object on the display medium, and the distance between the interactive object and the reference position.
The method according to any one of claims 1-3, wherein the reference position does not change with the change of the position or posture of the device.
The method according to claim 1 or 2, wherein a position on the display medium is set as a reference position by:

Set a certain preset or default position on the display medium as the reference position; or

The reference position is set according to the user's instruction.
A method for selecting interactive objects presented on a display medium of a device, wherein one or more interactive objects are located in the space around the device and can be displayed on the display medium, and wherein, the one The position of or multiple interactive objects relative to the device will change as the position or posture of the device changes, and the method includes:

Setting a selection area in a space around the device, wherein the position of the selection area relative to the device remains unchanged before being reset;

Obtaining the position of the one or more interactive objects relative to the device according to the position and posture of the device and the position of the one or more interactive objects; and

The interactive object is automatically selected from the one or more interactive objects according to the location of the selection area relative to the device and the location of the one or more interactive objects relative to the device.
The method according to claim 15, wherein the selected area is a three-dimensional structure.
The method according to claim 15, wherein the one or more interaction objects are determined based on the location of the selection area relative to the device and the location of the one or more interaction objects relative to the device. The automatically selected interactive objects include:

If it is determined that the interaction object is located in the selection area according to the location of the selection area relative to the device and the location of the interaction object relative to the device, then the interaction object is selected.
A storage medium in which a computer program is stored, and when the computer program is executed by a processor, it can be used to implement the method according to any one of claims 1-17.
An electronic device comprising a processor and a memory, and a computer program is stored in the memory, and when the computer program is executed by the processor, it can be used to implement the method according to any one of claims 1-17.
A computer program product, when the computer program product is executed by a processor, it can be used to implement the method of any one of claims 1-17.