TW202044007A - Method for selecting interactive objects on display medium of device - Google Patents

Abstract

The present invention provides a method for selecting an interactive object on a display medium of a device, wherein a position on the display medium is set as a reference position, and one or more interactive objects are displayed on the display medium and can be selected, the display position of the one or more interactive objects on the display medium will change as the position or posture of the device changes, the method includes: obtaining the one or more interactive objects on the display a display position on the medium; and automatically selecting an interactive object from the one or more interactive objects based on the obtained display position and the reference position.

Description

Method for selecting interactive objects on display medium of equipment

The present invention relates to an interaction method, and more specifically, to a method for selecting an interactive object on a display medium of a device.

The statements in this section are merely to provide background information related to the present invention to illustrate the understanding of the present invention, and such background information does not necessarily constitute prior art.

With the advancement of technology, various electronic devices have become more and more popular. Various interactive objects that can be operated by the user can be displayed on the display medium of the electronic device, such as icons, logos, text, graphics, virtual characters, virtual objects, and so on. Before operating these interactive objects, the user usually needs to select the interactive objects. In the prior art, the selection of the interactive object is usually achieved by clicking the interactive object on the display medium with a mouse or a user's finger, which is inconvenient in many cases. Especially when the user holds the electronic device with one hand, he must use the other hand to realize the selection of interactive objects on the display medium of the electronic device.

Therefore, there is a need for a method that can quickly and conveniently select interactive objects on the display medium of the device.

One aspect of the present invention relates to a display medium for selecting The method of interactive objects, wherein a position on the display medium is set as a reference position, one or more interactive objects are displayed on the display medium and can be selected, and the one or more interactive objects are displayed The display position on the medium will change as the position or posture of the device changes, and the method includes: obtaining the display position of the one or more interactive objects on the display medium; and according to the obtained display position and The reference position is used to automatically select an interactive object from the one or more interactive objects.

Optionally, wherein the obtaining the display position of the one or more interactive objects on the display medium includes: determining the one or more interactive objects during and/or after the position or posture of the device changes The display position on the display medium.

Optionally, wherein the obtaining the display position of the one or more interactive objects on the display medium includes: repeatedly obtaining the display position of the one or more interactive objects on the display medium.

Optionally, wherein the automatically selecting an interactive object from the one or more interactive objects according to the obtained display position and the reference position includes: according to the last obtained display position of each interactive object and the The reference position determines the distance between the interactive object and the reference position; and the interactive object is automatically selected according to the determined distance.

Optionally, wherein the automatically selecting the interactive object according to the determined distance includes: if the distance between the interactive object and the reference position is less than a threshold, selecting the interactive object.

Optionally, wherein, if the distance between each of the plurality of interactive objects and the reference position is less than the threshold, the interactive object closest to the reference position is selected.

Optionally, wherein the determining the distance between the interactive object and the reference position according to the last obtained display position of each interactive object and the reference position comprises: repeatedly obtaining the last obtained display position of each interactive object The display position and the reference position determine the distance between the interactive object and the reference position.

Optionally, wherein the automatically selecting the interactive object according to the determined distance includes: if the distance between the interactive object and the reference position continues to decrease, and the current distance is less than a threshold, then selecting the interactive object.

Optionally, wherein the automatically selecting interactive objects from the one or more interactive objects according to the obtained display position and the reference position includes: determining according to the display position of each interactive object on the display medium The direction of the reference position relative to the interactive object; determine the movement direction of the interactive object on the display medium according to the change of the display position of each interactive object on the display medium; and determine the direction of movement of the interactive object on the display medium according to the reference position relative to the interaction The direction of the object and the moving direction of the interactive object on the display medium automatically select the interactive object.

Optionally, wherein the automatically selecting the interactive object according to the direction of the reference position relative to the interactive object and the moving direction of the interactive object on the display medium includes: if the moving direction of the interactive object on the display medium is consistent with the reference If the position is closest to the direction of the interactive object, the interactive object is selected.

Optionally, wherein the automatically selecting interactive objects from the one or more interactive objects according to the obtained display position and the reference position includes: determining according to the display position of each interactive object on the display medium The reference position is relative to the direction of the interactive object; the direction of movement of the interactive object on the display medium is determined according to the change in the display position of each interactive object on the display medium; the final result obtained according to each interactive object The display position and the reference position determine the distance between the interactive object and the reference position; and according to the direction of the reference position relative to the interactive object, the moving direction of the interactive object on the display medium, and the interactive object and the reference position To automatically select interactive objects.

Optionally, wherein the reference position is a fixed position on the display medium.

Optionally, wherein the reference position remains unchanged before the selection of the interactive object.

Optionally, wherein, a position on the display medium is Set as the reference position: set a preset or default position on the display medium as the reference position; set the reference position according to the user's instructions; or set the reference position on the display medium according to the currently selected interactive object To set the reference position.

Another aspect of the present invention relates to a storage medium in which a computer program is stored, and when the computer program is executed by a processor, it can be used to implement the above-mentioned method.

Another aspect of the present invention relates to an electronic device, which includes a processor and a memory, and a computer program is stored in the memory. When the computer program is executed by the processor, the computer program can be used to implement the above method.

By adopting the solution of the present invention, quick and convenient selection of interactive objects on the display medium of the device can be realized, the efficiency of the user's operation of the device is improved, and the user experience is improved.

In order to have a more detailed understanding of the purpose, efficacy, features, and structure of the present invention, preferred embodiments are described below in conjunction with the drawings.

100: light label

101: The first light source

102: second light source

103: third light source

ID: Identification information

41: Interactive objects

42: Interactive objects

43: interactive objects

44: Interactive Objects

45: Interactive objects

701: Virtual Circle

501: step 501

502: step 502

601: step 601

602: step 602

603: Step 603

Figure 1 shows an exemplary optical label;

Figure 2 shows an exemplary optical label network;

Figure 3 shows a screenshot of the display medium of the device;

Fig. 4 schematically shows a display medium of a device according to an embodiment;

FIG. 5 shows a method for selecting interactive objects on the display medium shown in FIG. 4 according to an embodiment;

Figure 6 schematically shows a display medium after changing the position or posture of the device according to an embodiment;

FIG. 7 schematically shows the display medium shown in FIG. 6 with a virtual circle;

Figure 8 schematically shows a display after changing the position or posture of the device according to an embodiment Show media

FIG. 9 schematically shows a change diagram of the display position of an interactive object on a display medium when the position or posture of the device changes according to an embodiment;

FIG. 10 schematically shows a change diagram of the display position of an interactive object on a display medium when the position or posture of the device changes according to an embodiment;

Figure 11 schematically shows a display medium according to an embodiment;

FIG. 12 schematically shows a change diagram of the display position of the interactive object on the display medium when the position or posture of the device changes according to an embodiment;

Figure 13 schematically shows a display medium according to an embodiment;

Fig. 14 shows a method for selecting interactive objects on a display medium according to an embodiment.

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below through specific embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described here are only used to explain the present invention, but not to limit the present invention.

In this article, interactive objects may refer to various objects displayed on the display medium of the device that can be operated by the user, and the display position of the interactive objects on the display medium will change as the position or posture of the device changes. The device may be various computing devices or electronic devices that have a display medium or can communicate with the display medium, such as mobile phones, tablet computers, smart glasses, smart helmets, smart watches, and so on. The display medium can be integrated into the device and moved with the device. The display medium can also be a separate component from the device and can communicate with the device (for example, the display medium can receive information about the scene to be presented from the device). In this case, when the device's position or posture changes, the display The scene presented on the medium will change accordingly, but the display medium itself can keep its position or posture unchanged. The display medium can be, for example, an electronic screen, a screen, etc., but it is not limited thereto, and it can be various media that can be used to display interactive objects. For example, in some smart glasses, images containing interactive objects are projected onto prisms, lenses, Mirror, etc., and finally projected into the eyes of people. These prisms, lenses, mirrors, etc. are also considered to be display media. The interactive objects on the display medium of the device can be associated with actual objects in the real world scene, but can also be virtual objects displayed on the display medium of the device (for example, virtual objects displayed in virtual reality or augmented reality applications). ). The expression form of the interactive object may be, for example, an icon, a logo, a text, a figure, a virtual character, a virtual object, etc. displayed on a display medium.

Hereinafter, an illustration of the optical communication device displayed on the device display medium is used as an interactive object for illustration.

Optical communication devices are also called optical tags, and these two terms can be used interchangeably in this article. Optical tags can transmit information by emitting different lights. It has the advantages of long recognition distance, loose visible light conditions, and strong directivity. The information transmitted by optical tags can change over time, which can provide large information capacity and flexibility. Configuration capabilities. Compared with the traditional two-dimensional code, the optical label has a longer recognition distance and stronger information interaction ability, which can provide users with great convenience. Some types of optical labels are described in PCT patent application PCT/CN2017/099642, Chinese patent application CN201711374915.9, Chinese patent application CN201811119052.5, etc., and the entire contents of the above patent applications are incorporated herein by reference.

The optical tag usually includes a controller and at least one light source, and the controller can drive the light source through different driving modes to transmit different information outward. Fig. 1 shows an exemplary optical label 100, which includes three light sources (respectively a first light source 101, a second light source 102, and a third light source 103). The optical label 100 also includes a controller (not shown in FIG. 1) for selecting a corresponding driving mode for each light source according to the information to be transmitted. For example, in different driving modes, the controller can use different driving signals to control the light emitting mode of the light source, so that when the light label 100 is photographed by a device with imaging function, the image of the light source therein can show different Appearance (for example, different colors, patterns, brightness, etc.). By analyzing the imaging of the light source in the optical label 100, the driving mode of each light source at the moment can be analyzed, Thus, the information transmitted by the optical label 100 at the moment can be analyzed.

In order to provide users with corresponding services based on optical tags, each optical tag can be assigned an identification information (ID), which is used to uniquely identify or identify the manufacturer, manager, or user of the optical tag Light label. Generally, the light source can be driven by the controller in the optical tag to transmit the identification information outward, and the user can use the device to collect the image of the optical tag to obtain the identification information transmitted by the optical tag, which can be based on the identification information. Information to access the corresponding service, for example, visit a webpage associated with the identification information of the light tag, obtain other information associated with the identification information (for example, the location information of the light tag corresponding to the identification information), and so on. The device can obtain multiple images containing the optical label through continuous image acquisition of the optical label through the camera on it, and analyze the imaging of the optical label (or each light source in the optical label) in each image. Identify the information transmitted by the optical label.

The identification information (ID) of the optical tag and other information (such as location information) can be stored in the server. In reality, a large number of optical labels can be constructed into an optical label network. Fig. 2 shows an exemplary optical label network. The optical label network includes a plurality of optical labels and at least one server, wherein information related to each optical label can be stored on the server. For example, the identification information (ID) or other information of each light tag can be saved on the server, such as service information related to the light tag, description information or attributes related to the light tag, such as the location of the light tag Information, physical size information, physical shape information, orientation information, etc. The device can use the identification information of the identified optical label to obtain other information related to the optical label from the server. The position information of the optical tag may refer to the actual position of the optical tag in the physical world, which may be indicated by geographic coordinate information. The server can be a software program running on a computing device, a computing device, or a cluster composed of multiple computing devices.

When the user recognizes the optical labels using the device, the icons corresponding to the optical labels can be displayed on the display medium of the device. Figure 3 shows a screenshot of the display medium of the device. The upper part of the display medium of the device is used to display the icon corresponding to the optical label (as For interactive objects). As shown in FIG. 3, two circular illustrations are shown in the upper part of the display medium, which correspond to the two light tags in the field of view of the camera of the device. When the position or posture of the device changes, the position of the two optical tags in the field of view of the device will change, and accordingly, the display positions of the two icons corresponding to the two optical tags will also change.

Fig. 4 schematically shows a display medium of a device according to an embodiment. Four interactive objects 41, 42, 43, 44 are shown as four circular icons on the display medium. A cross-shaped mark is also displayed on the display medium, which is used to indicate a reference position on the display medium. The reference position is the center position of the display medium. However, it is understandable that the reference position can also be other positions on the display medium. position. In one embodiment, the reference position is preset or default, and can always remain unchanged. In one embodiment, the reference position can also be specified by the user of the device. For example, the user can specify a position as the reference position by clicking on a certain position on the display medium. Once the reference position is determined and is not specified again, it will not change as the position or posture of the device changes. The four interactive objects 41, 42, 43, 44 can be associated with actual objects (such as light tags) in the real world scene, or virtual objects displayed on the device display medium (such as in virtual reality or augmented reality). Virtual objects displayed in the application). The display position of the interactive objects 41, 42, 43, 44 on the display medium will change as the position or posture of the device changes. For example, when the interactive objects 41, 42, 43, 44 correspond to actual objects in the real world scene, as the position or posture of the device changes, the field of view of the camera on them will change, so that the actual object appears on the device display medium. The display position of will change, and accordingly, the display positions of the interactive objects 41, 42, 43, 44 corresponding to these actual objects will also change on the display medium. When interactive objects 41, 42, 43, 44 are virtual objects displayed in virtual reality or augmented reality applications, the display position of these virtual objects will change with the position or posture of the device (such as smart glasses, smart helmet, etc.) And change. In one embodiment, the mark for indicating the reference position may not be displayed on the display medium to avoid interference with normal display.

Fig. 5 shows a method for selecting an interactive object on the display medium shown in Fig. 4 according to an embodiment, wherein a position in the display medium is determined as a reference position. In the embodiment shown in FIG. 4, the center position of the display medium of the device is preset as the reference position. The method shown in Figure 5 includes:

Step 501: Obtain the display position of each interactive object on the display medium.

The device can determine the display position of each interactive object on the display medium during or after the change of its position or posture. When the interactive object is associated with an actual object (such as a light tag) in the real world scene, the display position of the interactive object on the display medium can be determined according to the position of the actual object detected by the device in the field of view of the device. When the interactive object is a virtual object displayed on the display medium of the device (for example, a virtual object displayed in a virtual reality or augmented reality application), it can be based on various sensors built into the device (for example, smart glasses, smart helmet, etc.) Device (for example, accelerometer, gyroscope, etc.) to track the position or posture change of the device, so as to obtain the display position of the interactive object on the display medium of the device. The interactive object usually occupies an area (such as a circular area) on the display medium. Therefore, in order to conveniently indicate its display position, in one embodiment, its center position can be selected as its display position. Of course, other methods are also feasible of.

When the user wants to select an interactive object on the display medium, he can change the position or posture of the device by translation, rotation, etc. For example, if the user wants to select the interactive object 42 on the display medium, he can translate or rotate the device so that the display position of the interactive object 42 on the display medium can be closer to the reference position (in this embodiment, the center of the display medium) position). In the process of shifting or rotating the device, the display positions of the interactive objects 41, 43, 44 will also change accordingly. Fig. 6 schematically shows the display medium after changing the position or posture of the device according to an embodiment.

In an embodiment, the device may also periodically or repeatedly obtain the display position of each interactive object on the display medium. For example, the device can be at a certain time interval (example For example, every 0.05 seconds, every 0.1 seconds, every 0.2 seconds, every 0.3 seconds, every 0.5 seconds, every 1 second, etc.) to repeatedly obtain the display position of each interactive object on the display medium, thereby obtaining each interaction A series of display positions of an object at a series of time points.

When the position or posture of the device is changed, some interactive objects may leave the display range of the display medium of the device (that is, leave the field of view of the device). In this case, the display of these leaving interactive objects may no longer be determined Position; and some interactive objects may enter the display range of the display medium of the device (that is, enter the field of view of the device), in this case, the display position of these entered interactive objects can be obtained.

Step 502: Select an interactive object according to the obtained display position and the reference position.

After obtaining the display position of the interactive object on the display medium, the device may select the interactive object according to the display position and the reference position. After determining the display position of the interactive object on the display medium each time, the device may select the interactive object according to the determined display position and the reference position. The device may also periodically or repeatedly select interactive objects according to the determined display position and the reference position. The display position may be, for example, the final display position of the interactive object, or it may be one, more, part or all of a series of display positions of the interactive object determined within a period of time.

In one embodiment, the distance between the interactive object and the reference position may be determined according to the final display position and the reference position of each interactive object. If the distance between the interactive object and the reference position is less than a certain If the preset threshold is set, the interactive object is selected.

Fig. 7 schematically shows the display medium shown in Fig. 6 with a virtual circle. The virtual circle 701 takes the reference position as the center and the preset threshold as the radius. The virtual circle 701 may be displayed on the display medium to illustrate that the user can better realize the selection of interactive objects, or it may not be displayed on the display medium.

In the situation where the center of interactive objects 41, 42, 43, 44 is used as its display position In this case, the distance between the center of each interactive object and the reference position can be determined. If the distance between the center of an interactive object and the reference position is less than a preset threshold, its center will enter the range of circle 701. As shown in Figure 7, the center of the interactive object 42 is within the range of the circle 701, which means that the distance between the interactive object 42 and the reference position is less than the preset threshold. Therefore, the device can automatically select the interactive object 42 for subsequent various operations. operating. After the device selects the interactive object, the selected interactive object can be specially marked (for example, highlighted) to inform the user of the currently selected interactive object.

In the above embodiment in which the interactive object is selected by judging whether the distance between the interactive object and the reference position is less than a certain threshold, it is possible that two or more interactive objects satisfy the condition at the same time. In one embodiment, the interactive object closest to the reference position can be selected. In one embodiment, these interactive objects can be selected at the same time. In one embodiment, the user of the device can be further instructed to select from these interactive objects that meet the conditions, for example, a prompt box can be displayed to the user, which contains a list of multiple interactive objects that meet the conditions, and the user Select an interactive object among them.

In one embodiment, in addition to considering the distance between the interactive object and the reference position, the device may further consider the changing trend of the distance. Specifically, the device may periodically or repeatedly determine the display position of each interactive object on the display medium, and repeatedly determine the interactive object and the reference based on the final display position of each interactive object and the reference position The distance between locations. If the distance between the interactive object and the reference position continues to decrease, and the current distance is less than the threshold, the interactive object is automatically selected. For example, if the user wishes to select the interactive object 41 after selecting the interactive object 42 (as shown in the display medium in FIG. 7), the user can translate or rotate the device so that the interactive object 41 is positioned on the display medium. The display position can be closer to the reference position. In the process of shifting or rotating the device, the display positions of the interactive objects 42, 43, 44 will also change accordingly. Fig. 8 schematically shows the display medium after changing the position or posture of the device according to an embodiment. As shown in Figure 7 Compared with the display medium, the display medium of FIG. 8 shows that the interactive object 41 is gradually approaching the reference position, but the distance from the reference position is still greater than the preset threshold, and the interactive object 42 is gradually moving away from the reference position, but the distance from the reference position is still less than the preset Threshold. The interactive objects 43 and 44 have left the display range of the display medium and are no longer displayed. In this case, although the distance between the interactive object 42 and the reference position is less than the preset threshold, since it gradually moves away from the reference position, the device does not select the interactive object 42. After the user further translates or rotates the device, the distance between the interactive object 41 and the reference position continues to decrease and is less than the preset threshold, and the device can select the interactive object 41 at this time.

If the distance between two or more interactive objects and the reference position continues to decrease, and the current distance is less than the threshold, you can select the interactive object closest to the reference position, or select these interactive objects at the same time, or Further instruct the user of the device to choose from these interactive objects that meet the conditions.

In one embodiment, the device may select the interactive object according to the moving direction of the interactive object. Specifically, the device can determine the direction of the reference position relative to the interactive object according to the display position of each interactive object on the display medium; determine the interactive object according to the change in the display position of each interactive object on the display medium The moving direction on the display medium; and, selecting the interactive object according to the direction of the reference position relative to the interactive object and the moving direction of the interactive object on the display medium. When determining the direction of the reference position relative to the interactive object, the device can select any one of a series of display positions of the interactive object on the display medium within a period of time as the display position of the interactive object on the display medium, for example, the beginning of a period of time The display position of the interactive object on the display medium, the display position of the interactive object on the display medium at the end of a period of time, or the display position of the interactive object on the display medium at the middle of a period of time. The device may also select an average value of a part or all of the above-mentioned series of display positions as the display position of the interactive object on the display medium. The device can select any two or more of a series of display positions of the interactive object on the display medium within a period of time to determine the moving direction of the interactive object. If in one or more interactive objects, an interactive object is displayed If the moving direction on the medium is closest to the direction of the reference position relative to the interactive object, the interactive object can be selected. FIG. 9 schematically shows a view of the display position change of the interactive object on the display medium when the position or posture of the device is changed according to an embodiment, in which a circle drawn by a dotted line is used to represent the interactive object 41, 42 The initial display position of, 43, 44, using a circle drawn by a solid line to represent the display position of the interactive object 41, 42, 43, 44 after the device changes its position or posture. In addition, in FIG. 9, the moving direction of each interactive object is shown with a solid arrow, and the direction of the reference position relative to the initial display position of each interactive object is shown with a dotted arrow. It can be seen from FIG. 9 that the moving direction of the interactive object 42 is closest to the direction of the reference position relative to it, so the interactive object 42 can be selected to perform corresponding operations on it.

In one embodiment, if the proximity (for example, the included angle) between the moving direction of an interactive object on the display medium and the reference position relative to the direction of the interactive object satisfies a predetermined condition (for example, the included angle is less than a certain A preset angle value, such as 10°), can select the interactive object.

The selection of interactive objects by the reference position relative to the direction of the interactive objects and the direction of movement of the interactive objects on the display medium can avoid large movements of the device in the process of selecting interactive objects, thereby greatly improving the efficiency of selecting interactive objects. The user’s interactive experience.

In one embodiment, the device may select the interactive object according to the moving direction of the interactive object and the distance from the reference position. Specifically, the device can determine the direction of the reference position relative to the interactive object according to the display position of each interactive object on the display medium; determine the interactive object according to the change in the display position of each interactive object on the display medium The direction of movement on the display medium; the distance between the interactive object and the reference position is determined according to the final display position of each interactive object and the reference position; and, according to the direction of the reference position relative to the interactive object, The moving direction of the interactive object on the display medium and the distance between the interactive object and the reference position are used to select the interactive object. Figure 10 schematically shows According to one embodiment, when the position or posture of the device changes, the display position change diagram of the interactive object on the display medium is displayed. Compared with FIG. 9, the display medium in FIG. 10 also has an interactive object 45, and the direction of the reference position relative to the interactive object 45 is basically the same as the direction of the reference position relative to the interactive object 42. The moving direction of the interactive object 45 is similar to that of the interactive object. The moving direction of the object 42 is also basically the same. In this case, it may be difficult to determine whether the interactive object 42 or the interactive object 45 should be selected by considering only the proximity of the moving direction of the interactive object to the reference position relative to the direction (for example, the proximity of the two may be the same or different Not big). In this case, the distance between the interactive object and the reference position can be further considered. Since the interactive object 42 is closer to the reference position, the interactive object 42 can be selected to perform corresponding operations on it.

In one embodiment, after the device selects the interactive object, the reference position can be set according to the display position of the currently selected interactive object on the display medium. For example, after the interactive object is selected, the reference position can be changed to the display position of the currently selected interactive object on the display medium. The selected interactive object may be specially marked (for example, highlighted) to inform the user that it is the currently selected interactive object and the reference position is currently located at the display position of the interactive object. After that, in the process of selecting the next interactive object, the reference position on the display medium may remain unchanged (for example, not changing with the change of the position and posture of the device) until the next interactive object is selected. FIG. 11 schematically shows the display medium after the display position of the interactive object shown in FIG. 9 is changed, in which a hexagon can be used to identify the currently selected interactive object 42, and the interactive object 42 or The current display position of the hexagon on the display medium is set as the new reference position. FIG. 12 schematically shows a change in the display position of the interactive object on the display medium when the position or posture of the device is changed according to an embodiment, wherein the user wants to select the interactive object after selecting the interactive object 42 For the object 41, the user can change the position or posture of the device to make the interactive object 41 move to the reference position on the display medium. According to the closeness between the moving direction of each interactive object on the display medium and the reference position relative to the direction of the interactive object, To determine that the interactive object 41 should be selected. FIG. 13 schematically shows the display medium after the interactive object 41 is selected, where a hexagon can be used to identify the currently selected interactive object 41, and the interactive object 41 or the hexagon can be currently in The display position on the display medium is set as the new reference position. This is very advantageous, because the user usually gazes at the currently selected interactive object, which allows the user to easily know the current reference position and move a small amount to the desired next interactive object (for example, making The device performs a small rotation in the direction of the next interactive object) to select the next interactive object, thereby improving the interaction efficiency and improving the user's interactive experience.

In one embodiment, during the movement of the device, the amount of change in the distance between the display position of each interactive object and the reference position can be determined, and the interactive object with the fastest or the most reduced distance can be selected. The method is actually similar to the method of selecting the interactive object according to the direction of the reference position relative to the interactive object and the moving direction of the interactive object on the display medium shown in FIGS. 9 and 12. Generally, during the movement of the device, if the moving direction of an interactive object on the display medium is closest to the reference position relative to the direction of the interactive object, then the distance between the interactive object and the reference position The distance decreases the fastest or the most. In some embodiments, a distance reduction threshold or a distance reduction speed threshold may be set to avoid possible misoperation.

In another embodiment, the reference position on the display medium can also be changed according to the user's instruction. For example, the user can click on the display medium to determine a new reference position.

In some cases, the interactive objects presented on the display medium of the device may be objects with three-dimensional spatial positions, such as virtual objects arranged in a virtual reality or augmented reality scene. The position of the interactive objects may be represented by three-dimensional coordinates, for example. To represent. Since objects in the three-dimensional space can have different depths or distances relative to the device, it may sometimes be difficult to select interactive objects based on the presentation position of these interactive objects on the device display medium (for example, For example, although some interactive objects are far apart in the space, they may have relatively close presentation positions on the device display medium). This situation is particularly serious when there are more interactive objects in the space. To this end, in one embodiment, a method for selecting interactive objects presented on the display medium of a device is provided, wherein one or more interactive objects are located in the space around the device and can be displayed on the On the display medium, and wherein the position of the one or more interactive objects relative to the device will change as the position or posture of the device changes. The method is shown in Figure 14, and may include the following steps:

Step 601: Set a selection area in a space around the device, wherein the position of the selection area relative to the device remains unchanged before being reset.

The selection area may be a three-dimensional structure, such as a sphere, a cylinder, a cube, or other regular or irregular three-dimensional structures. Setting the selection area may include setting the position of the selection area relative to the device (for example, the distance and direction relative to the device), setting the shape of the selection area, setting the size of the selection area, and so on. However, it can be understood that the shape, size, or position relative to the device of the selection area may also be preset or default. The position of the selection area relative to the device remains unchanged before being reset. Therefore, the position of the selection area in space will change as the position or posture of the device changes, as if it is fixedly coupled to the device same. In an embodiment, for example, the direction of the selection area relative to the device can be set by selecting a certain position on the display medium of the device, and the distance of the selection area relative to the device can be set by dragging a sliding block, inputting a value, etc. Similarly, the position of the selected area relative to the device can be set according to the position of the currently selected interactive object in space.

Step 602: Obtain the position of the one or more interactive objects relative to the device according to the position and posture of the device and the position of the one or more interactive objects.

During the movement or rotation of the device, the position and/or posture change of the device can be measured or tracked in various known ways. For example, the built-in acceleration sensor, magnetic sensor, direction sensor, and Gravity sensor, gyroscope, camera, etc. pass Methods known in the art (for example, inertial navigation, visual odometry, SLAM, VSLAM, SFM, etc.) are used to measure or track changes in position and/or attitude. According to the position and posture of the device and the position of one or more interactive objects, the position of the one or more interactive objects relative to the device can be obtained.

Step 603: Automatically select an interactive object from the one or more interactive objects according to the position of the selection area relative to the device and the position of the one or more interactive objects relative to the device.

In one embodiment, if it is determined that the interactive object is located in the selection area according to the position of the selection area relative to the device and the position of the interactive object relative to the device, the interactive object may be selected .

In an embodiment of the present invention, the present invention can be implemented in the form of a computer program. The computer program can be stored in various storage media (for example, hard disk, optical disc, flash memory, etc.), and when the computer program is executed by the processor, it can be used to implement the method of the present invention.

In another embodiment of the present invention, the present invention can be implemented in the form of an electronic device. The electronic device includes a processor and a memory, and a computer program is stored in the memory. When the computer program is executed by the processor, it can be used to implement the method of the present invention.

References to "various embodiments", "some embodiments", "one embodiment", or "an embodiment" herein refer to the specific features, structures, or properties described in connection with the embodiments included in In at least one embodiment. Therefore, the appearances of the phrases "in various embodiments", "in some embodiments", "in one embodiment", or "in an embodiment" in various places throughout this document do not necessarily refer to the same Examples. In addition, specific features, structures, or properties can be combined in any suitable manner in one or more embodiments. Therefore, a specific feature, structure, or property shown or described in one embodiment can be combined in whole or in part with the feature, structure, or property of one or more other embodiments without limitation, as long as the combination is not Not logical or not working. Similar to "According to A" or "Based on The expression "in A" means non-exclusive, that is, "according to A" can cover "according to A only" or "according to A and B", unless specifically stated or clearly known from the context, its meaning is "according to only A". The various steps described in a certain order in the method flow do not have to be executed sequentially. On the contrary, the execution order of some steps can be changed, and some steps can be executed concurrently, as long as it does not affect the realization of the solution. In addition, the various elements in the drawings of this application are for illustrative purposes only, and are not drawn to scale.

Thus, several aspects of at least one embodiment of the present invention have been described, and it can be understood that various changes, modifications and improvements can be easily made by those skilled in the art. Such changes, modifications and improvements are intended to be within the spirit and scope of the present invention. Although the present invention has been described through preferred embodiments, the present invention is not limited to the embodiments described here, and also includes various changes and changes made without departing from the scope of the present invention.

In summary, although the present invention has been disclosed as above in preferred embodiments, the above preferred embodiments are not intended to limit the present invention. Those of ordinary skill in the art can make various modifications and changes without departing from the spirit and scope of the present invention. Retouching, therefore, the protection scope of the present invention is subject to the scope defined by the claims.

Therefore, the present invention has excellent advancement and practicability among similar products. At the same time, after searching through domestic and foreign technical documents about this type, it is true that no identical or similar structure or technology exists before the application of this case. This case should have met the patent requirements of "inventiveness", "applicability for industrial use" and "progressiveness", and an application was filed in accordance with the law.

501: step 501

502: step 502

Claims (19)

A method for selecting interactive objects presented on a display medium of a device, wherein a position on the display medium is set as a reference position, and one or more interactive objects are displayed on the display medium and can be selected , The display position of the one or more interactive objects on the display medium will change as the position or posture of the device changes, and the method includes: Obtaining the display position of the one or more interactive objects on the display medium; and The interactive object is automatically selected from the one or more interactive objects according to the obtained display position and the reference position. The method according to item 1 of the scope of patent application, wherein the obtaining the display position of the one or more interactive objects on the display medium includes: The display position of the one or more interactive objects on the display medium is determined during and/or after the position or posture of the device changes. The method according to the first item of the scope of patent application, wherein after the interactive object is selected, the reference position is set according to the display position of the currently selected interactive object on the display medium. The method according to any one of items 1 to 3 of the scope of patent application, wherein automatically selecting an interactive object from the one or more interactive objects according to the obtained display position and the reference position includes: Determine the distance between the interactive object and the reference position according to the finally obtained display position of each interactive object and the reference position; and The interactive object is automatically selected according to the determined distance. The method according to item 4 of the scope of patent application, wherein said automatically selecting interactive objects according to the determined distance includes: If the distance between the interactive object and the reference position is less than the threshold, the interactive object is selected. The method according to item 5 of the scope of patent application, wherein if the distance between each of the plurality of interactive objects and the reference position is less than the threshold, the interactive object closest to the reference position is selected. The method according to item 4 of the scope of patent application, wherein the determining the distance between the interactive object and the reference position according to the last obtained display position of each interactive object and the reference position comprises: repeatedly according to The last obtained display position of each interactive object and the reference position determine the distance between the interactive object and the reference position. The method according to item 7 of the scope of patent application, wherein the automatically selecting the interactive object according to the determined distance includes: If the distance between the interactive object and the reference position continues to decrease, and the current distance is less than the threshold, the interactive object is selected. The method according to any one of items 1 to 3 of the scope of patent application, wherein automatically selecting an interactive object from the one or more interactive objects according to the obtained display position and the reference position includes: Determine the direction of the reference position relative to the interactive object according to the display position of each interactive object on the display medium; Determine the moving direction of each interactive object on the display medium according to the change of the display position of each interactive object on the display medium; and The interactive object is automatically selected according to the direction of the reference position relative to the interactive object and the moving direction of the interactive object on the display medium. The method according to item 9 of the scope of patent application, wherein the automatically selecting the interactive object according to the direction of the reference position relative to the interactive object and the moving direction of the interactive object on the display medium includes: Determine the proximity of the movement direction of each interactive object on the display medium and the reference position relative to the direction of the interactive object, and select the interactive object with the highest proximity; or If the proximity between the moving direction of the interactive object on the display medium and the direction of the reference position relative to the interactive object satisfies a predetermined condition, the interactive object is selected. The method according to any one of items 1 to 3 in the scope of the patent application, wherein the automatically selecting an interactive object from the one or more interactive objects according to the obtained display position and the reference position includes: Determine the amount of change in the distance between the display position of each interactive object and the reference position; and select the interactive object with the fastest or most reduced distance. The method according to any one of items 1 to 3 in the scope of the patent application, wherein the automatically selecting an interactive object from the one or more interactive objects according to the obtained display position and the reference position includes: Determine the direction of the reference position relative to the interactive object according to the display position of each interactive object on the display medium; Determine the moving direction of the interactive object on the display medium according to the change of the display position of each interactive object on the display medium; Determine the distance between the interactive object and the reference position according to the finally obtained display position of each interactive object and the reference position; and According to the direction of the reference position relative to the interactive object, the movement of the interactive object on the display medium The direction and the distance between the interactive object and the reference position are used to automatically select the interactive object. The method according to any one of items 1 to 3 in the scope of patent application, wherein the reference position does not change with the change of the position or posture of the device. The method described in item 1 or item 2 of the scope of patent application, wherein a position on the display medium is set as a reference position in the following manner: Set a certain preset or default position on the display medium as the reference position; or The reference position is set according to the user's instruction. A method for selecting interactive objects presented on a display medium of a device, wherein one or more interactive objects are located in the space around the device and can be displayed on the display medium, and wherein, the one or The positions of multiple interactive objects relative to the device will change as the position or posture of the device changes, and the method includes: Setting a selection area in a space around the device, wherein the position of the selection area relative to the device remains unchanged before being reset; Obtaining the position of the one or more interactive objects relative to the device according to the position and posture of the device and the position of the one or more interactive objects; and An interactive object is automatically selected from the one or more interactive objects according to the position of the selection area relative to the device and the position of the one or more interactive objects relative to the device. The method according to item 15 of the scope of patent application, wherein the selected area is a three-dimensional structure. The method according to item 15 of the scope of the patent application, wherein, according to the position of the selection area relative to the device and the position of the one or more interactive objects relative to the device, the one or more The automatic selection of interactive objects in interactive objects includes: If it is determined that the interactive object is located in the selection area according to the position of the selection area relative to the device and the position of the interactive object relative to the device, the interactive object is selected. A computer program product, which can be used to implement the method described in any one of claims 1-17 when the computer program product is executed by a processor. An electronic device comprising a processor and a memory, and a computer program is stored in the memory. When the computer program is executed by the processor, it can be used to implement the method according to any one of claims 1-17.

Images