CN114740973A - A tracking method and device - Google Patents

Abstract

The application discloses a tracking method and a device, wherein the tracking method comprises the following steps: the method comprises the steps of obtaining position information of a first object relative to a region to be controlled and sight line parameters of the first object, and inputting the position information and the sight line parameters into a target tracking model to determine the gaze position of the first object in the region to be controlled. The target tracking model comprises a first tracking model obtained by processing an original tracking model based on change information of a sensor module and the display area, and the sensor module is used for acquiring the position information and/or the sight line parameters.

Description

A tracking method and device

technical field

The present application relates to the field of electronic technologies, and in particular, to a tracking method and device.

Background technique

Eye tracking is a machine vision technology that captures the user's eye image through the device, analyzes it with an algorithm, and finally obtains the position of the user's gaze on the display screen, which has high application value. Eye tracking technology needs to first determine the relative positional relationship between the user's eyes and the display screen. The relative position of the screen.

In this regard, it is currently necessary for the display screen of a specific model to fix the eye tracking device at a place with a definite relative position to the display screen to realize eye tracking. However, for a touch screen display system, since the display screen has a definite relative positional relationship with the eye tracking device, and the size and position of the display screen are fixed, the user can Use the eye tracking system to position the display screen within a specific angular range. In addition, since the relative position of the eye-tracking device and the screen of the corresponding display device must be fixed, the user must use the eye-tracking system at a very close distance, which is more restricted and inconvenient to use.

SUMMARY OF THE INVENTION

Embodiments of the present application provide a tracking method and device.

According to a first aspect of the embodiments of the present application, a tracking method is provided, the method includes: obtaining position information of a first object relative to a to-be-controlled area and line-of-sight parameters of the first object, where the to-be-controlled area at least includes electronic The device is used to output the display area of the displayed content; the position information and the sight line parameter are input into a target tracking model to determine the gaze position of the first object in the area to be controlled; wherein, the target tracking model It includes a first tracking model obtained by processing the original tracking model based on the change information of the sensor module and the display area, and the sensor module is used for collecting the position information and/or the line of sight parameter.

According to an embodiment of the present application, inputting the first position information and the line-of-sight parameters into a target tracking model includes: obtaining change information of the sensor module and the display area; if the change information conforms to the adjustment condition, input the position information and the line of sight parameters into the first tracking model; and/or, if the change information does not meet the adjustment conditions, input the position information and the line of sight parameters into the original tracking model , the original tracking model is a model trained based on the first object or a second object different from the first object.

According to an embodiment of the present application, if the change amount of at least one of the display area, the sensor module, or the relative positional relationship between the sensor module and the display area is not less than the first A set value to determine that the change information meets the adjustment conditions; and/or, if any of the display area, the sensor module, and the relative positional relationship between the sensor module and the display area before There is no change or at least one of the changes is smaller than the second set value, and it is determined that the change information does not meet the adjustment condition.

According to an embodiment of the present application, the training process of the original tracking model includes: acquiring area information of the to-be-controlled area, where the area information at least includes the size and/or shape of the to-be-controlled area; The distance between the areas to be controlled and the sight line parameters of the calibration object looking at multiple designated positions in the area to be controlled, the sight line parameters include sight angle and sight height; based on the area information, the distance and The sight line parameter constructs a three-dimensional space coordinate system as the original tracking model; wherein, the calibration object is the first object or a second object different from the first object.

According to an embodiment of the present application, determining the gaze position of the first object in the area to be controlled includes: determining the vertical distance of the eyeball of the first object relative to the display area; determining the first object The angle between the current sight line and the display area and the current sight height of the first object; according to the included angle, the sight height and the vertical distance, determine the intersection point of the sight line on the display area, as the gaze position.

According to an embodiment of the present application, wherein processing the original tracking model based on the change information of the sensor module and the display area includes: determining the position of the first object relative to the area to be controlled based on the change information change information and line-of-sight change parameters of the first object; adjust parameters of the original tracking model based on the position change information and the line-of-sight change parameters to obtain the first tracking model.

According to an embodiment of the present application, the sensor module includes a first module and a second module; correspondingly, the obtaining the position information of the first object relative to the area to be controlled and the line-of-sight parameters of the first object, The method includes: if the first object exceeds the collection range of the first module, executing: outputting prompt information; and/or, controlling the second module to obtain the position information and the line of sight parameter.

According to an embodiment of the present application, the method further includes: obtaining position information of multiple objects relative to the area to be controlled and line-of-sight parameters of the multiple objects, where the multiple objects include a first object; The gaze position of an object in the to-be-controlled area, the positional relationship between the first object and the second object among the plurality of objects, and the line-of-sight parameters of the second object determine where the second object is located. Describe the gaze position in the area to be controlled.

According to an embodiment of the present application, the method further includes: obtaining first position information of the first object relative to the area to be controlled at the current moment and first sight line parameters of the first object; obtaining the motion of the first object information and line-of-sight change information; predict the second line-of-sight parameter of the first object at the target moment and its second position information relative to the area to be controlled based on the motion information and the line-of-sight change information; The second line of sight parameter and the second position information determine the gaze position of the first object in the area to be controlled.

According to a second aspect of the present application, there is also provided a tracking device, comprising: an acquisition module configured to acquire position information of a first object relative to a to-be-controlled area and line-of-sight parameters of the first object, wherein the to-be-controlled area at least It includes a display area where the electronic device is used to output display content; a tracking module is used to input the position information and the sight line parameter into a target tracking model to determine the gaze position of the first object in the to-be-controlled area; Wherein, the target tracking model includes a first tracking model obtained by processing the original tracking model based on the change information of the sensor module and the display area, and the sensor module is used to collect the position information and/or the line of sight parameters.

According to a third aspect of the present application, it is further provided that the device includes at least one processor, and at least one memory and a bus connected to the processor; wherein, the processor and the memory communicate with each other through the bus. communication; the processor is used for calling program instructions in the memory to execute the above tracking method.

The tracking method and device according to the embodiment of the present application obtain the position information of the first object relative to the area to be controlled and the line of sight parameters of the first object, and input the position information and the line of sight parameters into the target tracking model to determine the target tracking model. the gaze position of the first object in the to-be-controlled area. The to-be-controlled area at least includes a display area used by the electronic device to output display content, and the target tracking model includes a first tracking model obtained by processing the original tracking model based on the change information between the sensor module and the display area. The sensor module is used to collect the position information and/or the line of sight parameter. Thus, a target tracking model obtained by processing the original tracking model based on the change information of the sensor module and the display area can be used to track the gaze position of the first object by using the target tracking model. This effectively avoids the complex joint debugging of the sensor module and the electronic device when the position or model specification of the sensor module or the display area of the electronic device is changed. Sensor modules can be deployed independently and do not need to be fixed to a specific location on the screen. The use of the tracking function of the electronic device by the user is no longer limited to a short distance from the display area, effectively broadening the user's use position and viewing angle of the tracking function of the electronic device.

It should be understood that the teachings of the present application do not need to achieve all the above-mentioned beneficial effects, but specific technical solutions can achieve specific technical effects, and other embodiments of the present application can also achieve the beneficial effects not mentioned above. Effect.

Description of drawings

The above and other objects, features and advantages of exemplary embodiments of the present application will become readily understood by reading the following detailed description with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present application are shown by way of example and not limitation, wherein:

In the drawings, the same or corresponding reference numerals denote the same or corresponding parts.

FIG. 1 shows a schematic diagram of an application scenario of a tracking method according to an embodiment of the present application;

FIG. 2 shows a schematic flowchart of the implementation of the tracking method according to the embodiment of the present application;

FIG. 3 shows the spatial correspondence between the display area and the object of the tracking method according to the embodiment of the present application;

FIG. 4 shows a schematic diagram of the composition and structure of a tracking device according to an embodiment of the present application.

Detailed ways

The principles and spirit of the present application will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are only provided for those skilled in the art to better understand and implement the present application, but do not limit the scope of the present application in any way. Rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of this application to those skilled in the art.

The technical solutions of the present application will be further elaborated below with reference to the accompanying drawings and specific embodiments.

FIG. 1 shows a schematic diagram of an application scenario of the tracking method according to the embodiment of the present application.

Referring to Figure 1, the tracking system tracks the user's line of sight. The tracking system includes at least a sensor module. The user stands at any position facing the display area and within the working range of the sensor module. The sensor module can collect the user's line of sight. Parameters and user position information relative to the display area. The user may be a real person, a robot or an animal that needs to perform gaze tracking, etc. Therefore, the solution will be described below with a first object and a calibration object, etc. The sensor module may include an image acquisition device and the like. There are four preset gaze positions in the display area, and the display area here is the area to be controlled.

It should be noted that the above application scenarios are only application scenarios given to better describe the specific implementation details of the tracking method in the embodiment of the present application, and are not used to limit the implementation of the tracking method in the present application.

FIG. 2 shows a schematic flowchart of the implementation of the tracking method according to the embodiment of the present application.

Referring to FIG. 2 , the tracking method according to the embodiment of the present application includes at least the following operation flow: In operation 201, the position information of the first object relative to the to-be-controlled area and the line-of-sight parameters of the first object are obtained, and the to-be-controlled area at least includes an electronic device for outputting The display area of the display content; operation 202, input the position information and the sight line parameter into the target tracking model to determine the gaze position of the first object in the area to be controlled; wherein, the target tracking model includes the change information based on the sensor module and the display area The first tracking model obtained by processing the original tracking model, and the sensor module is used to collect position information and/or line of sight parameters.

In operation 201, position information of a first object relative to a to-be-controlled area and line-of-sight parameters of the first object are obtained, where the to-be-controlled area at least includes a display area used by the electronic device to output display content.

In this embodiment of the present application, the to-be-controlled area may be all areas of the display screen of the electronic device, or may only include the display area currently used to output display content in the display screen, or may be the projection area projected by the electronic device.

In this embodiment of the present application, the position information may include: the vertical relative distance of the first object relative to the area to be controlled, the relative height ratio of the eye height of the first object to the display area when the horizontal ground is used as the reference plane, The movement track or movement trend of the first object, etc. The line-of-sight parameters may include the line-of-sight height of the first object when the horizontal ground is used as the reference plane, the line-of-sight angle determined according to the eyeballs when the first object is gazing at the area to be controlled, and the line-of-sight duration for which the eyeballs of the first object remain stationary.

In this embodiment of the present application, the position information of the first object relative to the area to be controlled and the line-of-sight parameters of the first object can be acquired through the sensor module. The sensor module may include a first module and a second module. Correspondingly, in the process of obtaining the position information of the first object relative to the area to be controlled and the line-of-sight parameters of the first object, if the first object exceeds the collection range of the first module, prompt information may be output. For example: "Please wear AR glasses" or "Please be at the information collection table and look at the screen through the desktop sensor" for reminding the use of the second module for eye tracking. It is also possible to directly control the second module to obtain position information and line-of-sight parameters. Similarly, the second module can be controlled to obtain position information and line of sight parameters while outputting prompt information.

For example, the first module may be a fixed image acquisition device. The second module can be mobile glasses with eye tracking sensors or desktop sensors or the like. Wherein, the second data collection range of the second module is larger than the first data collection range of the first module and the like. Therefore, for some extreme cases where the first object is located far from the display area, for example, the distance between the first object and the display area is more than ten meters, you can configure glasses with eye-tracking sensors or desktop sensors, etc. It can effectively increase the accuracy of human eye tracking and further improve the accuracy of tracking.

In other embodiments, the first module and the second module may also be camera modules with different configurations, for example, the first module is a fixed-focus or short-focus camera or camera array, and the second module is a zoom Or telephoto camera or camera array, and even the second module can be a movable camera or camera array to distinguish it from the first module.

In operation 202, the position information and the line-of-sight parameters are input into the target tracking model to determine the gaze position of the first object in the area to be controlled, wherein the target tracking model includes performing an algorithm on the original tracking model based on the change information of the sensor module and the display area. After processing the obtained first tracking model, the sensor module is used to collect position information and/or line of sight parameters.

In this embodiment of the present application, the training process of the original tracking model may include the following operations: acquiring area information of the area to be controlled, the area information at least including the size and/or shape of the area to be controlled, and obtaining the calibration object and the area to be controlled The distance between them, and the sight line parameters of the calibration object looking at multiple specified positions in the area to be controlled, the sight line parameters include sight line angle and sight line height, so as to construct a three-dimensional space coordinate system based on the area information, distance and sight line parameters, as the original tracking model. The calibration object may be a first object or a second object different from the first object.

For example, when the user uses the tracking system for the first time, the user can perform correction operations at any position within the working range of the sensor module, facing the area to be controlled, and construct and train the original tracking model. First-time use refers to the situation where the tracking system is used for the first time. Any position within the working range of the sensor module refers to the range that can be detected by the eye tracking sensor and the image acquisition device in the sensor module.

With reference to FIG. 1 , the tracking system can sequentially display the positioning coordinates in the four corners of the display area, for example, the “+”-shaped identification positions in the four corners of the display area in FIG. 1 . The first object may look at each positioning coordinate in turn according to the display order of the positioning coordinates. The first object or the calibration image is shown in FIG. 1 and the following FIG. When the first object looks at any positioning coordinate at a fixed position within the working range of the sensor module, the sensor module in the tracking system can obtain the area information of the area to be controlled, and the area information at least includes the size of the area to be controlled and/or or shape, and obtain the distance between the calibration object and the area to be controlled, and the sight line parameters of the calibration object looking at multiple designated positions in the area to be controlled, where the sight line parameters include sight angle and sight height.

Specifically, the area information of the area to be controlled can be acquired through the image acquisition device in the sensor module. The area information may include relative positional relationship or/or size or/or shape among multiple designated locations in the area to be controlled. The area to be controlled is shown in a rectangle in FIG. 1 . Here, the area to be controlled is the same in position and size as the display area. In practical application, the area to be controlled may be a rectangle, and the size of the corresponding area to be controlled may include the length and width of the rectangle. The area to be controlled may also be a circle, and the corresponding size of the area to be controlled may include a center position and a radius of the circle. Similarly, the area to be controlled may also be an ellipse, a semicircle, a sector, or a combination of two or more graphics, and the like. In addition, the entire area of the partial area of the screen of the electronic device can also be determined as the to-be-controlled area, gaze tracking can be performed, and only the right 3/4 area of the rectangular screen area is configured as the display area. Taking Figure 1 as a reference, assuming that the fixed position of the calibration object shown in Figure 1 looks at the positioning coordinate 1, for the distance between the calibration object and the area to be controlled, the sensor module can collect the distance between the calibration object and the sensor module between the fixed position and the sensor module. The first distance, and the second distance between the sensor module and the display area. If the sensor module and the calibration object are on the same side of the plane where the display area is located, the distance between the calibration object and the area to be controlled is the sum of the first distance and the second distance. If the sensor module and the calibration object are on the plane where the display area is located The distance between the calibration object and the area to be controlled is the first distance minus the second distance. The line-of-sight parameters of the calibration object's gaze positioning coordinate 1 may include a line-of-sight angle and a line-of-sight height. The sight angle is also called the eyeball offset angle, which can be directly obtained by the eyeball sensor in the sensor module. Gaze height refers to the vertical distance between the eyeball and the ground when the calibration object looks at the positioning coordinate 1 at a fixed position. It can also be obtained directly by the module sensor.

Further, when the calibration object looks at the positioning coordinates 2, 3 and 4 at a fixed position, the corresponding line of sight parameters can be obtained by using the same method. Thus, a three-dimensional space coordinate system is constructed based on the distance between the calibration object and the area to be controlled, and the line of sight parameters of the calibration object looking at multiple designated positions in the area to be controlled, which can be used as the original tracking model.

FIG. 3 shows the spatial correspondence between the display area and the object of the tracking method according to the embodiment of the present application. Here, the three-dimensional space coordinate system is described with reference to FIG. 3 . The three-dimensional space coordinate system can show the three-dimensional coordinate position (x, y, z) of the eyeball position of the calibration object when the calibration object is looking at the area to be controlled at any position, and the calibration object at this coordinate is (x, y, z) The mapping between the gaze position of the area to be controlled and the eye angle of the calibration object. Therefore, under the condition that the relative position of the sensor module and the area to be controlled does not change, as long as the three-dimensional coordinate position (x1, y1, z1) of the eyeball of the first object and the current eyeball angle of the first object can be obtained, Then, the first object can be determined according to the mapping between the gaze position of the calibration object in the three-dimensional space coordinate system used to represent the original tracking model and the gaze position of the area to be controlled and the eye angle of the calibration object at the position where the coordinates are (x1, y1, z1). The gaze position of the currently gazed area to be controlled.

It should be noted that the line-of-sight parameters of the first object may also include the fixation time during which the first object keeps looking at the current position, and the fixation time duration may be determined according to actual needs, for example, set to 1s, 3s, or 3.5s, etc. The time is to ensure the stability of the tracking method, avoid the first object staying for a short time in the process of scanning the area to be controlled, and then determine the gaze position, and effectively reduce the calculation amount of the tracking method.

In this embodiment of the present application, the first position information and line of sight parameters are input into the target tracking model, and during the process, the change information of the sensor module and the display area can be obtained first. If the change information meets the adjustment conditions, the position information and the line of sight parameters are input into the first tracking model. If the change information does not meet the adjustment conditions, the position information and the line of sight parameters are input into the original tracking model, which is a model trained based on the first object or a second object different from the first object.

In this embodiment of the present application, wherein, if the variation of at least one of the display area, the sensor module, or the relative positional relationship between the sensor module and the display area is not less than the first set value, determine The change information meets the adjustment conditions; and/or, if any one of the display area, the sensor module, and the relative positional relationship between the sensor module and the display area is unchanged or at least one of the changes is less than the second set value , determine that the change information does not meet the adjustment conditions.

It should be noted that the first set value of the change amount here may be 0, that is, as long as there is a change, it is determined that the change information meets the adjustment condition.

Specifically, when the display area changes and the magnitude of the change exceeds a corresponding set threshold, it may be determined that the change information meets the adjustment conditions, for example, the size or shape of the display area changes.

It can also be determined that the change information meets the adjustment conditions when the relative position between the sensor module and the display area changes, and the magnitude of the change exceeds the corresponding set threshold. The angle, relative position, or the vertical distance of the sensor module from the display area, etc. change.

It can also be determined that the change information meets the adjustment conditions when the sensor module itself changes, such as: sensor module replacement type, firmware upgrade, configuration upgrade, and the like.

Conversely, if all configurations, such as the position and firmware of the display area and sensor module, are changed, the adjustment conditions are not met.

In this embodiment of the present application, it can also be set that the adjustment condition is not met when the relative position between the display area and the sensor module does not change or changes within a preset range.

In this embodiment of the present application, the following operations can be used to process the original tracking model based on the change information of the sensor module and the display area: determining the position change information of the first object relative to the area to be controlled and the first object based on the change information The original tracking model is adjusted based on the position change information and the line of sight change parameters to obtain the first tracking model.

For example, with reference to Figure 3, if the vertical distance between the sensor module and the display area is unchanged, the height of the sensor module from the ground is unchanged, and the relative position of the sensor module relative to the display area only changes in the X-axis direction, then you can The information of Y-axis and Z-axis remains unchanged. If the sensor module moves x1 along the positive axis of X-axis, x1 can be subtracted from the X-axis coordinates of all positions represented by three-dimensional coordinates in the original three-dimensional space coordinate system. For the situation where the position of the sensor module relative to the display area changes on the Y-axis or Z-axis, the corresponding principle can be adopted to adjust the parameters of the original tracking model based on the position change information and the line-of-sight change parameters to obtain the first tracking model.

In this embodiment of the present application, the following operations can be used to determine the gaze position of the first object in the area to be controlled: determine the vertical distance of the eyeball of the first object relative to the display area; determine the current line of sight of the first object and the display area and the current sight height of the first object; according to the included angle, sight height and vertical distance, determine the intersection point of the sight line on the display area as the gaze position.

For example, referring to FIG. 3 again, according to the vertical distance of the eyeball of the first object relative to the display area, the Y-axis information of the eyeball of the first object in the three-dimensional space coordinate system can be determined. In addition, in the process of determining the vertical distance of the eyeball of the first object relative to the display area, the vertical connection line between the eyeball of the first object and the display area can be determined, and further the X-axis information of the eyeball of the first object in the three-dimensional space can be determined accordingly. . The Z-axis information of the eyeball of the first object in the three-dimensional space coordinate system can be determined according to the current height of sight of the first object. The angle between the current line of sight of the first object and the display area can be determined according to the data of the eyeball shift angle sensor in the sensor module. Therefore, the problem of determining the gaze position of the first object in the area to be controlled can be converted into a problem of pure mathematical trigonometric functions for solving.

In this embodiment of the present application, position information of multiple objects relative to the to-be-controlled area and line-of-sight parameters of multiple objects can also be obtained, and based on the gaze position of the first object in the to-be-controlled area, the relationship between the first object and the multiple objects The positional relationship of the second object among the plurality of objects and the line-of-sight parameter of the second object determine the gaze position of the second object in the area to be controlled, wherein the multiple objects include the first object.

For example, if there are multiple objects looking at the same or different positions in the same area to be controlled at the same time, the gaze position of each object needs to be determined. Time-series tracking of sight lines of multiple objects. It is also possible to perform real-time tracking of the sight lines of multiple objects synchronously with multiple threads. In this process, the positional relationship between the objects can be determined according to the identifiers of multiple objects, and in the process of time-series line-of-sight tracking of the objects, the line-of-sight parameters and gaze position of the first object at the previous moment can be used, and The positional relationship between the first object and the second object and the current gaze parameter of the second object determine the current gaze position of the second object in the area to be controlled.

In this embodiment of the present application, the first position information of the first object relative to the area to be controlled at the current moment and the first sight line parameter of the first object are also obtained, and the motion information and sight line change information of the first object are obtained, Thus, the second line of sight parameter of the first object at the target time and its second position information relative to the area to be controlled are predicted based on the motion information and the line of sight change information, and further, the first object is determined based on the second line of sight parameter and the second position information. The gaze position of the object in the area to be controlled.

For example, if the sight line parameter of the first object does not change, but the first object only moves relative to the display area in the direction of the positive X-axis, the gaze position of the first object also follows the direction of the first object relative to the display area. Movement information, movement occurs in the positive direction of the X-axis. It should be noted that this is only an exemplary description, and does not form a limitation on the tracking method in practical applications.

The tracking method and device according to the embodiments of the present application obtain the position information of the first object relative to the to-be-controlled area and the line-of-sight parameters of the first object, and input the position information and line-of-sight parameters into the target tracking model to determine that the first object is in the to-be-controlled area gaze position in . The area to be controlled at least includes a display area used by the electronic device to output display content, and the target tracking model includes a first tracking model obtained by processing the original tracking model based on the change information between the sensor module and the display area. The sensor module uses for collecting location information and/or line-of-sight parameters. In this way, a target tracking model obtained by processing the original tracking model based on the change information of the sensor module and the display area can be used to track the gaze position of the first object by using the target tracking model. This effectively avoids the complex joint debugging of the sensor module and the electronic device when the position or model specification of the sensor module or the display area of the electronic device is changed. Sensor modules can be deployed independently and do not need to be fixed to a specific location on the screen. The use of the tracking function of the electronic device by the user is no longer limited to a short distance from the display area, effectively broadening the user's use position and viewing angle of the tracking function of the electronic device.

Similarly, based on the above tracking method, an embodiment of the present application further provides a computer-readable storage medium, where the computer-readable storage medium stores a program, and when the program is executed by the processor, the processor is caused to perform at least the following operation steps: In operation 201, the position information of the first object relative to the area to be controlled and the line-of-sight parameters of the first object are obtained, and the area to be controlled at least includes a display area used by the electronic device to output the display content; operation 202, the position information and the line-of-sight parameters are input into the target A tracking model to determine the gaze position of the first object in the area to be controlled; wherein, the target tracking model includes a first tracking model obtained by processing the original tracking model based on the change information of the sensor module and the display area, and the sensor module uses for collecting location information and/or line-of-sight parameters.

Further, based on the above tracking method, an embodiment of the present application further provides a tracking device, as shown in FIG. 4 , the tracking device 40 includes: an obtaining module 401 for obtaining the position information of the first object relative to the to-be-controlled area and the first The sight line parameter of the object, the area to be controlled at least includes the display area used by the electronic device to output the display content; the tracking module 402 is used to input the position information and the sight line parameter into the target tracking model to determine the gaze of the first object in the area to be controlled position; wherein, the target tracking model includes a first tracking model obtained by processing the original tracking model based on the change information of the sensor module and the display area, and the sensor module is used to collect position information and/or line-of-sight parameters.

In this embodiment of the present application, the tracking module 402 includes: a change acquisition sub-module for obtaining the change information of the sensor module and the display area; an input sub-module for obtaining the position information if the change information meets the adjustment conditions and line-of-sight parameters into the first tracking model; and/or, if the change information does not meet the adjustment conditions, input the position information and line-of-sight parameters into the original tracking model based on the first object or a second object different from the first object trained model.

In this embodiment of the present application, the input sub-module determines whether the change information meets the adjustment conditions according to the following operations: if at least one of the relative positional relationship between the display area, the sensor module, or the sensor module and the display area The amount of change of one of them is not less than the first set value, and it is determined that the change information meets the adjustment conditions; and/or, if any one of the display area, the sensor module, and the relative positional relationship between the sensor module and the display area is not If the amount of change or at least one of the changes is smaller than the second set value, it is determined that the change information does not meet the adjustment condition.

In this embodiment of the present application, the training process of the original tracking model includes: acquiring area information of the area to be controlled, where the area information at least includes the size and/or shape of the area to be controlled; obtaining the distance between the calibration object and the area to be controlled , and the sight parameters of the calibration object looking at multiple specified positions in the area to be controlled, the sight parameters include sight angle and sight height; based on the area information, distance and sight parameters, a three-dimensional space coordinate system is constructed as the original tracking model; among them, the calibration object is A first object or a second object different from the first object.

In this embodiment of the present application, the tracking module 402 includes: a distance determination sub-module for determining the vertical distance of the eyeball of the first object relative to the display area; a height determination sub-module for determining the current line of sight of the first object and the The included angle of the display area and the current sight height of the first object; the gaze position determination submodule is used to determine the intersection point of the sight line on the display area as the gaze position according to the included angle, the sight height and the vertical distance.

In this embodiment of the present application, the processing of the original tracking model based on the change information of the sensor module and the display area includes: determining the position change information of the first object relative to the area to be controlled and the first object based on the change information The original tracking model is adjusted based on the position change information and the line of sight change parameters to obtain the first tracking model.

In this embodiment of the present application, the sensor module includes a first module and a second module; correspondingly, the acquisition module 401 includes: a first acquisition sub-module for if the first object exceeds the size of the first module Collect the range, and execute: output prompt information; and/or, control the second module to obtain position information and line-of-sight parameters.

In this embodiment of the present application, the tracking device 40 further includes: a multi-object acquisition module, configured to obtain position information of the plurality of objects relative to the area to be controlled and line-of-sight parameters of the plurality of objects, wherein the plurality of objects includes the first object A relative tracking module for determining the position of the second object to be controlled based on the gaze position of the first object in the area to be controlled, the positional relationship between the first object and the second object in the plurality of objects, and the line of sight parameters of the second object Gaze position in the area.

In this embodiment of the present application, the tracking device 40 further includes: a real-time acquisition module for obtaining the first position information of the first object relative to the area to be controlled at the current moment and the first sight line parameter of the first object; motion acquisition The module is used to obtain the motion information and line-of-sight change information of the first object; the position determination module is used to predict the second line-of-sight parameter of the first object at the target moment based on the motion information and the line-of-sight change information and its first line of sight parameter relative to the area to be controlled. Two position information; a motion tracking module, configured to determine the gaze position of the first object in the area to be controlled based on the second sight line parameter and the second position information.

It should be pointed out here that the above description of the embodiment of the tracking device is similar to the description of the method embodiment shown in the foregoing FIGS. 1 to 3 , and has similar benefits to the method embodiment shown in the foregoing FIGS. effect, so I won't go into details. For technical details not disclosed in the embodiments of the tracking device of the present application, please refer to the descriptions of the method embodiments shown in FIG. 1 to FIG. 3 in the foregoing application for understanding, and to save space, details are therefore omitted.

It should be noted that, herein, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements, It also includes other elements not expressly listed or inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other manners. The device embodiments described above are only illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated. to another system, or some features can be ignored, or not implemented. In addition, the coupling, or direct coupling, or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical or other forms. of.

The unit described above as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit; it may be located in one place or distributed to multiple network units; Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present application may all be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above integration The unit can be implemented either in the form of hardware or in the form of hardware plus software functional units.

Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by program instructions related to hardware, the aforementioned program can be stored in a computer-readable storage medium, and when the program is executed, the execution includes: The steps of the above method embodiments; and the aforementioned storage medium includes: a removable storage device, a read only memory (Read Only Memory, ROM), a magnetic disk or an optical disk and other media that can store program codes.

Alternatively, if the above-mentioned integrated units of the present application are implemented in the form of software function modules and sold or used as independent products, they may also be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application can be embodied in the form of software products in essence or in the parts that make contributions to the prior art. The computer software products are stored in a storage medium and include several instructions for A computer device (which may be a personal computer, a server, or a network device, etc.) is caused to execute all or part of the methods of the various embodiments of the present application. The aforementioned storage medium includes various media that can store program codes, such as a removable storage device, a ROM, a magnetic disk, or an optical disk.

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, and should cover within the scope of protection of this application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (10)

1. A method of tracking, comprising:

obtaining position information of a first object relative to a to-be-controlled area and sight line parameters of the first object, wherein the to-be-controlled area at least comprises a display area used by electronic equipment for outputting display content;

inputting the position information and the sight line parameters into a target tracking model to determine a fixation position of the first object in the area to be controlled;

the target tracking model comprises a first tracking model obtained by processing an original tracking model based on change information of a sensor module and the display area, and the sensor module is used for acquiring the position information and/or the sight line parameters.

2. The method of claim 1, wherein inputting the first location information and the gaze parameters into a target tracking model comprises:

obtaining the change information of the sensor module and the display area;

if the change information meets the adjustment condition, inputting the position information and the sight line parameter into the first tracking model; and/or the presence of a gas in the atmosphere,

if the change information does not meet an adjustment condition, inputting the position information and the gaze parameter into the original tracking model, which is a model trained on the first object or a second object different from the first object.

3. The method of claim 2, wherein,

if the variation of at least one of the display area, the sensor module or the relative position relationship between the sensor module and the display area is not less than a first set value, determining that the variation information meets the adjustment condition; and/or the presence of a gas in the atmosphere,

and if any one of the display area, the sensor module and the relative position relation between the sensor module and the display area is not changed or the variation of at least one of the sensor module and the relative position relation is smaller than a second set value, determining that the change information does not accord with the adjustment condition.

4. The method according to claim 1 or 2, the training process of the original tracking model comprising:

acquiring area information of the area to be controlled, wherein the area information at least comprises the size and/or the shape of the area to be controlled;

obtaining the distance between a calibration object and the area to be controlled and sight line parameters of the calibration object gazing at a plurality of designated positions in the area to be controlled, wherein the sight line parameters comprise sight line angles and sight line heights;

constructing a three-dimensional space coordinate system based on the region information, the distance and the sight line parameters to serve as the original tracking model;

wherein the calibration object is the first object or a second object different from the first object.

5. The method of any of claims 1 to 3, wherein determining the gaze location of the first object in the area to be controlled comprises:

determining a vertical distance of an eyeball of a first object relative to the display area;

determining an included angle between the current sight line of the first object and the display area and the current sight line height of the first object;

and determining the intersection point of the sight line on the display area as the gaze position according to the included angle, the sight line height and the vertical distance.

6. The method of claim 1, wherein processing the raw tracking model based on the change information of the sensor module and the display area comprises:

determining position change information of the first object relative to the area to be controlled and a sight line change parameter of the first object based on the change information;

and adjusting parameters of the original tracking model based on the position change information and the sight line change parameters to obtain the first tracking model.

7. The method of claim 1, wherein the first and second light sources are selected from the group consisting of,

the sensor module comprises a first module and a second module; accordingly, the method can be used for solving the problems that,

the obtaining of the position information of the first object relative to the area to be controlled and the sight line parameter of the first object includes:

if the first object is beyond the acquisition range of the first module, executing the following steps:

outputting prompt information; and/or the presence of a gas in the gas,

and controlling a second module to obtain the position information and the sight line parameter.

8. The method of claim 1, further comprising:

obtaining position information of a plurality of objects relative to an area to be controlled and sight line parameters of the plurality of objects, wherein the plurality of objects comprise a first object;

determining a gaze location of the second object in the area to be controlled based on a gaze location of the first object in the area to be controlled, a positional relationship of the first object to a second object of the plurality of objects, and a line of sight parameter of the second object.

9. The method of claim 1, further comprising:

obtaining first position information of a first object relative to a region to be controlled at the current moment and a first sight line parameter of the first object;

obtaining motion information and gaze change information of the first object;

predicting a second sight line parameter of the first object at a target moment and second position information of the first object relative to the area to be controlled based on the motion information and the sight line change information;

and determining the fixation position of the first object in the area to be controlled based on the second sight line parameter and the second position information.

10. A tracking device, comprising:

the device comprises an acquisition module, a control module and a display module, wherein the acquisition module is used for acquiring position information of a first object relative to a to-be-controlled area and sight line parameters of the first object, and the to-be-controlled area at least comprises a display area used for outputting display content by electronic equipment;

the tracking module is used for inputting the position information and the sight line parameters into a target tracking model so as to determine the fixation position of the first object in the area to be controlled;

the target tracking model comprises a first tracking model obtained by processing an original tracking model based on change information of a sensor module and the display area, and the sensor module is used for acquiring the position information and/or the sight line parameters.

Images