CN114302054A - A photographing method of an AR device and an AR device thereof - Google Patents

Abstract

The present application relates to the technical field of augmented reality, and more particularly, the present application relates to a photographing method of an AR device and an AR device thereof. The AR device includes an eye-tracking camera and a photographing camera, and the photographing method includes: the eye-tracking camera takes a photo of the user's eyes; the AR device obtains the current gaze of the user's eyes according to the eye photo A focus area; controlling the photographing camera to shoot a current image according to the current gaze focus area, wherein the current image corresponds to the current gaze focus area. The present application omits the process of previewing through the screen, which saves the overhead caused by computing resources, power consumption, and memory, and improves the user experience.

Description

A photographing method of an AR device and an AR device thereof

technical field

The present application relates to the technical field of augmented reality, and more particularly, the present application relates to a photographing method of an AR device and an AR device thereof.

Background technique

In recent years, with the rise of Augmented Reality (AR) devices, various applications of virtual reality and augmented reality have gradually entered people's lives. In order to meet the various experience requirements of users, augmented reality devices usually set up cameras for taking pictures.

Most of the current AR equipment software and hardware systems follow the mobile phone platform. During the photographing process, the image captured by the camera is sent to the display system of the AR device so that the user can preview the image and perform the photographing action after the user previews the image. To achieve screen preview, the camera needs to transmit data to the CPU, and data transmission needs to occupy an image-sized memory in the CPU. The entire transmission process consumes memory and generates large power consumption.

SUMMARY OF THE INVENTION

Based on the above technical problems, the present invention aims to provide a new resource-saving photographing method, which controls the photographing camera to photograph the current front according to the user's current gaze focus area.

A first aspect of the present invention provides a photographing method for an AR device, the AR device includes an eye tracking camera and a photographing camera, and the photographing method includes:

the eye-tracking camera takes a photo of the user's eyes;

obtaining, by the AR device, the current focus area of the user's eye according to the eye photo;

The photographing camera is controlled to shoot a current image according to the current gaze focus area, wherein the current image corresponds to the current gaze focus area.

Specifically, the controlling the camera to shoot the current picture according to the current gaze focus area includes:

Determine the photographing range and photographing angle of the photographing camera according to the current gaze focus area;

The photographing camera is controlled to photograph according to the photographing range and photographing angle to obtain the current picture.

More specifically, the AR device obtains the current focus area of the user's eye according to the eye photo; including:

Determine a mapping function, the mapping function corresponds to the relationship between the rotation data of the eyeball and the line of sight angle;

Get the current rotation data;

The current gaze focus area is acquired according to the distance between the eyes, the current rotation data and the mapping function.

Further, the determining the mapping function includes:

Get the field of view;

Select the four corners and the center of the field of view as the first point, the second point, the third point, the fourth point and the center point respectively;

Marking the corresponding position in the eye tracking camera image as the origin when the eye is fixed on the center point;

Recording the number of pixels offset from the origin in the eye tracking camera image corresponding to the first point, the second point, the third point and the fourth point when the user looks at the first point, the second point, the third point and the fourth point respectively;

The correspondence between the offset pixel number and the eyeball rotation angle is calculated.

Further preferably, calculating the corresponding relationship between the offset pixel number and the eyeball rotation angle, including:

Obtain the relevant line segment and the included angle through the first point, the second point, the third point, the fourth point and the origin;

Denote points other than the first point, the second point, the third point, the fourth point and the origin as other fixation points;

Based on the relevant line segment and the included angle, the correspondence between the number of pixels shifted in the image of the eye tracking camera and the rotation angle of the eyeball is calculated when each eyeball looks at other fixation points.

Still further, calculating the correspondence between the number of pixels shifted in the image of the eye tracking camera and the rotation angle of the eyeball when each eyeball looks at other gazing points based on the relevant line segment and the included angle, including:

Obtaining the line segment from each eyeball of the user to the origin is recorded as the first line segment;

Denote the line segments from each eyeball of the user to the first point, the second point, the third point and the fourth point as the second line segment, the third line segment, the fourth line segment and the fifth line segment, respectively ;

respectively acquiring the included angles of the second line segment, the third line segment, the fourth line segment, the fifth line segment and the first line segment;

The first point, the second point, the third point and the fourth point correspond to the pixels offset from the origin in the eye tracking camera image according to the included angle and when the user looks at the first point, the second point, the third point and the fourth point Calculate the corresponding relationship between the number of pixels offset in the image of the eye tracking camera and the rotation angle of the eyeball when each eyeball looks at other fixation points.

A second aspect of the present invention provides an AR device, including an eye-tracking camera and a photographing camera; the eye-tracking camera is used to take a picture of a user's eyes; the AR device is used to obtain the eye-picture according to the eye-picture. The current gazing focus area of the user's eyes, and controlling the photographing camera to shoot the current picture according to the current gazing focus area.

Preferably, the AR device further includes a control module and a display module.

A third aspect of the present invention provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the photographing method of the AR device in each embodiment of the present invention.

A fourth aspect of the present invention provides a computer program product, including a computer program, which, when executed by a processor, implements the photographing method of the AR device in each embodiment of the present invention.

The beneficial effects of the present application are as follows: the present application controls the photographing camera to photograph the current front according to the user's current focus area, which is a new resource-saving photographing method. The eye tracking camera of the present application captures a photo of the user's eyes, the AR device obtains the current gazing focus area of the user's eyes according to the eye photo, and controls the photographing camera to shoot the current picture according to the current gazing focus area, The current picture corresponds to the current focus area, the picture preview process is omitted, the overhead caused by computing resources, power consumption and memory is saved, and the user experience is improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention.

Description of drawings

The accompanying drawings, which form a part of the specification, illustrate embodiments of the application and together with the description serve to explain the principles of the application.

The present application may be more clearly understood from the following detailed description with reference to the accompanying drawings, wherein:

FIG. 1 shows a schematic diagram of the method steps of an exemplary embodiment of the present application;

FIG. 2 shows a schematic diagram of a field of view of a calibration display module according to an exemplary embodiment of the present application;

FIG. 3 shows a schematic diagram of the focus area of human eye gaze in an exemplary embodiment of the present application;

FIG. 4 shows a schematic diagram of a saving operation process generated by the photographing method in an exemplary embodiment of the present application;

FIG. 5 shows a schematic structural diagram of an apparatus in an exemplary embodiment of the present application;

FIG. 6 shows a schematic structural diagram of an AR device provided by an exemplary embodiment of the present application;

FIG. 7 shows a schematic diagram of a storage medium provided by an exemplary embodiment of the present application.

Detailed ways

Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. It should be understood, however, that these descriptions are exemplary only, and are not intended to limit the scope of the application. Also, in the following description, descriptions of well-known structures and techniques are omitted to avoid unnecessarily obscuring the concepts of the present application. It will be apparent to those skilled in the art that the present application may be practiced without one or more of these details. In other instances, some technical features known in the art have not been described in order to avoid confusion with the present application.

It should be noted that the terminology used herein is for the purpose of describing specific embodiments only, and is not intended to limit the exemplary embodiments in accordance with the present application. As used herein, the singular forms are also intended to include the plural forms unless the context clearly dictates otherwise. Furthermore, it should also be understood that when the terms "comprising" and/or "comprising" are used in this specification, they indicate the presence of stated features, integers, steps, operations, elements and/or components, but do not exclude the presence or Addition of one or more other features, integers, steps, operations, elements, components and/or combinations thereof.

Now, exemplary embodiments according to the present application will be described in more detail with reference to the accompanying drawings. These exemplary embodiments may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. The drawings are not to scale, some details may be exaggerated and some details may be omitted for clarity. The shapes of the various regions and layers shown in the figures, as well as their relative sizes and positional relationships are only exemplary, and in practice, there may be deviations due to manufacturing tolerances or technical limitations, and those skilled in the art should Regions/layers with different shapes, sizes, relative positions can be additionally designed as desired.

Exemplary embodiments according to the present application are described below by giving several embodiments in conjunction with FIGS. 1-7 of the specification. It should be noted that the following application scenarios are only shown to facilitate understanding of the spirit and principles of the present application, and the embodiments of the present application are not limited in this respect. Rather, the embodiments of the present application can be applied to any scenario where applicable.

The current AR devices usually have the function of taking pictures, and users can take pictures after previewing the screen according to their needs. Specifically, the camera on the AR device captures the external image, the camera sends the external image to the display system of the AR device, and the display system presents the external image to the user in the form of a virtual image, and then the user triggers the photo-taking operation as needed.

An embodiment of the present application provides a method for photographing an AR device, wherein the AR device may be AR glasses or an AR headset, and the AR device is worn on the user's head during use. The AR device includes two types of cameras: eye-tracking cameras and photographing cameras. The eye-tracking cameras may include one or more cameras, which are used to take pictures of the user's eyes, and are usually located on the side close to the user's eyes; the photographing camera may also include One or more of them are used to take external photos, and are usually arranged on the side away from the user's eyes; in some embodiments, the eye tracking camera and the photographing camera are respectively located on opposite sides of the AR device.

Some embodiments of the present application provide a photographing method for an AR device. As shown in FIG. 1 , the photographing method includes:

First, eye-tracking cameras on AR devices take pictures of the user's eyes. For example, the left eye-tracking camera captures the left eye photo and the right eye-tracking camera captures the right eye photo; or the same eye-tracking camera captures both the left eye photo and the right eye photo.

Then, the AR device obtains the user's current gaze focus area according to the eye photo. The current gaze focus area is an area within the user's eye field of view, and the user's eyes are currently focused on this area. Information outside this area is used for It is not easy to perceive; the area is at a predetermined distance from the user's eyes, and the area is located on the side of the AR device away from the user's eyes.

Finally, the camera is controlled to shoot the current picture according to the current gaze focus area, wherein the current image corresponds to the current gaze focus area.

Usually, the photos taken by AR devices are outside scenes several meters or even tens of meters away, and the straight-line distance between the camera on the AR device and the user's eyes is generally only about 1cm. Coincident, that is, the field of view of the photographing camera is coincident with the field of view of the eye.

The current field of view of the camera can be determined according to the current focus area, and the camera can be controlled to take pictures within the current field of view.

In the photographing method provided by the embodiment of the present application, the current field of view of the photographing camera is determined according to the user's current gaze focus area obtained by the eye tracking camera, and a photograph is taken accordingly. In this solution, the camera does not need to send the preview image to the display system of the AR device, nor does the display system need to project the preview image into the user's eyes, which saves the preview process, not only saves memory space, but also eliminates the need for transmission of preview images. power consumption.

In some embodiments of the present application, when the AR device obtains the user's current focus area according to the eye photo, it may first determine the mapping function, then obtain the current rotation data, and finally obtain the current rotation data and the distance between the eyes (that is, the user's interpupillary distance) according to the current rotation data. And the mapping function obtains the current focus area, wherein the mapping function corresponds to the relationship between the rotation data of the eyeball (the rotation data may include rotation distance and/or rotation angle) and the line of sight angle.

In a specific embodiment, the AR device obtains the current focus area of the user's eye according to the eye photo; including: determining a mapping function, the mapping function corresponds to the relationship between the rotation data of the eyeball and the line of sight angle; obtaining the current rotation data; The current rotation data and the mapping function get the current gaze focus area. Further, determine the mapping function, including: obtaining the angle of view; selecting the four corners and the center of the angle of view as the first point, the second point, the third point, the fourth point and the center point respectively; when the eyeball is fixed on the center point The corresponding position in the eye-tracking camera image is marked as the origin; the first point, the second point, the third point and the fourth point corresponding to the pixels offset from the origin in the eye-tracking camera image are recorded separately when the user turns each eyeball. number; calculates the correspondence between the number of pixels shifted and the angle of eye rotation.

In a possible specific implementation, referring to FIG. 2 , the first point, the second point, the third point and the fourth point respectively correspond to the points B, C, D and E in FIG. 2 , and the center The point corresponds to point A in Figure 2. Here, the two eyeballs should be calibrated separately, the corresponding position in the eye tracking camera image when the eyeball is fixed on the center point is marked as the origin, the user's left eye is marked as _PL , and the right eye is marked as _PR , _PL and How far the P _R rotates corresponds to the number of pixels that the gaze point is offset from the origin, which can be marked by the eye tracking camera, and the angle between the extension line of the user's gaze and the center of the user's field of view (which can be the line of sight or the line of sight) is calculated. In addition, when calibrating, you need to make the eyes look far away, so that the rendered point is in the far distance, so the extension lines of the eyes of the two eyes are approximately parallel.

Specifically, calculating the corresponding relationship between the number of offset pixels and the rotation angle of the eyeball includes: obtaining the relevant line segment and the included angle through the first point, the second point, the third point, the fourth point and the origin; removing the The first point, the second point, the third point, the fourth point and the points other than the origin are recorded as other gaze points; The correspondence between the number of pixels offset in the image of the eye-tracking camera and the angle of eye rotation at other fixation points.

Further, calculating the correspondence between the number of pixels shifted in the eye tracking camera image and the rotation angle of the eyeball when each eyeball looks at other gazing points based on the relevant line segment and the included angle includes: obtaining each eyeball of the user; The line segment to the origin is recorded as the first line segment; the line segments from each eyeball of the user to the first point, the second point, the third point and the fourth point are respectively recorded as the second line segment, The third line segment, the fourth line segment and the fifth line segment; respectively obtain the included angles of the second line segment, the third line segment, the fourth line segment, the fifth line segment and the first line segment; The included angle and the first point, the second point, the third point and the fourth point when the user looks at the first point, the second point, the third point and the fourth point correspond to the number of pixels offset from the origin in the eye tracking camera image, Calculate the correspondence between the number of pixels offset in the image of the eye-tracking camera and the angle of eye rotation when each eye looks at other fixation points.

In a specific embodiment, (Δx, Δy) indicates that the first point, the second point, the third point and the fourth point when the user rotates each eyeball, corresponds to the offset from the origin in the eye-tracking camera image. The number of pixels, and the corresponding relationship between the number of pixels shifted in the eye tracking camera image and the rotation angle of the eyeball when each eyeball looks at other fixation points will form an f function, namely Δθ=f(Δx, Δy), according to Different eye tracking cameras will generate different f functions. The f function may be linear or nonlinear. Through different correspondences, other gaze points can be obtained corresponding to the number of pixels offset from the origin in the image of the eye tracking camera. Further, when the user's eyeball rotates and the gaze point is shifted from one point to another point, the specific offset pixel number is realized, so that the captured image accurately captures the user's eyes.

In a possible implementation manner, the relative positions of the fixation points of the eyes to the eyes may also be calculated according to the trigonometric function and the distance between the eyes, and the focus area of the fixation of the eyes may be calculated according to the trigonometric function and the distance between the eyes. In a possible specific implementation, referring to FIG. 3 , θ _L represents the angle between the line segment between the left eyeball and the fixation point and the first line segment, and θ _R represents the angle between the line segment between the right eyeball and the fixation point and the first line segment. angle.

In another possible implementation, the eye-tracking camera can also be used to calibrate the number of pixels offset in the image of the eye-tracking camera caused by the rotation of each eyeball of the user based on the field of view of the display module. The distance between the eyes of the user is determined, and as shown in FIG. 3 , the distance between the eyes of the user is L.

It should be noted that the existing camera, that is, the camera, needs to transmit data to the CPU at a specific frame rate, which is generally 30hz/60hz in the current mobile phone platform. At the same time, the data transmission needs to occupy an image-sized memory in the CPU. There will be a lot of resource (power consumption, memory) overhead. As shown in FIG. 4 , using the AR device photographing method described in this application can save the operation process. The user can directly see the outside world through the screen, so the consumption of system resources can be reduced. As shown in the lower part of Figure 4, the user's gaze point can be captured by the eye tracking camera (ie ET Camera), and the focus can be accurately completed to complete the photo.

The present application controls the photographing camera to photograph the current front according to the user's current gaze focus area, which is a new resource-saving photographing method. The eye tracking camera of the present application captures a photo of the user's eyes, the AR device obtains the current gaze focus area of the user's eyes according to the eye photo, and controls the camera to capture the current image according to the current gaze focus area, wherein the current image and the current gaze focus Area correspondence, omitting the process of screen preview and the process of manually selecting the focus area by the user, saving the overhead caused by computing resources, power consumption and memory, improving the user's experience, and enhancing the user's ability to take pictures. Immersion.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention.

In some exemplary embodiments, an AR device is also provided, as shown in FIG. 5 , including an eye-tracking camera 501 and a photographing camera 502; the eye-tracking camera is used to take pictures of the user's eyes; the AR device The device is configured to obtain the current gaze focus area of the user's eyes according to the eye photo, and control the photographing camera to shoot a current image according to the current gaze focus area, wherein the current image and the current gaze focus corresponding to the region.

Preferably, the AR device further includes a control module 503 and a display module 504, and the eye tracking camera 501, the photographing camera 502, the control module 503 and the display module 504 work together.

It should also be emphasized that the augmented reality system provided in the embodiments of the present application can acquire and process related data based on artificial intelligence technology. Among them, artificial intelligence (AI) is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. . The basic technologies of artificial intelligence generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation/interaction systems, and mechatronics. Artificial intelligence software technology mainly includes computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning/deep learning.

Please refer to FIG. 6 below, which shows a schematic diagram of an AR device provided by some embodiments of the present application. As shown in FIG. 6 , the AR device 2 includes: a processor 200 , a memory 201 , a bus 202 and a communication interface 203 , the processor 200 , the communication interface 203 and the memory 201 are connected through the bus 202 ; There is a computer program that can be run on the processor 200. When the processor 200 runs the computer program, the method for photographing an AR device provided by any of the foregoing embodiments of the present application is executed.

The memory 201 may include a high-speed random access memory (RAM: Random Access Memory), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory. The communication connection between the network element of the system and at least one other network element is realized through at least one communication interface 203 (which may be wired or wireless), which may use the Internet, a wide area network, a local network, a metropolitan area network, and the like.

The bus 202 may be an ISA bus, a PCI bus, an EISA bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, and the like. The memory 201 is used to store a program, and the processor 200 executes the program after receiving the execution instruction, and the method for photographing the AR device disclosed in any of the foregoing embodiments of the present application can be applied to the processor 200 , or implemented by the processor 200 .

The processor 200 may be an integrated circuit chip with signal processing capability. In the implementation process, each step of the above-mentioned method can be completed by an integrated logic circuit of hardware in the processor 200 or an instruction in the form of software. The above-mentioned processor 200 may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component. The methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in conjunction with the embodiments of the present application may be directly embodied as executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers and other storage media mature in the art. The storage medium is located in the memory 201, and the processor 200 reads the information in the memory 201, and completes the steps of the above method in combination with its hardware.

The AR device provided by the embodiment of the present application and the AR device photographing method provided by the embodiment of the present application are based on the same inventive concept, and have the same beneficial effects as the method adopted, operated or implemented.

Embodiments of the present application further provide a computer-readable storage medium corresponding to the AR device photographing method provided by the foregoing embodiments. Please refer to FIG. 7 . The computer-readable storage medium shown in FIG. 7 is an optical disc 30 on which storage There is a computer program (ie, a program product), when the computer program is run by the processor, the computer program will execute the photographing method of the AR device provided by any of the foregoing embodiments.

In addition, examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be described in detail here.

The computer-readable storage medium provided by the above-mentioned embodiments of the present application and the quantum key distribution channel allocation method in the space-division multiplexing optical network provided by the embodiments of the present application are based on the same inventive concept, and have the same inventive concept as the application program stored in the computer-readable storage medium. run or achieve the same beneficial effect.

Embodiments of the present application further provide a computer program product, including a computer program, which, when executed by a processor, implements the steps of the AR device photographing method provided by any of the foregoing embodiments.

It should be noted that the algorithms and displays provided herein are not inherently related to any particular computer, virtual appliance or other device. Various general-purpose devices can also be used with the teachings based on this. The structure required to construct such a device is apparent from the above description. Furthermore, this application is not directed to any particular programming language. It should be understood that the content of the application described herein can be implemented using a variety of programming languages and that the descriptions of specific languages above are intended to disclose the best mode of the application. In the description provided herein, numerous specific details are set forth. It will be understood, however, that the embodiments of the present application may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

Similarly, it is to be understood that in the above description of exemplary embodiments of the present application, various features of the present application are sometimes grouped together into a single embodiment, figure or its description. This disclosure, however, should not be interpreted as reflecting an intention that the claimed application requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of this application.

Those skilled in the art will understand that the modules in the device in the embodiment can be adaptively changed and arranged in one or more devices different from the embodiment. The modules or units or components in the embodiments may be combined into one module or unit or component, and further they may be divided into multiple sub-modules or sub-units or sub-assemblies. All features disclosed in this specification and all processes or elements of any method or apparatus so disclosed may be combined in any combination, except that at least some of such features and/or procedures or elements are mutually exclusive. Unless expressly stated otherwise, each feature disclosed in this specification may be replaced by alternative features serving the same, equivalent or similar purpose.

Various component embodiments of the present application may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that, in practice, a microprocessor or a digital signal processor (DSP) may be used to implement some or all functions of some or all components in the apparatus for creating a virtual machine according to the embodiments of the present application. The present application can also be implemented as an apparatus or apparatus program for performing part or all of the methods described herein. A program implementing the present application may be stored on a computer-readable medium, or may be in the form of one or more signals. Such signals may be downloaded from Internet sites, or provided on carrier signals, or in any other form.

The above description is only a preferred embodiment of the present application, but the protection scope of the present application is not limited to this. Substitutions should be covered within the protection scope of this application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (10)

1. A photographing method of an AR device is characterized in that the AR device comprises an eyeball tracking camera and a photographing camera, and the photographing method comprises the following steps:

the eyeball tracking camera shoots the eye picture of the user;

the AR equipment acquires the current watching focus area of the eyes of the user according to the eye photo;

and controlling the photographing camera to photograph a current picture according to the current watching focus area, wherein the current picture corresponds to the current watching focus area.

2. The photographing method according to claim 1, wherein the controlling the photographing camera to photograph a current picture according to the current gaze focus area comprises:

determining the photographing range and the photographing angle of the photographing camera according to the current watching focus area;

and controlling the camera to shoot according to the shooting range and the shooting angle so as to obtain the current picture.

3. The photographing method according to claim 1 or 2, wherein the AR device obtains a current gaze focus area of the user's eyes from the eye photograph; the method comprises the following steps:

determining a mapping function corresponding to the relationship between the rotation data of the eyeballs and the sight angle;

acquiring current rotation data;

and acquiring the current gazing focus area according to the distance between the two eyes, the current rotation data and the mapping function.

4. The photographing method according to claim 3, wherein the determining a mapping function includes:

acquiring a field angle;

selecting four corners and the center of the field angle as a first point, a second point, a third point, a fourth point and a center point respectively;

marking the corresponding position in the eyeball tracking camera image as the origin when the eyeball focuses on the central point;

respectively recording the pixel numbers of the first point, the second point, the third point and the fourth point which are respectively watched by the user when each eyeball rotates and are offset from the original point in the image of the eyeball tracking camera;

and calculating the corresponding relation between the deviated pixel number and the eyeball rotation angle.

5. The photographing method according to claim 4, wherein calculating the correspondence between the number of pixels of the offset and the eyeball rotation angle includes:

obtaining a relevant line segment and an included angle through the first point, the second point, the third point, the fourth point and the origin;

recording points other than the first point, the second point, the third point, the fourth point, and the origin as other gazing points;

and calculating the corresponding relation between the pixel number deviated in the eyeball tracking camera image and the eyeball rotation angle when each eyeball looks at other fixation points based on the related line segments and the included angles.

6. The photographing method according to claim 5, wherein the calculating of the correspondence between the number of pixels shifted in the image of the eye-tracking camera when each eye looks at other fixation points and the rotation angle of the eye based on the associated line segment and the included angle comprises:

acquiring a line segment from each eyeball of the user to the origin point and recording the line segment as a first line segment;

recording line segments from each eyeball of the user to the first point, the second point, the third point and the fourth point as a second line segment, a third line segment, a fourth line segment and a fifth line segment respectively;

respectively acquiring included angles between the second line segment, the third line segment, the fourth line segment and the fifth line segment and the first line segment;

and calculating the corresponding relation between the number of pixels of each eyeball which are offset in the image of the eyeball tracking camera when the eyeball looks at other fixation points and the eyeball rotation angle according to the included angle and the number of pixels of each eyeball which are offset from the original point in the image of the eyeball tracking camera when the user looks at each eyeball.

7. An AR device comprising an eye tracking camera and a camera; the eyeball tracking camera is used for shooting an eye picture of a user;

the AR equipment is used for acquiring a current watching focus area of the eyes of the user according to the eye picture and controlling the photographing camera to photograph a current picture according to the current watching focus area, wherein the current picture corresponds to the current watching focus area.

8. The AR apparatus of claim 7, further comprising a control module and a display module.

9. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that the computer program realizes the steps of the method of any one of claims 1 to 6 when executed by a processor.

Images