CN114910052A

CN114910052A - Camera-based distance measurement method, control method and device and electronic equipment

Info

Publication number: CN114910052A
Application number: CN202210587132.3A
Authority: CN
Inventors: 杨亚军; 游城; 魏学华; 杨克庆
Original assignee: Shenzhen Stereo Technology Co ltd
Current assignee: Shenzhen Stereo Technology Co ltd
Priority date: 2022-05-27
Filing date: 2022-05-27
Publication date: 2022-08-16
Anticipated expiration: 2042-05-27
Also published as: CN114910052B

Abstract

The embodiment of the invention discloses a distance measuring method based on a camera, a control method, a device and electronic equipment, wherein the method can be applied to a terminal, the terminal is provided with a display screen and a front camera, pupil recognition is carried out on a face image by acquiring the face image of a user, the face image is obtained by shooting by the front camera, first pupil position information and second pupil position information of eyes of the user are determined, the picture interpupillary distance of the user in the face image is obtained by calculation according to the first pupil position information and the second pupil position information, finally the watching distance from the eyes of the user to the display screen is obtained by calculation according to the picture interpupillary distance, the distance measuring can be realized by the front camera arranged on the terminal, the picture position of the user is obtained by analyzing the image obtained by the front camera, and the required picture interpupillary distance is calculated, and finally, the viewing distance from the eyes of the user to the display screen is obtained, and the distance measurement cost is low.

Description

Camera-based distance measurement method, control method and device and electronic equipment

Technical Field

The invention relates to the technical field of distance measurement, in particular to a distance measurement method, a distance measurement control method, a distance measurement device and electronic equipment based on a camera.

Background

At present, intelligent terminals such as mobile phones and tablet computers gradually enter the lives of people, how to accurately measure the distance between a user and the terminal is the research direction of the industry, and the measured accurate distance between the user and the terminal can provide more intelligent operation for the terminal.

In the related art, the terminal generally adopts the mode of setting up infrared sensor to get and terminal user's distance between, but infrared sensor can only simply measure the distance that obtains between user's face and the terminal, can not measure the distance that obtains between user's eyes and the terminal, and this needs additionally to set up an accurate infrared sensor on the terminal and measures, can occupy the inner space at terminal, improves the range finding cost.

Disclosure of Invention

The embodiment of the invention provides a camera-based distance measurement method, a software control method and a terminal, which can obtain the watching distance between the eyes of a user and the terminal through camera measurement and have low distance measurement cost.

In a first aspect, an embodiment of the present invention provides a camera-based distance measurement method, which is applied to a terminal, where the terminal is provided with a display screen and a front camera, and the method includes: acquiring a face image of a user, wherein the face image is obtained by shooting through the front-facing camera; performing pupil identification on the face image, and determining first pupil position information and second pupil position information of the eyes of the user; calculating the picture interpupillary distance of the user in the face image according to the first pupil position information and the second pupil position information; and calculating the watching distance from the eyes of the user to the display screen according to the picture interpupillary distance.

In some embodiments, the performing pupil recognition on the facial image and determining first pupil position information and second pupil position information of the user's eyes includes: converting the face image into a gray level image, and carrying out binarization processing on the gray level image to obtain a first preprocessed image; carrying out corrosion and expansion treatment on the first preprocessed image, and eliminating noise in the image to obtain a second preprocessed image; extracting the position of a circular area which represents the pupil of the user in the second preprocessed image by using a circular structural element; and calculating the central point of the circular area to obtain first pupil position information and second pupil position information of the two eyes of the user.

In some embodiments, the calculating, according to the picture interpupillary distance, a viewing distance from an eye of the user to the display screen includes: acquiring a preset standard interpupillary distance; acquiring the focal length of the face image shot by the front camera, and obtaining the initial distance of the face image to an imaging point according to the focal length; and obtaining a first proportion according to the picture interpupillary distance and the standard interpupillary distance, and obtaining the viewing distance from the eyes of the user to the display screen according to the first proportion and the initial distance.

In some embodiments, said deriving a viewing distance from the user's eyes to the display screen according to the first scale and the initial distance comprises: obtaining a first distance from the user's eyes to the imaging point according to the first proportion and the initial distance; and calculating a difference value of the first distance and the initial distance, and taking the difference value as the viewing distance from the eyes of the user to the display screen.

In some embodiments, the obtaining the viewing distance from the user's eyes to the display screen according to the picture interpupillary distance includes: calculating the picture face space of the user in the face image, and obtaining a picture face proportion according to the picture face space; acquiring a preset standard face proportion; obtaining the rotation angle of the face of the user according to the picture face proportion and the standard face proportion; and calculating the viewing distance from the eyes of the user to the display screen according to the rotation angle and the picture interpupillary distance.

In some embodiments, the calculating a viewing distance from the user's eyes to the display screen according to the rotation angle and the picture interpupillary distance includes: acquiring a preset standard interpupillary distance and a focal length of the face image shot by the front camera, and obtaining an initial distance from the face image to an imaging point according to the focal length; obtaining a first distance between the eyes of the user on a target plane according to the standard pupillary distance, the picture pupillary distance, the initial distance and the rotation angle; obtaining a second proportion according to the picture interpupillary distance and the first distance, and obtaining a second distance from the target plane to the display screen according to the second proportion and the initial distance; obtaining a third distance from the eyes of the user to the target plane according to the standard interpupillary distance and the rotation angle; and obtaining the viewing distance from the eyes of the user to the display screen according to the sum of the second distance and the third distance.

In some embodiments, the obtaining a first distance between the eyes of the user on the target plane according to the standard interpupillary distance, the frame interpupillary distance, the initial distance, and the rotation angle includes: obtaining a portrait angle on the imaging point according to the picture pupil distance and the initial distance; obtaining a target angle according to the portrait angle and the rotation angle, wherein the target angle is an angle between the user eyes at the far end side of the imaging point and the user eyes at the near end side of the imaging point; and obtaining a first distance between the eyes of the user on the target plane according to the target angle, the rotation angle and the standard interpupillary distance.

In some embodiments, said deriving a second distance from the target plane to the display screen according to the second ratio and the initial distance includes: obtaining a fourth distance corresponding to the imaging point by the target plane according to the second proportion and the initial distance; and calculating a difference value between the fourth distance and the initial distance, and taking the difference value as a second distance from the target plane to the display screen.

In some embodiments, the calculating, according to the picture interpupillary distance, a viewing distance from an eye of the user to the display screen includes: acquiring a preset distance lookup table; and looking up a table from the distance lookup table according to the picture interpupillary distance to obtain the viewing distance from the eyes of the user to the display screen.

In some embodiments, the distance lookup table is derived according to the following steps: obtaining a sample interpupillary distance in a sample; obtaining the distance from the eyes of the user to the display screen in the sample to obtain the watching distance of the sample; and establishing the distance lookup table based on the interpupillary distance to the viewing distance according to the sample interpupillary distance and the sample viewing distance.

In some embodiments, the calculating, according to the picture interpupillary distance, a viewing distance from an eye of the user to the display screen includes: acquiring a reference distance, a reference object size and a picture size corresponding to the reference object shot by the front camera; acquiring a preset standard interpupillary distance; and obtaining the viewing distance from the eyes of the user to the display screen according to the reference distance, the reference object size, the picture interpupillary distance and the standard interpupillary distance.

In some embodiments, the obtaining the viewing distance from the user's eyes to the display screen according to the reference distance, the reference object size, the picture interpupillary distance, and the standard interpupillary distance includes: dividing the standard interpupillary distance by the picture interpupillary distance to obtain a first coefficient; obtaining a second coefficient according to the size of the reference object in the picture size; dividing the reference distance by the second coefficient to obtain a third coefficient; and obtaining the viewing distance from the eyes of the user to the display screen according to the product of the third coefficient and the first coefficient.

In some embodiments, the performing pupil recognition on the facial image and determining first pupil position information and second pupil position information of the user's eyes includes: performing pupil identification on the face image, determining a plurality of different users in the image, and respectively obtaining first pupil position information and second pupil position information of eyes of each user; calculating the viewing distance from the eyes of the user to the display screen according to the picture interpupillary distance, wherein the method comprises the following steps: calculating to obtain the watching distance from each target user eye to the display screen according to the picture interpupillary distances of a plurality of different users; and calculating the picture interpupillary distance of different users according to the first pupil position information and the second pupil position information of each user.

In some embodiments, the front camera is a sub-screen camera, and the sub-screen camera is disposed in a center of the display screen.

In a second aspect, an embodiment of the present invention further provides a control method implemented by a camera-based distance measurement method, where the control method is applied to a terminal, where the terminal is provided with a display screen and a front camera, and the method includes: acquiring a face image of a user, wherein the face image is obtained by the front camera; performing pupil identification on the face image, and determining first pupil position information and second pupil position information of the eyes of the user; calculating the picture interpupillary distance of the user in the face image according to the first pupil position information and the second pupil position information; calculating the watching distance from the eyes of the user to the display screen according to the picture interpupillary distance; and executing a software control strategy according to the viewing distance.

In some embodiments, the terminal displays a 3D picture through the display screen, and the executing a software control policy according to the viewing distance includes: acquiring a recommended distance range for 3D picture watching; comparing and analyzing the recommended distance and the watching distance to obtain a distance comparison result; and when the distance comparison result indicates that the user is not in the recommended distance range, sending distance prompt information.

In a third aspect, an embodiment of the present invention further provides a distance measuring method based on a camera, where the distance measuring method is applied to a terminal, where the terminal is provided with a camera, and the method includes: acquiring a face image of a user, wherein the face image is shot by the camera; performing pupil identification on the face image, and determining first pupil position information and second pupil position information of the eyes of the user; calculating the picture interpupillary distance of the user in the face image according to the first pupil position information and the second pupil position information; and calculating the distance from the eyes of the user to the terminal according to the picture interpupillary distance.

In a fourth aspect, an embodiment of the present invention further provides a distance measuring device based on a camera, including: the device comprises a first module, a second module and a third module, wherein the first module is used for acquiring a face image of a user; the second module is used for carrying out pupil identification on the face image and determining first pupil position information and second pupil position information of the eyes of the user; the image processing device is used for calculating the picture inter-pupillary distance of the user in the face image according to the first pupil position information and the second pupil position information; and calculating the watching distance from the eyes of the user to the display screen according to the picture interpupillary distance.

In a fifth aspect, an embodiment of the present invention further provides an electronic device, which includes a memory and a processor, where the memory stores a computer program, and the processor implements the camera-based ranging method according to any one of the embodiments of the first aspect of the present invention or the control method implemented by the camera-based ranging method according to any one of the embodiments of the second aspect of the present invention when executing the computer program.

In a sixth aspect, the present invention further provides a computer-readable storage medium, where the storage medium stores a program, and the program is executed by a processor to implement the camera-based ranging method according to any one of the first aspect of the present invention or the control method implemented by the camera-based ranging method according to any one of the second aspect of the present invention.

The embodiment of the invention at least comprises the following beneficial effects: the embodiment of the invention provides a camera-based distance measurement method, a control method, a device and electronic equipment, wherein the camera-based distance measurement method can be applied to a terminal, the terminal is provided with a display screen and a front camera, the face image of a user is obtained by shooting through the front camera, pupil recognition is carried out on the face image, first pupil position information and second pupil position information of eyes of the user are determined, the picture inter-pupillary distance of the user in the face image is calculated according to the first pupil position information and the second pupil position information, finally the watching distance from the eyes of the user to the display screen is calculated according to the picture inter-pupillary distance, the distance measurement can be realized through the front camera arranged on the terminal, the pupil position of the user is obtained through image analysis obtained through the front camera, and the required picture inter-pupillary distance is calculated, the picture interpupillary distance represents the interpupillary distance of the user in the acquired image, the watching distance from the eyes of the user to the display screen can be calculated according to the picture interpupillary distance, the distance measurement cost is low, and the distance measurement can be realized without additionally arranging other sensors.

Drawings

Fig. 1 is a schematic diagram of a terminal provided by an embodiment of the present invention;

fig. 2 is a schematic flowchart of a camera-based distance measuring method according to an embodiment of the present invention;

FIG. 3a is a schematic diagram of a face image provided by one embodiment of the present invention;

FIG. 3b is a schematic diagram of a face image provided by another embodiment of the present invention;

fig. 4 is a schematic flowchart of a camera-based distance measuring method according to another embodiment of the present invention;

FIG. 5 is a schematic diagram of processing a face image to obtain a pupil position according to an embodiment of the present invention;

fig. 6 is a schematic flowchart of a camera-based distance measuring method according to another embodiment of the present invention;

FIG. 7 is a schematic diagram of relative lens (imaging point) imaging provided by one embodiment of the present invention;

FIG. 8 is a schematic diagram of the calculation of viewing distance by the triangle principle provided by one embodiment of the present invention;

fig. 9 is a schematic flowchart of a camera-based distance measuring method according to another embodiment of the present invention;

fig. 10 is a schematic flowchart of a camera-based distance measuring method according to another embodiment of the present invention;

fig. 11 is a schematic flowchart of a camera-based distance measuring method according to another embodiment of the present invention;

fig. 12 is a schematic flowchart of a camera-based distance measuring method according to another embodiment of the present invention;

FIG. 13 is a schematic diagram of the calculation of viewing distance by the triangle principle according to another embodiment of the present invention;

fig. 14 is a schematic flowchart of a camera-based distance measuring method according to another embodiment of the present invention;

fig. 15 is a schematic flowchart of a camera-based distance measuring method according to another embodiment of the present invention;

fig. 16 is a schematic flowchart of a camera-based distance measuring method according to another embodiment of the present invention;

fig. 17 is a schematic flowchart of a camera-based distance measuring method according to another embodiment of the present invention;

fig. 18 is a schematic flowchart of a camera-based distance measuring method according to another embodiment of the present invention;

FIG. 19 is a schematic diagram of finding a viewing distance from a reference frame according to one embodiment of the present invention;

FIG. 20 is a schematic diagram of viewing distance from a reference frame according to another embodiment of the present invention;

fig. 21 is a schematic flowchart of a control method implemented by the camera-based distance measuring method according to an embodiment of the present invention;

fig. 22 is a schematic flowchart of a control method implemented by a camera-based distance measuring method according to another embodiment of the present invention;

fig. 23 is a schematic flowchart of a camera-based distance measuring method according to another embodiment of the present invention;

FIG. 24 is a schematic structural diagram of a camera-based distance measuring device according to an embodiment of the present invention;

fig. 25 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

It should be understood that in the description of the embodiments of the present invention, a plurality (or a plurality) means two or more, more than, less than, more than, etc. are understood as excluding the number, and more than, less than, etc. are understood as including the number. If the description of "first", "second", etc. is used for the purpose of distinguishing technical features, it is not intended to indicate or imply relative importance or to implicitly indicate the number of indicated technical features or to implicitly indicate the precedence of the indicated technical features.

At present, intelligent terminals such as mobile phones and tablet computers gradually walk into the lives of people, how to go to the accurate measurement to obtain the distance between a user and a terminal is the research direction of the industry, the accurate distance between the user and the terminal is obtained, more intelligent operation can be provided for the terminal, when the user watches a terminal display screen, the user can watch the screen by eyes and can watch other places, and how to carry out further intelligent control according to the watching condition of the user becomes the key point of research of people.

In the related art, the terminal generally adopts a mode of arranging an infrared sensor to acquire the distance between the terminal and a terminal user, but the infrared sensor can only simply measure the distance between the face of the user and the terminal and cannot measure the distance between the eyes of the user and the terminal, so that further intelligent control cannot be performed based on the watching condition of the user.

However, in the current terminal, a camera is generally arranged, for example, a front camera and a rear camera are arranged on terminal devices such as a mobile phone and a tablet computer, the front camera is generally used for taking pictures and is arranged on a front panel together with a display screen, on this basis, if the terminal in the prior art needs to measure the distance between a user and the terminal, an infrared sensor needs to be additionally arranged, the occupied area of the device on the front panel of the terminal (such as a mobile phone) is increased, and the hardware cost is increased.

Based on this, embodiments of the present invention provide a camera-based distance measurement method, a control method, an apparatus, and an electronic device, which can implement distance measurement through a front camera disposed on a terminal, analyze an image acquired by the front camera to obtain a pupil position of a user, and calculate a required image inter-pupillary distance, where the image inter pupillary distance represents an inter pupillary distance of the user in the acquired image, and can calculate a viewing distance from eyes of the user to a display screen according to the image inter pupillary distance, and the distance measurement cost is low, and the distance measurement can be implemented without additionally disposing other sensors.

The terminal in the embodiment of the invention can be mobile terminal equipment and can also be non-mobile terminal equipment. The mobile terminal equipment can be a mobile phone, a tablet computer, a notebook computer, a palm computer, vehicle-mounted terminal equipment, wearable equipment, a super mobile personal computer, a netbook, a personal digital assistant and the like; the non-mobile terminal equipment can be a personal computer, a television, a teller machine or a self-service machine and the like; the embodiments of the present invention are not particularly limited.

The terminal may include a processor, an external memory interface, an internal memory, a Universal Serial Bus (USB) interface, a charging management module, a power management module, a battery, a mobile communication module, a wireless communication module, an audio module, a speaker, a receiver, a microphone, an earphone interface, a sensor module, a button, a motor, an indicator, a front camera, a rear camera, a display screen, and a Subscriber Identity Module (SIM) card interface, etc. The terminal can realize shooting functions through a front camera, a rear camera, a video coder-decoder, a GPU, a display screen, an application processor and the like.

The front camera or the rear camera is used for capturing still images or videos. The object generates an optical image through the lens and projects the optical image to the photosensitive element. The photosensitive element may be a Charge Coupled Device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to an ISP (image signal processor) to be converted into a digital image signal. And the ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into image signal in standard RGB, YUV and other formats. In some embodiments, the terminal may include 1 or N front-facing cameras, where N is a positive integer greater than 1.

The terminal realizes the display function through the GPU, the display screen, the application processor and the like. The GPU is a microprocessor for image processing and is connected with a display screen and an application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor may include one or more GPUs that execute program instructions to generate or alter display information.

The display screen is used to display images, videos, and the like. The display screen includes a display panel. The display panel may be a Liquid Crystal Display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (active-matrix organic light-emitting diode, AMOLED), a flexible light-emitting diode (FLED), a miniature, a Micro-oeld, a quantum dot light-emitting diode (QLED), or the like.

In an embodiment, the display screen may be a naked eye 3D display screen, the multimedia data may be processed by the naked eye 3D display screen, and the multimedia data is split into a left part and a right part, for example, the 2D video is cut into two parts, and light-emitting refraction directions of the two parts are changed, after being viewed by eyes of a user, a 3D picture may be seen, the formed 3D picture has a negative parallax, and may be displayed between the user and the display screen, so as to achieve a naked eye 3D viewing effect, or the display screen may be a 2D display screen, the terminal may be externally provided with a 3D grating film, and refracts light emitted from the 2D display screen, so that the user views a 3D display effect after viewing the display screen through the 3D grating film.

For example, as shown in fig. 1, in the embodiment of the present invention, a terminal is taken as an example of a mobile phone, for example, a front panel 10 on the mobile phone is provided with a front camera 11 to acquire image information, and the front panel 10 is further provided with a display screen 12 to perform screen display, and it is understood that when the terminal is a 3D vision training terminal or a mobile phone capable of displaying a 3D screen, a 3D screen may be displayed through the display screen 12.

It should be noted that, in the embodiment of the present invention, the front camera 11 may be disposed on the same plane as the disposed display screen 12, and the position of the front camera 11 is fixed, in an embodiment, the front camera 11 may be perpendicular to the display screen 12 or not, and the front camera 11 may be located within the display screen 12 or located at the periphery of the display screen 12, as shown in fig. 1, in addition, the front camera 11 may also have a perpendicular distance from the display screen 12, so that the front camera 11 and the display screen 12 are not located on the same plane, and the terminal may perform parameter verification according to the front cameras 11 disposed differently, so as to implement the camera-based distance measuring method and the control method in the embodiment of the present invention.

In the following, a camera-based distance measurement method, a control method, a device and an electronic apparatus in an embodiment of the present invention are described, and first, the camera-based distance measurement method in the embodiment of the present invention is described.

Referring to fig. 2, an embodiment of the present invention provides a camera-based ranging method, which may be applied in a terminal, and the ranging method may include, but is not limited to, steps S101 to S104.

In step S101, a face image of the user is acquired, and the face image is captured by the front camera.

Step S102, performing pupil identification on the face image, and determining first pupil position information and second pupil position information of the eyes of the user.

And step S103, calculating the picture interpupillary distance of the user in the face image according to the first pupil position information and the second pupil position information.

And step S104, calculating the watching distance from the eyes of the user to the display screen according to the picture interpupillary distance.

It should be noted that, in the distance measuring method based on a camera in the embodiment of the present invention, a face image of a user is obtained by capturing the face image by a front-facing camera, pupil recognition is performed on the face image, first pupil position information and second pupil position information of eyes of the user are determined, a picture inter-pupillary distance of the user in the face image is calculated according to the first pupil position information and the second pupil position information of eyes of the user, which can be determined according to the number of pixels of a display screen, and finally, a viewing distance from the eyes of the user to the display screen is calculated according to the picture inter-pupillary distance, in the embodiment of the present invention, distance measurement can be implemented by the front-facing camera arranged on a terminal, a pupil position of the user is obtained by analyzing an image obtained by the front-facing camera, and a required picture inter-pupillary distance is calculated, the picture inter-pupillary distance is an inter pupillary distance representing the user in the obtained image, and the watching distance from the eyes of the user to the display screen can be calculated according to the interpupillary distance of the picture, the distance measurement cost is low, and the distance measurement can be realized without additionally arranging other sensors.

It can be understood that the terminal obtains the face image through the front camera, the inter-pupillary distance of the user on the face image is the inter-pupillary distance of the picture, when the front camera is used as a reference object in the shooting process, the position of the user for the camera can be changed at any time, through the imaging of the camera, the user can make different image sizes at different distances, as shown in figure 3a and figure 3b, in fig. 3a, the user is closer to the camera, and the size of the user's face is larger in the resulting image, so the interpupillary distance is larger, in fig. 3b, the user is far from the camera, the size of the user's face in the image is smaller, therefore, the interpupillary distance is smaller, and the watching distance from the eyes of the user to the display screen can be judged according to the interpupillary distance in the user image shot by the front camera.

For example, a reference object of known dimensions may be used to measure at a known distance and the viewing distance calculated according to a predetermined formula. For example, there is a 10cm reference object (such as a ruler) 50cm in front of the display screen, according to the parameter characteristics of the front camera, the shot picture is taken, the 10cm object becomes a certain size (can be determined according to the number of pixels) in the image obtained by the front camera, now it is known that the target object (i.e. two pupils) 6.3cm in the formed image, the size in the picture is determined, and therefore the distance from the target object to the display screen can be calculated.

It should be noted that the inter-pupillary distance is a distance between pupils of both eyes of a user, which may also be referred to as a pupillary distance for short, and refers to a length between centers of both pupils, a normal range of the inter-pupillary distance of an adult is between 58-64mm, and the inter-pupillary distance itself is determined by heredity and development of an individual, so that inter-pupillary distances of different ages are different, and for a certain user, the inter-pupillary distance is constant, so that the distance from the user to the terminal can be determined according to the size of the inter-pupillary distance of a picture in a face image, and the viewing distance from the eyes of the user to the display screen can be calculated.

It should be noted that, in the embodiment of the present invention, the image of the user is recognized by the front-facing camera, and the distance measurement can be implemented without setting an additional sensor device, so that the design cost is low, and no additional hardware setting is required, and the method can be applied to a terminal with the front-facing camera.

It should be noted that, in the embodiment of the present invention, the face image may be obtained by directly recognizing the face of the user by the front camera, and in an embodiment, the face image is obtained by cutting the image obtained by the front camera, for example, the terminal obtains the image by the front camera, and the image includes the face of the user and may also include some other impurities, which may interfere with pupil recognition.

In an embodiment, the front-facing camera is a camera under the screen, and the display screen is an OLED screen, so that the front-facing camera can be disposed under the display screen, specifically, the camera under the screen is disposed at a center position of the display screen, and the image of the face of the user is acquired at the center position of the display screen, so that the inter-pupillary distance of the user can be measured more accurately, and the distance measurement with higher accuracy is realized.

Referring to fig. 4, in an embodiment, the step S102 may further include, but is not limited to, step S201 to step S204.

Step S201, converting the face image into a grayscale image, and performing binarization processing on the grayscale image to obtain a first preprocessed image.

And step S202, carrying out corrosion and expansion treatment on the first preprocessed image, and eliminating noise in the image to obtain a second preprocessed image.

In step S203, the position of the circular region representing the pupil of the user in the second preprocessed image is extracted by using the circular structural element.

Step S204, calculating the central point of the circular area to obtain the first pupil position information and the second pupil position information of the two eyes of the user.

It should be noted that, in the embodiment of the present invention, image processing is performed on a face image to obtain pupil position information of a user's face, specifically, as shown in fig. 5, first, the face image is converted into a gray-scale image, and binarization processing is performed on the gray-scale image to obtain a first pre-processed image, and as for an eyeball in the first pre-processed image after binarization processing, a circular structural element is used to open the image, the image is first subjected to erosion and expansion processing, after erosion and expansion, a central circular region has noise, which needs to be removed to obtain a second pre-processed image, and finally, a position of the circular region representing the user's pupil in the second pre-processed image is extracted according to the circular structural element, and the circular region is identified in the whole face image, therefore, the central points of the left and right eye circular areas of the user can be calculated, and the first pupil position information and the second pupil position information of the two eyes of the user can be obtained.

It can be understood that, in the embodiment of the present invention, the image inter-pupillary distance can be calculated after the first pupil position information and the second pupil position information are obtained, for example, in an embodiment, both the obtained first pupil position information and the obtained second pupil position information are coordinate information, and the image inter-pupillary distance can be obtained by performing calculation according to the two coordinate information, where the image inter pupillary distance is an inter pupillary distance in a user face image shot by a front camera and is not a user inter pupillary distance in display, and a viewing distance from a user eye to a display screen can be calculated according to a size of the image inter pupillary distance.

Referring to fig. 6, in an embodiment, the step S104 may further include, but is not limited to, step S301 to step S303.

Step S301, acquiring a preset standard interpupillary distance.

Step S302, acquiring the focal length of the face image shot by the front camera, and obtaining the initial distance of the face image to the imaging point according to the focal length.

Step S303, obtaining a first proportion according to the picture interpupillary distance and the standard interpupillary distance, and obtaining the watching distance from the eyes of the user to the display screen according to the first proportion and the initial distance.

It should be noted that, in obtaining the viewing distance from the user's eyes to the display screen by calculating the inter-pupillary distance of the picture, specifically, in the embodiment of the present invention, a preset standard inter-pupillary distance is obtained first, where the standard inter-pupillary distance is an actual inter-pupillary distance of the user, and the standard inter-pupillary distance may be set by default, for example, to 63mm, or the standard inter-pupillary distance may be input by the user, so that the user may accurately input the inter-pupillary distance, or through big data and artificial intelligence analysis, the inter-pupillary distances of people of different ages and different genders are different, and the data analysis conclusion is substituted for the inter-pupillary distance of an adult of 63mm, so that a more accurate inter-pupillary distance may be obtained, and a more accurate viewing distance may be obtained. And then, acquiring the focal length of the face image shot by the front camera, obtaining the initial distance of the face image corresponding to an imaging point according to the focal length, finally obtaining a first proportion according to the picture interpupillary distance and the standard interpupillary distance, and obtaining the viewing distance from the eyes of the user to the display screen according to the first proportion and the initial distance.

It can be understood that each camera has a certain field of view (FOV) and a certain focal length when shooting, the focal length and the FOV of each camera are in one-to-one correspondence, and can be obtained through a public manner or can be measured, the FOV is an included angle between two ends of a camera cone, and the focal length is a distance from a lens of the camera to an internal "sensor", however, in practice, the camera and the sensor are behind the lens, and for simplification, it can be assumed that the lens is in front of the sensor and is mirrored with respect to the lens, and a picture shown in fig. 7 can be obtained, for example, a plane where the sensor is located is a plane where a face image is located, the formed face image is equivalent to being above the plane where the lens is located, the position where the lens is located can be described as an imaging point in the embodiment of the present invention, the plane where the imaging point is located is below the plane where the face image is located, and the plane of the face image is parallel to the plane of the imaging point, so that the position of the plane of the face image relative to the plane of the imaging point can be obtained according to the focal distance.

It can be understood that the plane where the face image is located corresponds to the plane where the display screen is located, and is determined according to the wide angle and the focal length of the front camera, in an embodiment, the plane where the face image is located is the plane where the display screen is located, or the plane where the face image is located can be obtained by subtracting a small distance from the plane where the face image is located, which can be obtained by calculating in advance according to the physical parameters of the front camera and applied to subsequent processing.

It should be added that, in the embodiment of the present invention, the initial distance is obtained according to the focal length, and the initial distance may also be obtained by obtaining the angle of view of shooting, but since the angle of view and the focal length are in a one-to-one correspondence relationship, taking the example of obtaining the focal length to perform processing, it should be noted that the initial distance may be obtained by calculation according to the characteristics of the camera imaging, or may be obtained by measurement in advance, but it is understood that each different focal length corresponds to one initial distance, and no specific limitation is made herein.

It can be understood that, as shown in fig. 8, according to the imaging characteristics of the camera, a triangle is formed between the line segment of the actual inter-pupillary distance of the user and the imaging point, and the line segment of the picture inter-pupillary distance is located in the triangle and is parallel to the line segment of the actual inter-pupillary distance.

Referring to fig. 9, in an embodiment, the step S303 may further include, but is not limited to, step S401 to step S402.

Step S401, a first distance from the user' S eye to the imaging point is obtained according to the first proportion and the initial distance.

Step S402, calculating the difference value between the first distance and the initial distance, and taking the difference value as the viewing distance from the eyes of the user to the display screen.

It should be noted that, in the process of obtaining the viewing distance from the eyes of the user to the display screen, calculation is performed according to the characteristics of the triangle, in an embodiment, a triangle between the line segment where the picture interpupillary distance is located and the imaging point in fig. 8 is defined as a first triangle, a triangle between the line segment where the actual interpupillary distance is located and the imaging point in fig. 8 is defined as a second triangle, and the first triangle and the second triangle are similar triangles.

As shown in fig. 8, the first distance from the user's eye to the imaging point can be obtained according to a first ratio and an initial distance, the first ratio is the picture interpupillary distance Q divided by the actual interpupillary distance K, then the initial distance H0 is divided by the first ratio to obtain a first distance H1, and finally the first distance H1 is subtracted by the initial distance H0 to obtain a viewing distance H from the user's eye to the display screen, wherein the calculation formula of H is as follows:

H＝H1-H0 (1)

H1＝H0/(Q/K) (2)

referring to fig. 10, in an embodiment, the step S104 may further include, but is not limited to, the steps S501 to S504.

Step S501, calculating the distance between the faces of the pictures of the user in the face image, and obtaining the proportion of the faces of the pictures according to the distance between the faces of the pictures.

Step S502, acquiring a preset standard face proportion.

Step S503, obtaining the rotation angle of the user face according to the picture face proportion and the standard face proportion.

And step S504, calculating the watching distance from the eyes of the user to the display screen according to the rotation angle and the picture interpupillary distance.

It should be noted that, during the calculation of the viewing distance from the eyes of the user to the display screen according to the inter-pupillary distance of the image, correction can be performed, specifically, when the user views the display screen, the user may view the display screen at a certain angle, for the front camera, the acquired image is a two-dimensional planar image, the angle of rotation of the face of the user cannot be distinguished simply according to the image, if the inter-pupillary distance is directly calculated at this time, an error may be caused, and therefore, distance measurement is not accurate, and therefore, geometric calculation may be performed according to the specific position of the front camera to correct the parameter. In addition, when the pupil is not right in front of the front camera, correction is also carried out through the geometrical principle, and when the display screen and the front camera are not on the same plane, correction can also be carried out according to the distance difference.

Based on this, the embodiment of the invention corrects the rotation angle of the user to obtain the accurate viewing distance. The embodiment of the invention firstly calculates the picture face space of the user in the face image, obtains the picture face proportion according to the picture face space, and then obtains the preset standard face proportion, wherein the standard face proportion is the face proportion of the user in reality, namely the actual face proportion, the standard face proportion can be set by default or input by the user, and is not repeated herein, it can be understood that the embodiment of the invention can calculate the length and the width of the user face in the face image to obtain the picture face proportion, when the user face is over against the front camera, the ratio of the width to the length of the user face is the largest, namely the ratio is the largest, and along with the rotation of the user face, the ratio of the width to the length of the user face in the face image obtained by the front camera can be reduced, and the rotation angle of the user face can be obtained according to the picture face proportion and the standard face proportion, and then, the embodiment of the invention calculates the watching distance from the eyes of the user to the display screen according to the rotation angle and the picture interpupillary distance.

It can be understood that, in the embodiment of the present invention, the length and the width of the face are calculated according to the captured face image, the inter-frame face distance of the user in the face image is obtained, and the frame face proportion is obtained according to the ratio of the length to the width.

It should be noted that, in the embodiment of the present invention, a rotation angle is obtained by performing analysis and comparison according to the picture face proportion and the standard face proportion, and a mapping relationship with the angle may be established in advance according to an experiment according to a basis of the comparison analysis, for example, in an embodiment, an angle under each condition is recorded according to a comparison relationship between the sample picture face proportion and the standard face proportion, so as to subsequently obtain a rotation angle of the user face directly according to the picture face proportion and the standard face proportion, in another embodiment, values of the picture face proportion and the standard face proportion may be input into a neural network model to obtain the rotation angle, and the neural network model may be established according to data in the sample, which is not limited in particular by the embodiment of the present invention.

Referring to fig. 11, in an embodiment, the step S504 may further include, but is not limited to, the steps S601 to S605.

Step S601, acquiring a preset standard interpupillary distance and a focal length of a face image shot by a front camera, and obtaining an initial distance of the face image to an imaging point according to the focal length.

Step S602, obtaining a first distance between the eyes of the user on the target plane according to the standard interpupillary distance, the picture interpupillary distance, the initial distance and the rotation angle.

And step S603, obtaining a second proportion according to the picture pupillary distance and the first distance, and obtaining a second distance from the target plane to the display screen according to the second proportion and the initial distance.

And step S604, obtaining a third distance from the user' S eyes to the target plane according to the standard interpupillary distance and the rotation angle.

And step S605, obtaining the watching distance from the eyes of the user to the display screen according to the sum of the second distance and the third distance.

It should be noted that, in the process of calculating the viewing distance from the user's eyes to the display screen according to the rotation angle and the picture interpupillary distance, specifically, a preset standard interpupillary distance and a focal distance of a face image shot by the front camera are obtained first, an initial distance corresponding to an imaging point of the face image is obtained according to the focal distance, a first distance between the user's eyes on a target plane is obtained according to the standard interpupillary distance, the picture interpupillary distance, the initial distance and the rotation angle, a plane where the first distance is located is a plane parallel to the face image and corresponds to a plane where the user's table standard through hole distance is located in the above embodiment, then, the embodiment of the present invention performs a segmentation calculation, a second ratio is obtained first according to the picture interpupillary distance and the first distance, a second distance from the target plane to the display screen is obtained according to the second ratio and the initial distance, a third distance from the user's eyes to the target plane is obtained according to the standard interpupillary distance and the rotation angle, and adding the two to obtain the viewing distance from the eyes of the user to the display screen.

Referring to fig. 12, in an embodiment, the step S602 may further include, but is not limited to, the steps S701 to S703.

And step S701, obtaining a portrait angle on an imaging point according to the picture pupil distance and the initial distance.

Step S702, a target angle is obtained according to the portrait angle and the rotation angle, wherein the target angle is an angle between the user 'S eyes at the far end side of the imaging point and the user' S eyes at the near end side of the imaging point.

Step S703, obtaining a first distance between the two eyes of the user on the target plane according to the target angle, the rotation angle and the standard interpupillary distance.

It should be noted that, in the process of obtaining the first distance, in the embodiment of the present invention, firstly, the portrait angle at the imaging point is obtained according to the picture pupil distance and the initial distance, and according to the imaging characteristics of the camera, when the face of the user is obliquely facing the front camera, because the image of the camera still is the image within the range of the angle of view, the triangle formed by the line segment at the first distance on the target plane and the imaging point can be regarded as an isosceles triangle to be calculated, because the triangle formed between the line segment at the picture pupil distance and the imaging point of the IC picture is also a short isosceles triangle, as shown in fig. 13, because the initial distance H0 is known, the picture pupil distance Q is also known, because the triangle formed between the line segment at the picture pupil distance and the imaging point is also a short isosceles triangle, the portrait angle α can be obtained, and according to the sum of the internal angles of the triangle is 180 degrees, therefore ≦ θ 1 and ≦ θ 2 can be determined.

In a triangle formed by a line segment with a standard pupil interval, a rotation angle and a line segment with a first interval, the angle theta 1 is determined, the rotation angle beta is known, so that the angle gamma is theta 1-angle beta, for a triangle, one side (the standard pupil interval K) and two angles (the angle beta and the angle gamma) on the side are known, the shape of the triangle is determined, the other side of the triangle can be obtained through computer operation, and the value of the first interval K1 is obtained.

Referring to fig. 14, in an embodiment, the step S603 may further include, but is not limited to, steps S801 to S802.

Step S801, obtaining a fourth distance from the target plane to the imaging point according to the second proportion and the initial distance.

Step S802, calculating a difference between the fourth distance and the initial distance, and taking the difference as a second distance from the target plane to the display screen.

It should be noted that, in the process of obtaining the viewing distance from the eyes of the user to the display screen, calculation is performed according to the characteristics of the triangle, in an embodiment, a triangle between the line segment where the inter-pupillary distance of the picture in fig. 13 and the imaging point is defined as a third triangle, a triangle between the line segment where the first distance in fig. 13 and the imaging point is defined as a fourth triangle, and the third triangle and the fourth triangle are similar triangles.

As shown in fig. 13, the fourth distance from the target plane to the imaging point can be obtained according to the second ratio of the picture pupillary distance Q divided by the first distance K1, then the second ratio is used to divide the initial distance H0 by the second ratio to obtain the fourth distance H4, and finally the fourth distance H4 is subtracted by the initial distance H0 to obtain the second distance H2 from the target plane to the display screen, wherein the calculation formula of H2 is as follows:

H2＝H4-H0 (3)

H4＝H0/(Q/K1) (4)

it will be appreciated that the third distance in the above embodiment may be one of the following formulas:

H3＝K×cos∠β (5)

or:

H3＝(K×cos∠β)/2 (6)

immediately, when formula (5) represents the distance to the eye of the user on the side farthest from the display screen and formula (6) represents the distance to the middle position between the eyes of the user, the viewing distance H from the eye of the user to the display screen can be obtained from the sum of the second distance H2 and the third distance H3.

Referring to fig. 15, in an embodiment, the step S104 may further include, but is not limited to, step S901 to step S902.

Step S901, a preset distance lookup table is obtained.

Step S902, looking up a table from a distance lookup table according to the interpupillary distance of the picture to obtain the viewing distance from the eyes of the user to the display screen.

It should be noted that the viewing distance in the embodiment of the present invention may also be obtained by querying according to a preset distance lookup table, specifically, in the embodiment of the present invention, a mapping relationship table from the picture pupil distance to the viewing distance may be pre-established, in the process of obtaining the viewing distance by calculation, the preset distance lookup table is obtained first, and the viewing distance from the user's eyes to the display screen is obtained by looking up the table from the distance lookup table according to the measured picture pupil distance.

Referring to fig. 16, in an embodiment, the distance lookup table is obtained according to the following steps, which may include, but are not limited to, steps S1001 to S1003.

Step S1001, a sample interpupillary distance in the sample is acquired.

Step S1002, the distance between the eyes of the user in the sample and the display screen is obtained to obtain the sample viewing distance.

Step S1003, establishing a distance lookup table based on the interpupillary distance to the viewing distance according to the sample interpupillary distance and the sample viewing distance.

It should be noted that, the distance lookup table in the above embodiment may be obtained by calculation according to data in a sample, specifically, in the table creating process, a sample interpupillary distance in the sample is obtained first, where the sample interpupillary distance is an interpupillary distance identified by a face image in the sample data, and then a distance from a user eye corresponding to the face image to a display screen is calculated to obtain a sample viewing distance, it is understood that different sample interpupillary distances in the sample will have a corresponding sample viewing distance, so a distance lookup table based on an interpupillary distance to a viewing distance may be established according to the sample interpupillary distance and the sample viewing distance, when the distance measuring method in the embodiment of the present invention is executed, when a picture interpupillary distance is obtained, a sample interpupillary distance corresponding to a numerical value may be found from the distance lookup table, and then a sample viewing distance corresponding to the sample interpupillary distance is found by looking up the table, the sample viewing distance is used as the viewing distance required in the embodiment of the present invention.

It is understood that the distance lookup table may also be established based on the rotation angle when error reduction by measuring the face proportion of the user is required, and is not particularly limited herein.

Referring to fig. 17, in an embodiment, the step S104 may further include, but is not limited to, steps S1101 to S1103.

In step S1101, a reference distance, a reference object size, and a screen size corresponding to the reference object photographed by the front camera are acquired.

Step S1102, a preset standard interpupillary distance is obtained.

Step S1103 obtains a viewing distance from the user' S eyes to the display screen according to the reference distance, the reference object size, the picture interpupillary distance, and the standard interpupillary distance.

It should be noted that, in the embodiment of the present invention, the viewing distance may also be obtained according to the establishment of the reference system, specifically, in the embodiment of the present invention, a reference object is first placed in front of the terminal, the reference distance from the reference object to the display screen and the object size of the reference object are measured, a front camera is used to capture an image of the reference object, the size of the reference object in the image is calculated in the obtained image, so as to obtain the picture size, and then the reference system may be established according to the picture size, so that the viewing distance from the user's eyes to the display screen may be obtained according to the reference distance, the reference object size, the picture pupillary distance, and the standard pupillary distance by obtaining a preset standard pupillary distance.

Referring to fig. 18, in an embodiment, the step S1103 may further include, but is not limited to, steps S1201 to S1204.

Step S1201, a first coefficient is obtained by dividing the standard interpupillary distance by the picture interpupillary distance.

Step S1202, a second coefficient is obtained according to the size of the reference object in the screen size.

In step S1203, a third coefficient is obtained by dividing the reference distance by the second coefficient.

Step S1204, obtaining a viewing distance from the user' S eye to the display screen according to a product of the third coefficient and the first coefficient.

It should be noted that, in the process of obtaining the viewing distance from the user's eyes to the display screen according to the reference distance, the reference object size, the picture interpupillary distance and the standard interpupillary distance, the present invention first obtains a first coefficient by dividing the standard interpupillary distance by the picture interpupillary distance, then obtains a second coefficient according to the picture size of the reference object size, obtains a third coefficient by dividing the reference distance by the second coefficient, and finally obtains the viewing distance from the user's eyes to the display screen according to the product of the third coefficient and the first coefficient.

For example, as shown in fig. 19 and 20, when the standard reference object has an object size of 10cm, a reference distance of 50cm, a screen size AB, and a standard interpupillary distance of 6.3cm, a screen interpupillary distance of AB, where the first coefficient is 6.3 ÷ AB and the second coefficient is 10 ÷ AB, the following formula for the viewing distance h can be finally established:

50÷(10÷AB)＝h÷(6.3÷ab) (7)

since the picture size AB and the picture interpupillary distance AB are known, the viewing distance h can be obtained according to equation (7).

It should be noted that, in general, the image captured by the front-facing camera may include faces of multiple users, and in the embodiment of the present invention, different users in the image may be identified to obtain pupil position information of different users, so as to measure a distance between different users, specifically, in the above-mentioned embodiment, the pupil recognition is carried out on the face image, a plurality of different users in the image can be determined, and the first pupil position information and the second pupil position information of the eyes of each user are respectively obtained, according to the first pupil position information and the second pupil position information of each user, the picture pupillary distance of different users can be obtained through calculation, finally, in the embodiment, the watching distance from each target user eye to the display screen can be obtained through calculation according to the picture pupillary distances of a plurality of different users, and finally, the distance measurement of different users is realized through the front camera.

It can be understood that, in the embodiment of the present invention, the face image may be processed to determine different users in the image, for example, in an embodiment, the face image may be input into a neural network model, and the nose feature is identified by using the neural network, so as to identify the corresponding eyes of each user, and finally obtain the pupil position information of the eyes of each user; in another embodiment, the face image can be identified according to the left eye feature and the right eye feature, so that different users in the image can be judged according to the left eyes and the right eyes of different users; in another embodiment, the face in the image may be framed according to a face detection frame in the recognized face image, and the face in the image may be framed according to the face detection frame, so that different user situations may be obtained according to different face frames, which is not limited in the embodiment of the present invention.

Referring to fig. 21, an embodiment of the present invention further provides a control method implemented by a camera-based distance measurement method, where the control method is applied to a terminal, the terminal is provided with a display screen and a front camera, and the control method may include but is not limited to steps S1301 to S1305.

Step S1301, a face image of the user is obtained, where the face image is obtained by a front-facing camera.

Step S1302, performing pupil identification on the face image, and determining first pupil position information and second pupil position information of the user' S eyes.

And step S1303, calculating the inter-pupillary distance of the user in the picture of the face image according to the first pupil position information and the second pupil position information.

And step S1304, calculating the watching distance from the eyes of the user to the display screen according to the interpupillary distance of the picture.

In step S1305, a software control policy is executed according to the viewing distance.

It should be noted that the control method in the embodiment of the present invention may be obtained based on the distance measurement method in the above embodiment, the distance measurement method is not described herein again, and after the viewing distance from the user's eyes to the display screen is obtained through calculation according to the inter-pupillary distance of the picture, the control method in the embodiment of the present invention may execute a software control strategy based on the obtained viewing distance.

In an embodiment, the terminal displays a 3D screen through a display screen, and as shown in fig. 22, the step S1305 may further include, but is not limited to, the steps S1401 to S1403.

In step S1401, a recommended distance range for viewing a 3D screen is acquired.

Step S1402, comparing and analyzing the recommended distance and the viewing distance to obtain a distance comparison result.

And step S1403, when the distance comparison result indicates that the user is not in the recommended distance range, sending distance prompt information.

It should be noted that the terminal in the embodiment of the present invention may be a terminal device capable of displaying a 3D picture, for example, the terminal device displays a 3D picture through a display screen of a mobile phone, or the terminal device is a 3D visual training terminal, and the terminal device can display a 3D picture through a display screen on the 3D visual training terminal, and the terminal device in the embodiment of the present invention is taken as an example of the 3D visual training terminal.

It can be understood that the 3D vision training terminal can perform naked eye 3D imaging, a user can view a naked eye 3D picture through a display screen on the device, the 3D vision training terminal can cut a 2D video into two parts and change the light-emitting refraction directions of the two parts, after the user eyes view the picture, the user can view a 3D picture, the formed 3D picture has a negative parallax and can be displayed between the user and the display screen, it needs to be explained that the formed 3D picture needs to be displayed at a specific distance to view the best viewing effect, and viewing blur occurs in case of being too far or too close.

Specifically, the 3D vision training terminal displays the 3D picture, the eyes of the user can be noted on the 3D picture, and plane fixation is broken, so that the damage of the plane fixation to the eyes of the user is avoided, myopia prevention and control can be realized, and in order to enable the user to be in a proper viewing range in the myopia prevention and control process, the viewing distance of the user needs to be judged.

For example, when the viewing distance of the user is too close, the user is reminded of the distance by hardware such as a display screen or a loudspeaker, and the user is reminded of moving backwards, and when the viewing distance of the user is too far, the user is reminded of the distance by hardware such as the display screen or the loudspeaker, and the user is reminded of moving forwards, so that the user can be reminded of reaching a better viewing distance, and a certain myopia prevention and control effect and viewing experience are guaranteed.

Referring to fig. 23, an embodiment of the present invention further provides a camera-based distance measuring method, which is applied to a terminal, where the terminal is provided with a camera, and the distance measuring method may include, but is not limited to, step S1501 to step S1504.

In step S1501, a face image of the user is acquired, and the face image is captured by the camera.

Step S1502 performs pupil recognition on the face image, and determines first pupil position information and second pupil position information of the user' S eyes.

And S1503, calculating the picture interpupillary distance of the user in the face image according to the first pupil position information and the second pupil position information.

And step S1504, calculating the distance from the eyes of the user to the terminal according to the picture interpupillary distance.

It should be noted that, in the distance measuring method based on a camera in the embodiment of the present invention, a face image of a user is obtained by capturing with the camera, pupil identification is performed on the face image, first pupil position information and second pupil position information of eyes of the user are determined, a picture inter-pupil distance of the user in the face image is calculated according to the first pupil position information and the second pupil position information of both eyes of the user, and may be determined according to a pixel number of a terminal, and finally, a viewing distance from the eyes of the user to the terminal is calculated according to the picture inter-pupil distance. And the watching distance from the eyes of the user to the terminal can be calculated according to the interpupillary distance of the picture, the distance measurement cost is low, and the distance measurement can be realized without additionally arranging other sensors.

It should be noted that the camera-based distance measurement method in the embodiment of the present invention may be implemented by the camera-based distance measurement method in any of the above embodiments, which is not described herein again, and the general distance measurement method in the embodiment of the present invention may be widely applied to various terminal devices having cameras or optical lenses to implement distance measurement.

Referring to fig. 24, an embodiment of the present invention further provides a camera-based distance measuring device, where the distance measuring device includes:

a first module 2401, configured to obtain an image of a face of a user.

A second module 2402, configured to perform pupil identification on the face image, and determine first pupil position information and second pupil position information of the eyes of the user; the image pupil distance calculation module is used for calculating the image pupil distance of the user in the face image according to the first pupil position information and the second pupil position information; and calculating the watching distance from the eyes of the user to the display screen according to the interpupillary distance of the picture.

It should be noted that, the distance measuring device based on a camera in the embodiment of the present invention may implement the distance measuring method in any one of the above embodiments, and the distance measuring device may be a terminal device such as a mobile phone, a tablet computer, a 3D vision training terminal, and the like, where the distance measuring device is configured with a first module 2401 for acquiring a face image of a user, and a second module 2402 for performing pupil recognition on the face image, and determining first pupil position information and second pupil position information of eyes of the user, so that a picture interpupillary distance of the user calculated and obtained in the face image may be determined according to the number of pixels, and is used for calculating a viewing distance from the eyes of the user to a display screen according to the picture interpupillary distance.

In an embodiment, the first module 2401 may be a front camera, the second module 2402 may be a processor, the face image of the user is obtained by obtaining the face image of the user, the face image is obtained by shooting the front camera, performing pupil recognition on the face image, determining first pupil position information and second pupil position information of eyes of the user, calculating a picture interpupillary distance of the user in the face image according to the first pupil position information and the second pupil position information, and finally calculating a viewing distance from the eyes of the user to a display screen according to the picture interpupillary distance, in an embodiment of the present invention, ranging may be implemented by the front camera arranged on a terminal, the pupil position of the user is obtained by analyzing an image obtained by the front camera, and a required picture interpupillary distance is calculated, the picture interpupillary distance is an interpupillary distance representing the user in the obtained image, and the watching distance from the eyes of the user to the display screen can be calculated according to the interpupillary distance of the picture, the distance measurement cost is low, and the distance measurement can be realized without additionally arranging other sensors.

Fig. 25 illustrates an electronic device 2500 provided by an embodiment of the invention. The electronic device 2500 includes: a processor 2501, a memory 2502 and a computer program stored on the memory 2502 and executable on the processor 2501, the computer program when executed is for performing the camera-based ranging method described above.

The processor 2501 and memory 2502 may be connected by a bus or other means.

The memory 2502 is a non-transitory computer readable storage medium, and can be used to store non-transitory software programs and non-transitory computer executable programs, such as the camera-based distance measuring method described in the embodiments of the present invention. The processor 2501 implements the camera-based ranging method described above by running non-transitory software programs and instructions stored in the memory 2502.

The memory 2502 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store and execute the camera-based ranging method described above. Further, the memory 2502 may include high speed random access memory 2502, and may also include non-transitory memory 2502, such as at least one storage device, flash memory device, or other non-transitory solid state storage device. In some implementations, the memory 2502 optionally includes memory 2502 located remotely from the processor 2501, the remote memory 2502 being connectable to the electronic device 2500 through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

Non-transitory software programs and instructions necessary to implement the camera-based ranging method described above are stored in the memory 2502 and, when executed by the one or more processors 2501, perform the camera-based ranging method described above, e.g., performing method steps S101-S104 in fig. 2, method steps S201-S204 in fig. 4, method steps S301-S303 in fig. 6, method steps S401-S402 in fig. 9, method steps S501-S504 in fig. 10, method steps S601-S605 in fig. 11, method steps S701-S1103 in fig. 12, method steps S801-S802 in fig. 14, method steps S501-S902 in fig. 15, method steps S1001-S1003 in fig. 16, method steps S1101-S1301 in fig. 17, method steps S1201-S1204 in fig. 18, method steps S1305 in fig. 21, and method steps S1305 in fig. 21, Method steps S1401 to S1403 in fig. 22, method steps S1501 to S1504 in fig. 23.

The above-described embodiments of the apparatus are merely illustrative and the modules may be selected according to actual needs.

One of ordinary skill in the art will appreciate that all or some of the steps, systems, and methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those of ordinary skill in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, storage device storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to those skilled in the art.

It should also be appreciated that the various implementations provided by the embodiments of the present invention can be combined arbitrarily to achieve different technical effects. While the preferred embodiments of the present invention have been described in detail, it will be understood, however, that the invention is not limited thereto, and that various equivalent modifications and substitutions can be made by those skilled in the art without departing from the spirit of the present invention.

Claims

1. A distance measuring method based on a camera is applied to a terminal, the terminal is provided with a display screen and a front camera, and the method is characterized by comprising the following steps:

acquiring a face image of a user, wherein the face image is obtained by shooting through the front-facing camera;

performing pupil identification on the face image, and determining first pupil position information and second pupil position information of the eyes of the user;

calculating the picture interpupillary distance of the user in the face image according to the first pupil position information and the second pupil position information;

and calculating the watching distance from the eyes of the user to the display screen according to the picture pupillary distance.

2. The camera-based ranging method according to claim 1, wherein the performing pupil recognition on the face image to determine first pupil position information and second pupil position information of the eyes of the user comprises:

converting the face image into a gray level image, and carrying out binarization processing on the gray level image to obtain a first preprocessed image;

carrying out corrosion and expansion treatment on the first preprocessed image, and eliminating noise in the image to obtain a second preprocessed image;

extracting the position of a circular area which represents the pupil of the user in the second preprocessed image by using a circular structural element;

and calculating the central point of the circular area to obtain the first pupil position information and the second pupil position information of the two eyes of the user.

3. The camera-based distance measuring method according to claim 1, wherein the calculating a viewing distance from the user's eyes to the display screen according to the picture interpupillary distance comprises:

acquiring a preset standard interpupillary distance;

acquiring the focal length of the face image shot by the front camera, and obtaining the initial distance of the face image to an imaging point according to the focal length;

and obtaining a first proportion according to the picture interpupillary distance and the standard interpupillary distance, and obtaining the viewing distance from the eyes of the user to the display screen according to the first proportion and the initial distance.

4. The camera-based ranging method according to claim 3, wherein the obtaining of the viewing distance from the eyes of the user to the display screen according to the first ratio and the initial distance comprises:

obtaining a first distance from the user's eyes to the imaging point according to the first proportion and the initial distance;

and calculating a difference value between the first distance and the initial distance, and taking the difference value as the viewing distance from the eyes of the user to the display screen.

5. The camera-based distance measuring method according to claim 1, wherein the calculating a viewing distance from the user's eyes to the display screen according to the picture interpupillary distance comprises:

calculating the picture face space of the user in the face image, and obtaining a picture face proportion according to the picture face space;

acquiring a preset standard face proportion;

obtaining the rotation angle of the face of the user according to the picture face proportion and the standard face proportion;

and calculating the viewing distance from the eyes of the user to the display screen according to the rotation angle and the picture interpupillary distance.

6. The camera-based distance measuring method according to claim 5, wherein the calculating of the viewing distance from the user's eyes to the display screen according to the rotation angle and the picture interpupillary distance comprises:

acquiring a preset standard interpupillary distance and a focal length of the face image shot by the front camera, and obtaining an initial distance of the face image to an imaging point according to the focal length;

obtaining a first distance between the eyes of the user on a target plane according to the standard pupillary distance, the picture pupillary distance, the initial distance and the rotation angle;

obtaining a second proportion according to the picture interpupillary distance and the first distance, and obtaining a second distance from the target plane to the display screen according to the second proportion and the initial distance;

obtaining a third distance from the eyes of the user to the target plane according to the standard interpupillary distance and the rotation angle;

and obtaining the viewing distance from the eyes of the user to the display screen according to the sum of the second distance and the third distance.

7. The camera-based distance measuring method according to claim 6, wherein the obtaining a first distance between the eyes of the user on the target plane according to the standard interpupillary distance, the frame interpupillary distance, the initial distance and the rotation angle comprises:

obtaining a portrait angle on the imaging point according to the picture pupil distance and the initial distance;

obtaining a target angle according to the portrait angle and the rotation angle, wherein the target angle is an angle between the user eyes at the far end side of the imaging point and the user eyes at the imaging point and the near end side of the imaging point;

and obtaining a first distance between the eyes of the user on the target plane according to the target angle, the rotation angle and the standard interpupillary distance.

8. The camera-based range finding method of claim 6, wherein the deriving the second distance from the target plane to the display screen according to the second ratio and the initial distance comprises:

obtaining a fourth distance corresponding to the imaging point by the target plane according to the second proportion and the initial distance;

and calculating a difference value between the fourth distance and the initial distance, and taking the difference value as a second distance from the target plane to the display screen.

9. The camera-based distance measuring method according to claim 1, wherein the calculating a viewing distance from the user's eyes to the display screen according to the picture interpupillary distance comprises:

acquiring a preset distance lookup table;

and looking up a table from the distance lookup table according to the picture interpupillary distance to obtain the viewing distance from the eyes of the user to the display screen.

10. The camera-based ranging method of claim 9, wherein the range lookup table is obtained according to the steps of:

obtaining a sample interpupillary distance in a sample;

obtaining the distance from the user eyes to the display screen in the sample to obtain the sample viewing distance;

and establishing the distance lookup table based on the interpupillary distance to the viewing distance according to the sample interpupillary distance and the sample viewing distance.

11. The camera-based distance measuring method according to claim 1, wherein the calculating a viewing distance from the user's eyes to the display screen according to the picture interpupillary distance comprises:

acquiring a reference distance, a reference object size and a picture size corresponding to the reference object shot by the front camera;

acquiring a preset standard interpupillary distance;

and obtaining the viewing distance from the eyes of the user to the display screen according to the reference distance, the reference object size, the picture interpupillary distance and the standard interpupillary distance.

12. The camera-based ranging method according to claim 11, wherein the obtaining of the viewing distance from the user's eyes to the display screen according to the reference distance, the reference object size, the picture interpupillary distance and the standard interpupillary distance comprises:

dividing the standard interpupillary distance by the picture interpupillary distance to obtain a first coefficient;

obtaining a second coefficient according to the size of the reference object in the picture size;

dividing the reference distance by the second coefficient to obtain a third coefficient;

and obtaining the viewing distance from the eyes of the user to the display screen according to the product of the third coefficient and the first coefficient.

13. The camera-based ranging method according to claim 1, wherein the performing pupil recognition on the face image to determine first pupil position information and second pupil position information of the eyes of the user comprises:

performing pupil identification on the face image, determining a plurality of different users in the image, and respectively obtaining first pupil position information and second pupil position information of eyes of each user;

calculating the viewing distance from the eyes of the user to the display screen according to the picture interpupillary distance, wherein the method comprises the following steps:

calculating to obtain the watching distance from each target user eye to the display screen according to the picture interpupillary distances of a plurality of different users;

and calculating the picture interpupillary distance of different users according to the first pupil position information and the second pupil position information of each user.

14. The camera-based range finding method of claim 1, wherein the front camera is an off-screen camera, and the off-screen camera is disposed at a center position of the display screen.

15. A control method realized by a distance measuring method based on a camera is applied to a terminal, the terminal is provided with a display screen and a front camera, and the method is characterized by comprising the following steps:

acquiring a face image of a user, wherein the face image is obtained by the front camera;

calculating the picture inter-pupillary distance of the user in the face image according to the first pupil position information and the second pupil position information;

calculating the watching distance from the eyes of the user to the display screen according to the picture interpupillary distance;

and executing a software control strategy according to the viewing distance.

16. The control method according to claim 15, wherein the terminal displays a 3D picture through the display screen, and the executing of the software control policy according to the viewing distance includes:

acquiring a recommended distance range for 3D picture watching;

comparing and analyzing the recommended distance and the watching distance to obtain a distance comparison result;

and when the distance comparison result indicates that the user is not in the recommended distance range, sending distance prompt information.

17. A distance measuring method based on a camera is applied to a terminal, the terminal is provided with the camera, and the method is characterized by comprising the following steps:

acquiring a face image of a user, wherein the face image is shot by the camera;

and calculating the distance from the eyes of the user to the terminal according to the picture interpupillary distance.

18. The utility model provides a range unit based on camera which characterized in that includes:

the system comprises a first module, a second module and a third module, wherein the first module is used for acquiring a face image of a user;

the second module is used for carrying out pupil identification on the face image and determining first pupil position information and second pupil position information of the eyes of the user;

the image pupil distance calculation module is used for calculating the image pupil distance of the user in the face image according to the first pupil position information and the second pupil position information;

and calculating the watching distance from the eyes of the user to the display screen according to the picture interpupillary distance.

19. An electronic device, comprising a memory storing a computer program, and a processor that when executing the computer program implements the camera-based ranging method of any one of claims 1 to 14, or the control method implemented by the camera-based ranging method of any one of claims 15 to 16, or the camera-based ranging method of claim 17.

20. A computer-readable storage medium, characterized in that the storage medium stores a program which is executed by a processor to implement the camera-based ranging method according to any one of claims 1 to 14, or the control method implemented by the camera-based ranging method according to any one of claims 15 to 16, or the camera-based ranging method according to claim 17.