WO2021253756A1

WO2021253756A1 - Body temperature measurement method of mobile terminal device, and mobile terminal device and medium

Info

Publication number: WO2021253756A1
Application number: PCT/CN2020/135648
Authority: WO
Inventors: 黄河
Original assignee: 中芯集成电路（宁波）有限公司上海分公司
Priority date: 2020-06-16
Filing date: 2020-12-11
Publication date: 2021-12-23
Also published as: CN113796838A

Abstract

A body temperature measurement method of a mobile terminal device, and a mobile terminal device and a medium. The mobile terminal device comprises a temperature measurement unit and an imaging unit. The method comprises: setting a reference coordinate system, wherein the reference coordinate system comprises an X axis and a Y axis (S1); setting coordinates of a temperature measurement unit in the reference coordinate system as first coordinates, wherein the temperature measurement unit is used for measuring the temperature of a human face in real time in a first direction, and the first direction is perpendicular to a plane where the X axis and the Y axis are located (S2); acquiring an image of the human face in real time in the first direction by using an imaging unit (S3); determining the coordinates of the image of a temperature measurement point of the human face in the reference coordinate system to be second coordinates (S4); determining whether the second coordinates are overlapped or basically overlapped with the first coordinates, and determining whether the distance between a mobile terminal device and the temperature measurement point in the first direction is within a preset range (S5); and if so, outputting a temperature signal detected by the temperature measurement unit (S6). Convenient, quick and accurate body temperature measurement is thus implemented on a mobile terminal device.

Description

Method for measuring body temperature of mobile terminal equipment, mobile terminal equipment and medium

Technical field

The invention belongs to the field of body temperature measurement, and in particular relates to a body temperature measurement method of a mobile terminal device, a mobile terminal device and a medium.

Background technique

Existing body temperature measurement, commonly used thermometers, forehead thermometers, etc., all require special temperature measurement tools for measurement.

technical problem

It is very inconvenient for users to carry temperature measurement tools or daily temperature measurement. At the same time, the existing infrared detection type temperature measuring device is often inaccurate in the position of the measuring point. For example, when a hand-held forehead thermometer is used to measure the temperature of the human body, the position of the measuring point is only a rough position, such as the wrist or forehead, using manual methods. It is often impossible to accurately locate the temperature measurement point, so that the accuracy of the temperature measurement is not high, and manpower is wasted. The infrared body temperature measurement used in places with a lot of people is by detecting the temperature of the entire human body. The equipment is large in size and costly, and it is not convenient to use in places with relatively small people.

Therefore, it is necessary to propose a body temperature measurement method that is portable, easy to use, and can accurately locate the temperature measurement point.

Technical solutions

In order to solve the above-mentioned problems, embodiments of the present invention provide a body temperature measurement method of a mobile terminal device, a mobile terminal device, and a medium, which are portable, easy to use, and capable of accurately positioning temperature measurement points.

In order to achieve the foregoing objectives, in the first aspect, the present invention proposes a method for measuring body temperature of a mobile terminal device, the mobile terminal device includes a temperature detection unit and an imaging unit, and the method includes:

Setting a reference coordinate system, the reference coordinate system including an X axis and a Y axis;

The coordinates of the temperature detection unit in the reference coordinate system are set as the first coordinates, and the temperature detection unit is used to detect the temperature of the face in real time along a first direction, and the first direction is perpendicular to all the coordinates. The plane where the X axis and the Y axis are located;

Acquiring the image of the human face in real time along the first direction by using the imaging unit;

Determining that the coordinates of the image of the temperature measurement point of the human face in the reference coordinate system are the second coordinates;

Judging whether the second coordinate and the first coordinate are coincident or substantially coincident; and judging whether the distance between the mobile terminal and the temperature measurement point along the first direction is within a preset range;

If yes, output the temperature signal detected by the temperature detection unit.

In the second aspect, the present invention also provides a mobile terminal device, including:

The imaging unit is used to obtain an image of a human face along the first direction;

A temperature detection unit, configured to detect the temperature of the human face along the first direction;

The storage unit stores: related information of a reference coordinate system, the related information of the reference coordinate system includes: X axis and Y axis, and the first direction is perpendicular to the plane where the X axis and the Y axis are located; The coordinates of the temperature detection unit in the reference coordinate system are the first coordinates;

A processing unit, determining that the coordinates of the image of the temperature measurement point of the face in the reference coordinate system are the second coordinates;

A judging unit for judging whether the second coordinate and the first coordinate are coincident or substantially coincident; and judging whether the distance between the mobile terminal and the temperature measurement point in the first direction is within a preset range;

The display unit is configured to display the temperature value detected by the temperature inspection unit when the judgment result of the judgment unit is yes.

In a third aspect, the present invention also provides a non-transitory computer-readable storage medium that stores computer instructions for making a computer perform body temperature measurement of the mobile terminal device of the first aspect method.

Beneficial effect

The beneficial effects of the present invention are:

In the present invention, the imaging unit collects facial images in real time along the first direction, and the temperature detection unit detects the temperature of the human face along the first direction. Given the first coordinates of the temperature detection unit in the reference coordinate system, the first direction is perpendicular to The X-axis and Y-axis of the reference coordinate system are used to determine the second coordinate of the image of the temperature measurement point in the reference coordinate system based on the collected image of the face, and by judging whether the second coordinate and the first coordinate are in the reference coordinate system Coincident or substantially coincident, so as to determine whether the temperature measurement point and the temperature detection unit are aligned. When it is judged to be coincident or substantially coincident, the detection value of the temperature detection unit is the effective temperature value, thereby realizing the temperature measurement of the face on the mobile terminal device Accurate positioning of points, so that accurate measurement can be achieved. Moreover, the method is used on a mobile terminal, so that people can use mobile terminal equipment such as mobile phones to measure temperature at any time, which is convenient and quick.

Description of the drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.

FIG. 1 is a step diagram of a method for measuring body temperature of a mobile terminal device according to the first embodiment of the present invention;

2 is a schematic structural diagram of a mobile terminal device according to the second embodiment of the present invention;

FIG. 3 is a schematic diagram of the relationship between functional units in a mobile terminal device according to the second embodiment of the present invention;

FIG. 4 is a reference diagram of the use of a mobile terminal device according to the second embodiment of the present invention;

Description of reference signs:

In Figure 2 and Figure 3:

1. Mobile terminal equipment; 2. Imaging unit; 3. Temperature detection unit; 4. Storage unit; 5. Processing unit; 501, First determination subunit; 502, Second determination subunit; 503, Third determination subunit 504, temperature measurement point identification unit; 6, judgment unit; 601, first sub-judgment unit; 602, second sub-judgment unit; 603, output unit; 7, display unit; 8, temperature measurement point; 9, distance acquisition Unit; 10. Distance sensor.

Embodiments of the present invention

At present, professional tools or equipment such as clinical thermometers, hand-held forehead thermometers, or infrared imaging devices are commonly used for body temperature measurement. It is very inconvenient for users to carry temperature measurement tools or perform daily temperature measurement. The present invention integrates the temperature detection unit on the mobile terminal device, which can facilitate people's daily temperature measurement needs.

In order to achieve accurate body temperature measurement, it involves the technical problem of aligning the temperature detection unit with the temperature measurement point. The present invention establishes a reference coordinate system and assigns the first coordinates of the temperature detection unit, and collects and determines the facial image by the imaging unit. For the image of the temperature measurement point, calculate and determine the second coordinate of the temperature measurement point in the set reference coordinate system. Because the direction of the imaging unit receiving light radiation is perpendicular to the plane of the reference coordinate system, and the converted first and second coordinates The coordinates can correspond to the projection of the temperature detection unit and the real physical position of the temperature measurement point on the XY plane along the first direction and have a specific and the same conversion relationship, and then determine whether the first coordinate and the second coordinate coincide or basically coincide to solve the problem. How to judge whether the temperature measurement point is aligned with the temperature detection unit is a technical problem, and then accurate body temperature measurement can be realized on the mobile terminal device.

The preferred embodiments of the present invention will be described in more detail below. Although the preferred embodiments of the present invention are described below, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein.

Example one

As shown in Fig. 1, a method for measuring body temperature of a mobile terminal device. The mobile terminal device includes a temperature detection unit and an imaging unit. The method includes:

Step S1: Set a reference coordinate system, the reference coordinate system includes X axis and Y axis;

Step S2: Set the coordinates of the temperature detection unit in the reference coordinate system as the first coordinate. The temperature detection unit is used to detect the temperature of the face in real time along the first direction, which is perpendicular to the plane where the X axis and the Y axis are located. ；

Step S3: Use the imaging unit to obtain real-time images of the human face along the first direction;

Step S4: Determine that the coordinates of the image of the temperature measurement point of the face in the reference coordinate system are the second coordinates;

Step S5: Determine whether the second coordinate and the first coordinate coincide or substantially coincide; and determine whether the distance between the mobile terminal device and the temperature measurement point in the first direction is within a preset range;

Step S6: If yes, output the temperature signal detected by the temperature detection unit.

The present invention collects human face images in real time through the imaging unit. Given the first coordinates of the temperature detection unit in the reference coordinate system, the second image of the temperature measurement point in the reference coordinate system is determined based on the collected images of the human face and face. Coordinates, by judging whether the second coordinate and the first coordinate coincide or basically coincide in the reference coordinate system, and then judge whether the temperature detection unit and the temperature measurement point are aligned. When it is judged that the temperature detection unit and the temperature measurement point are aligned, the temperature detection unit The detection value of is the effective temperature value, so as to realize the accurate positioning of the face temperature measurement point on the mobile terminal device, so as to realize the accurate body temperature measurement. Moreover, the method is used on a mobile terminal, so that people can use mobile terminal equipment such as mobile phones to measure temperature at any time, which is convenient and quick.

The specific embodiments of the present invention will be described in detail below in conjunction with specific embodiments.

In this embodiment, the imaging unit and the temperature detection unit are integrated on the mobile terminal device in advance, and the temperature measurement point is set to the central area of the forehead of the human face. The imaging unit receives light radiation and the temperature detection unit receives infrared radiation. The target to be tested.

Regarding step S1: setting a reference coordinate system, the reference coordinate system includes an origin, an X axis, and a Y axis.

In this embodiment, the image coordinate system (two-dimensional coordinate system) of the imaging unit can be set as the reference coordinate system, and the reference coordinate system includes the X axis and the Y axis; the origin coincides with the optical axis of the imaging unit. In other embodiments, the camera coordinate system of the imaging unit can also be set as the reference coordinate system, or the pixel coordinate system of the imaging unit can be set as the reference coordinate system, etc. The foregoing multiple coordinate systems can be based on the lens of the imaging unit. The parameters realize the corresponding coordinate conversion, so as long as the reference coordinate system satisfies one of the arbitrary coordinate systems belonging to the imaging unit itself, or the established reference coordinate system can have a known corresponding conversion relationship with any coordinate system of the imaging unit itself That's it. In this embodiment, the image coordinate system is selected as the reference coordinate system for real-time description.

The purpose of setting the reference coordinate system is to convert the physical position of the temperature measurement point to the second coordinate in the reference coordinate system according to the facial image obtained by the imaging unit, and to convert and store the physical position of the temperature detection unit at the same time. The first coordinates in the same reference coordinate system, for example, the coordinates (X0, Y0) of the temperature detection unit and the coordinates (Xt, Yt) of the temperature measurement point can be determined in the same reference coordinate system, due to the direction in which the imaging unit receives light radiation It is perpendicular to the plane where the X axis and the Y axis are located, and the converted first and second coordinates can correspond to the projection of the temperature detection unit and the real physical position of the temperature measurement point along the first direction on the XY plane and have a specific and the same conversion Therefore, when (X0, Y0) and (Xt, Yt) coincide or basically coincide in the reference coordinate system, it means that the temperature detection unit is basically directly opposite to the temperature measurement point.

Regarding step S2: Set the position of the temperature detection unit in the reference coordinate system as the first coordinate, and the temperature detection unit is used to detect the temperature of the face of the subject in real time along the first direction, the first direction being perpendicular to the X axis And the plane on which the Y axis lies.

In this embodiment, according to the position of the temperature detection unit in the mobile terminal device, the first coordinates of the temperature detection unit in the reference coordinate system are stored in advance.

Specifically, since the temperature detection unit and the imaging unit are both fixedly arranged on the mobile terminal device, the position of the temperature detection unit on the mobile terminal device is fixed, such as being arranged on the top of the mobile terminal device, that is, the coordinates of the temperature detection unit are relative to each other. Compared with the origin of the reference coordinate system, the position of the origin is unchanged, so it can be converted to the first coordinate in the reference coordinate system based on the physical fixed position of the temperature detection unit. The first coordinate is recorded as (X0, Y0). In this embodiment, the temperature detection unit The temperature of the human face is measured in a first direction. Optionally, the first direction is parallel to the optical axis of the imaging unit.

Regarding step S3: using the imaging unit to obtain an image of the human face in the first direction in real time; and step S4: determining the position of the image of the temperature measurement point of the human face in the reference coordinate system as the second coordinate.

The method for determining the second coordinate may include the first method or the second method. The basic principle of the first method is: acquiring a two-dimensional or three-dimensional image of a human face based on an imaging unit, recognizing a two-dimensional image or The feature points in the three-dimensional image that have a specific geometric relationship with the temperature measurement point (the center of the forehead) are calculated according to the coordinates of the feature point to calculate the second coordinates of the temperature measurement point; the basic principle of the second method is: to obtain the face based on the imaging unit The two-dimensional image or three-dimensional image is based on the trained face intelligent recognition algorithm model. After the face is recognized, based on the temperature measurement point position marked in the model, the temperature measurement point is directly located and the second temperature measurement point is calculated. coordinate.

The first method is described in detail below. The method for determining the second coordinate of the first method includes:

Step AS41: Determine the image of the feature point of the face according to the image of the face, and the feature point has a specific geometric position relationship with the temperature measurement point;

Step AS42: Determine the coordinates of the feature point image in the reference coordinate system;

Step AS43: Calculate the second coordinate of the image of the temperature measurement point according to the coordinates of the image of the characteristic point in the reference coordinate system.

In an example, regarding step AS41, when the measured temperature is the forehead temperature, the temperature measurement point is the central area of the forehead, the characteristic points may be eyes, and each eye is a characteristic point. When the feature points are binoculars, the feature information of the binoculars can be pre-stored in the mobile terminal device, and the images of the human face and the binocular feature information can be compared to identify the binoculars.

Regarding step AS42, after identifying the feature points, that is, after identifying the eyes, the coordinates of the eyes in the reference coordinate system are determined through a certain algorithm.

Regarding step AS42, the second coordinate of the temperature measurement point, that is, the center of the forehead, can be calculated through a certain algorithm (such as a triangulation algorithm) based on the specific geometric position relationship between the eyes and the central area of the forehead.

In a specific application scenario, the AI face recognition algorithm preset in the mobile terminal can be used to recognize the human face, that is, the face, and then recognize the first feature point and the second feature point, namely the eyes, and calculate that the eyes are imaging The coordinates in the coordinate system are the third coordinate (X1, X1) and the fourth coordinate (X2, Y2). Based on the coordinates of the eyes in the face image and the unique geometric relationship between the eyes and the forehead, the central area of the forehead is determined The coordinates in the imaging coordinate system, that is, the coordinates of the temperature measurement point in the imaging coordinate system are calculated as (Xt, Yt), where the optical parameters of the optical camera are fixed, so the physical coordinate system and the reference coordinate system of the face ( There is a fixed conversion relationship between the imaging coordinate system), and finally the coordinates of the temperature measurement point in the reference coordinate system are calculated through the fixed conversion relationship, that is, (Xt, Yt) can correspond to the real temperature measurement point relative to the mobile device Positional relationship.

Among them, the following schemes can be adopted for the identification of feature points:

Store the relevant feature point information of the face and the temperature measurement point (the center of the forehead) in the mobile terminal device, collect a large number of face images, and calibrate the facial features in each image, such as eyes, eyebrows, ears, and mouth The positions of feature points such as, nose, and facial contours, and establish a specific geometric position relationship between these feature points and the center of the forehead, and form training data samples for intelligent image algorithms (such as OPENCV, etc.). When the image acquired by the imaging unit contains When all facial images or partial facial images including temperature measurement points and feature points are acquired, the corresponding feature points can be automatically identified through intelligent image algorithms, and then the location of the temperature measurement points can be determined according to the feature points.

In another embodiment of this method, the characteristic points may also be two ears, and each ear is a characteristic point. The determination of the coordinates of the temperature measuring point on the forehead can be achieved in a manner similar to that described above through the coordinates of the eyes, and the method is the same as the above, and will not be repeated here.

In other embodiments of this method, the feature points may also include at least two points of the face contour, as long as the two points are easy to identify and have a specific geometric position relationship with the forehead temperature measurement point. The above method is used to determine the second coordinate of the forehead temperature measurement point by means of the coordinates of the eyes, which will not be repeated here.

The above method can also be used to measure ear temperature. In this case, the temperature measurement point is the ear canal; the characteristic points include at least two points of the ear contour. The second coordinate of the ear canal temperature measurement point can be determined in a manner similar to the above-mentioned method of using the coordinates of the eyes, which will not be repeated here.

It should be noted that, in the present invention, the face of the human face can be the front of the entire human face or the front of a partial human face including feature points and temperature measurement points, or it can be the side of the human face or include feature points and temperature measurement points. The side surface of the partial face, when it is the front of the face or the front of the partial face including feature points and temperature measurement points, can be used to measure the forehead temperature. When it is the side surface of a human face or the side surface of a partial human face including feature points and temperature measurement points, the imaging unit mainly photographs the contour of the human ear to prepare for the temperature detection unit to measure the ear temperature.

The second method is described in detail below. The method for determining the second coordinate of the second method includes:

Step BS41: Identify the temperature measurement point image of the face image based on the characteristics of the temperature measurement point, and the characteristics of the temperature measurement point include at least one of a shape and an image feature;

Step BS42: Determine the second coordinate of the temperature measurement point image in the reference coordinate system based on the recognized image of the temperature measurement point.

In one example, regarding step BS41, based on the inherent characteristics of the temperature measurement point itself, such as shape, image features, etc., directly locate and identify the image of the temperature measurement point in the acquired face;

Regarding step BS42, after the image of the temperature measurement point is recognized, the coordinates of the temperature measurement point in the reference system are directly calculated.

In a specific application scenario, the two-dimensional or three-dimensional image of the human face is acquired in real time through the imaging unit. After the overall image of the human face is acquired, the location of the temperature measurement point (such as the forehead) is directly located based on the intelligent recognition algorithm , And then determine its second coordinate in the reference coordinate system.

Among them, the following schemes can be adopted for direct identification of temperature measurement points:

Store the contour, shape, image and other characteristic information related to the temperature measurement point (forehead) in the face in the mobile terminal device, collect a large number of face images, and mark the image area position of the forehead area in the face image, and Form training data samples of intelligent image algorithms (such as OPENCV, etc.). When the image acquired by the imaging unit contains a human face, the intelligent image algorithm can automatically identify and locate the location of the forehead area, and then directly determine the temperature measurement point of the forehead Location.

In the foregoing embodiment, the acquisition of the two-dimensional image of the human face can be achieved by using a monocular optical camera or a double-sided optical camera, and the three-dimensional image of the human face can be acquired by a TOF sensor or a 3D structured light sensor as an imaging unit.

Among them, the principle of real-time acquisition of the three-dimensional image of the human face through the TOF sensor is: the TOF camera continuously emits the laser dot matrix to the human face, and at the same time records the return time of the reflected light, and calculates the human face based on the return time of the reflected light The depth information is based on the TOF three-dimensional reconstruction algorithm to obtain the three-dimensional image of the detected object.

Specifically, the TOF sensor includes a laser emitting source and a CCD receiver. The basic principle is that the laser emitting source emits multiple lasers with a certain viewing angle, where each laser has a duration of dt (from t1 to t2), and each pixel of the CCD Two synchronous trigger switches S1 (t1 to t2) and S2 (t2 to t2+dt) are used to control the period of time during which the charge holding element of each pixel collects the reflected light intensity, and the responses C1 and C2 are obtained. The distance between the object and each pixel is L=0.5*c*dt*c2/(c1+c2), where c is the speed of light (this formula can remove the influence of the difference in the reflective characteristics of the reflector on the ranging). Based on the distance information of different laser points, a three-dimensional image of a human face can be established by the existing TOF three-dimensional reconstruction algorithm. Based on the established three-dimensional image, the central area of the forehead can be accurately identified according to the overall three-dimensional structure of the human face. Furthermore, based on the position of the temperature measuring point on the forehead, the second coordinate (Xt, Yt) of the temperature measuring point in the reference coordinate system can be easily calculated. It should be noted that face recognition through the TOF sensor is an existing mature technology, which is easily implemented by those skilled in the art, and will not be repeated here.

The principle of obtaining three-dimensional images of human faces in real time through a 3D structured light camera is: the 3D structured light camera emits continuous and structurally characteristic laser speckles (continuous non-pulsed surface light source) to the face and face, and at the same time through the 3D structured light The optical camera in the camera collects the image of the laser speckle, and the image based on the laser speckle obtains the three-dimensional image of the detected object through the 3D structured light three-dimensional reconstruction algorithm. The basic principle of using a 3D structured light camera to build a three-dimensional image is: sending laser speckles to the face, and random diffraction spots formed when the laser hits rough objects or penetrates ground glass. These speckles are highly random and change patterns with different distances, that is, the speckle patterns at any two places in the space are different. Therefore, as long as the space is marked with such structured light, the entire space will be marked. For the face in the space, only the speckle pattern on the face and face needs to be recognized to obtain the position of the face and face.

In this solution, the spatial distribution of the speckle pattern needs to be calibrated for the light source in advance. For example, at intervals of a distance, such as 10cm, take a reference plane and record the speckle pattern on the reference plane. Within a recognizable distance range, multiple different speckle patterns will be recorded. During the measurement, take a speckle image of a human face, and perform cross-correlation calculations between this image and multiple pre-stored reference images in turn to obtain multiple correlation images, and the human face in the space The position of the part will show the peak value on the correlation image. By superposing and interpolating these peaks, a three-dimensional image of the entire human face can be obtained. Afterwards, the central area of the forehead can be accurately identified according to the overall three-dimensional structure of the face, and then the second coordinates (Xt, Yt) of the temperature measurement point in the reference coordinate system can be easily calculated based on the position of the temperature measurement point on the forehead. It should be noted that face recognition based on a 3D structured light camera is an existing mature technology, which can be easily implemented by a person skilled in the art, and will not be repeated here.

Regarding step S5: judging whether the second coordinate and the first coordinate coincide or substantially coincide in the first direction; and judging whether the distance between the mobile terminal and the temperature measurement point in the first direction is within a preset range.

Among them, the method for judging whether the second coordinate and the first coordinate coincide or substantially coincide in the first direction is: based on the X value and Y value in the first coordinate (X0, Y0) and the second coordinate (Xt, Yt), namely It can be judged whether the first coordinate and the second coordinate coincide or basically coincide in the reference coordinate system, and then judge whether the temperature measurement point (the center of the forehead) is aligned with the temperature detection unit.

Determine whether the distance between the temperature measurement point and the temperature detection unit is within the preset range, including the following steps:

S51: Acquire the distance between the temperature measurement point of the face and the mobile terminal in real time;

S52: Compare the measured distance with the preset range;

S53: Determine whether the measured distance is within the preset range.

Among them, the temperature detection unit adopts the existing infrared temperature sensor, and the infrared temperature sensor includes an infrared heat pile for receiving the thermal radiation of the human face, so it is judged whether the distance between the temperature measurement point and the temperature detection unit is within the preset range That is, it is judged whether the detection is within the range of the depth of field of the infrared temperature sensor, optionally, the range of the depth of field is 0 cm to 50 cm.

Regarding step S51, the method for obtaining the distance between the temperature measurement point of the human face and the mobile terminal in real time may be:

The first way:

The distance between the temperature detection unit and the temperature measurement point along the first direction is estimated according to the image of the characteristic point. Specifically, when the imaging unit adopts an optical monocular camera, the distance between the face and the imaging unit can be estimated using the existing triangulation method based on the lens focal length in the lens parameters of the optical monocular camera, and then the distance between the face and the imaging unit can be estimated. The distance between the temperature detection unit and the temperature measurement point.

When the imaging unit adopts an optical binocular camera, the distance measurement based on the optical binocular camera is currently generally used for high-precision measurement at close distances. The basic principle is that the two cameras in the optical binocular camera simultaneously collect two facial images. Based on the phase difference (parallax) between the face and the face in the two images, the distance between the face and the face is calculated to determine whether the distance between the mobile terminal and the temperature measurement point in the first direction is within a preset range. It should be noted that the distance measurement of the binocular camera is an existing mature technology, which can be easily realized by those skilled in the art and will not be repeated here.

The second way:

A preset distance sensor in the mobile terminal device is used to detect the distance between the temperature measurement point and the mobile terminal device.

Specifically, distance measurement is performed by integrating a distance sensor for assisting distance measurement on the mobile terminal. The distance sensor can be a laser radar, a TOF sensor, or an infrared distance sensor. The distance sensor also performs distance measurement along the first direction. When it is recognized that the second coordinates of the forehead area on the face and the first coordinates of the temperature detection unit basically coincide, the distance between the preset distance sensor and the temperature measurement point (or between the real-time measurement and the temperature measurement point) is measured by the preset distance sensor. The distance between the temperature detection unit and the temperature measurement point in the first direction can be determined more accurately whether the distance in the first direction is within the preset range. It should be noted that when the imaging unit is an optical monocular camera, it is preferable to use a distance sensor for auxiliary ranging to achieve accurate ranging.

The third way is:

When the image acquired by the imaging unit is a three-dimensional image, the distance between the temperature detection unit and the temperature measurement point is estimated based on the acquired depth information of the human face.

Specifically, when the image acquired by the imaging unit is a three-dimensional image, for example, when the imaging unit adopts a TOF sensor, the depth between the face and the face has been acquired in the process of recognizing the face and establishing the three-dimensional image. Therefore, the distance to the temperature measurement point can be directly estimated, that is, the distance between the temperature detection unit and the temperature measurement point can be estimated based on the return time of the reflected light at the temperature measurement point in the three-dimensional image.

Or, when the imaging unit is a 3D structure sensor, the depth information of the human face has already been acquired when the three-dimensional image of the human face is established, so the distance to the temperature measurement point can be directly estimated. That is, based on the image information of the laser speckle at the temperature measurement point in the three-dimensional image, the distance between the imaging unit and the temperature measurement point is calculated, and then the distance between the temperature detection unit and the temperature measurement point in the first direction is estimated. Regarding the ranging principle of the TOF sensor and the 3D structured light sensor, please refer to the content of the aforementioned three-dimensional imaging principle of the TOF sensor and the 3D structured light sensor, which will not be repeated here.

Regarding S52 and S53, the calculated distance between the temperature detection unit and the temperature measurement point along the first direction is compared with the preset range (that is, the depth of field range of the temperature detection unit); and then it is judged whether the measured distance is within Within the preset range. For example, the depth of field range of the temperature detection unit in this embodiment is 0-50cm. When the measured or calculated distance is less than 50cm, it means that the temperature measuring point of the face or forehead is within the preset depth of field (effective temperature measurement distance) If the distance is greater than 50cm, it is judged to exceed the depth of field range.

Regarding step S6: if yes, output the temperature signal detected by the temperature detection unit.

Specifically, the screen of the mobile terminal device can be used to display relevant information, and output the temperature signal monitored by the temperature detection unit.

Before outputting the temperature signal, it also includes:

Determine whether the temperature signal is a valid temperature signal; when the second coordinate and the first coordinate coincide or substantially coincide, and the distance between the mobile terminal and the temperature measurement point in the first direction is preset When it is within the range, the detected temperature signal is an effective temperature signal; otherwise, it is an invalid temperature signal.

Specifically, when it is satisfied that the temperature measurement point is accurately identified and the temperature measurement point is aligned with the temperature detection unit, and the temperature measurement point is within the depth of field of the temperature detection unit, the temperature measurement point can be recorded in the mobile terminal device ( The detection value of the forehead) is the effective temperature value and displayed to the user to achieve accurate body temperature measurement. If the temperature measurement point is not accurately identified, misaligned (the temperature measurement point and the temperature detection unit do not basically coincide in the first direction), the distance between the temperature detection unit and the temperature measurement point is not within any one of the preset ranges If the item is not satisfied, the measured temperature is invalid temperature value, and the measured temperature value may not be measured or displayed.

Example two

FIG. 2 is a schematic structural diagram of a mobile terminal device according to Embodiment 1 of the present invention; FIG. 3 is a working flow diagram of each functional unit in a mobile terminal device according to Embodiment 2 of the present invention.

Referring to Fig. 2, a mobile terminal device 1 includes:

The imaging unit 2 is used to obtain an image of a human face along a first direction;

The temperature detection unit 3 is used to detect the temperature of the human face along the first direction;

The storage unit 4 stores: related information of the reference coordinate system, the related information of the reference coordinate system includes: X axis, Y axis; the first direction is perpendicular to the plane where the X axis and the Y axis are located; the temperature detection unit 3 The first coordinate in the reference coordinate system;

The processing unit 5 determines the position of the image of the temperature measurement point of the human face in the reference coordinate system as the second coordinate;

The judging unit 6 is used to judge whether the second coordinate and the first coordinate are coincident or substantially coincident; and judging whether the distance between the mobile terminal 1 and the temperature measuring point 8 along the first direction is within a preset range;

The display unit 7 displays the temperature value detected by the temperature inspection unit 3 when the judgment result of the judgment unit 6 is yes.

In this embodiment, the imaging unit 2 includes at least one of an optical monocular camera, an optical binocular camera, a TOF sensor, and a 3D structured light sensor. Among them, the real-time facial images obtained by the optical monocular camera and the optical binocular camera are two-dimensional images, and the facial images obtained by the TOF sensor and the 3D structured light sensor are three-dimensional images.

In this embodiment, the temperature detection unit 3 is an existing infrared temperature sensor, and the infrared temperature sensor includes an infrared heat pile for receiving the thermal radiation of the human face. The preset range is the depth of field (effective temperature measurement distance) of the temperature detection unit 3, and optionally, the range of the depth of field is 0 cm to 50 cm. It should be noted that the depth of field of the temperature detection unit 3 can be selected according to actual design requirements of temperature sensors (thermal pile chips) with different depth ranges, which will not be repeated here.

Referring to FIG. 3, in this embodiment, the processing unit 5 includes:

The first determining subunit 501 is configured to determine the image of the feature point of the face and face according to the image of the face and face acquired by the imaging unit 2, and the storage unit 4 also stores the specific geometric position relationship between the feature point and the temperature measurement point 8;

The second determining subunit 502 is configured to determine the coordinates of the image of the feature point in the reference coordinate system according to the image of the feature point determined by the first determining unit;

The third determining subunit 503 is used to obtain the geometric position relationship, and calculate the second coordinate of the temperature measurement point image according to the coordinates of the feature point image in the reference coordinate system and the geometric position relationship.

The judging unit 6 includes:

The first sub-judgment unit 601 is used to judge whether the second coordinate and the first coordinate coincide or substantially coincide in the reference coordinate system;

The second sub-judging unit 602 is used for judging whether the distance between the mobile terminal device 1 and the temperature measuring point 8 along the first direction is within a preset range;

The output unit 603 outputs a judgment signal. When the judgment results of the first sub judgment unit and the second sub judgment unit are both yes, the judgment signal is an effective temperature signal; the judgment results of the first sub judgment unit and the second sub judgment unit When both are no, the judgment signal is an invalid temperature signal.

Among them, the temperature measuring point 8 is the central area of the forehead, the characteristic points are the eyes, and each eye is a characteristic point; in other embodiments, the characteristic points may also be both ears, and each ear is a characteristic point, or the characteristic points include At least two points of the face contour.

In another embodiment, the temperature measurement point is the ear canal; the characteristic points include at least two points of the ear contour.

Optionally, it further includes: a temperature measurement point recognition unit 504, configured to recognize the temperature measurement point 8 image of the face image according to the characteristics of the temperature measurement point 8, and determine the temperature measurement point 8 based on the recognized image of the temperature measurement point 8 The second coordinate of the image of the temperature point 8 in the reference coordinate system; the characteristics of the temperature measurement point 8 include at least one of a shape and an image feature.

In this embodiment, the reference coordinate system is the image coordinate system of the imaging unit 2. In other embodiments, the reference coordinate system may also be the camera coordinate system or the pixel coordinate system of the imaging unit 2.

In this embodiment, the first coordinate is determined based on the position of the temperature detection unit 3 in the mobile terminal device 1 and stored in the storage unit 4 in advance; in other embodiments, the first coordinate is based on the temperature detection unit 3 and the imaging unit 2 The physical relative position of is determined and stored in the storage unit 4 in advance.

In this embodiment, a distance acquisition unit 9 is further included. According to the sensor type selected by the imaging unit, the distance between the temperature detection unit and the temperature measurement point in the first direction can be estimated according to the image coordinates of the feature point; or, when the mobile terminal device is preset When a distance sensor is installed, the distance sensor is directly used to detect the distance between the temperature measuring point 8 and the mobile terminal device 1; or, when the image acquired by the imaging unit 2 is a three-dimensional image, it is estimated based on the depth information of the acquired face The distance between the temperature detection unit 3 and the temperature measurement point 8. Optionally, the distance sensor 10 may be an infrared distance sensor, a TOF sensor, a lidar, etc., preset on the top of the mobile device.

Further, when the optical monocular camera is selected, it is preferable to configure an infrared distance sensor to measure the distance between the temperature measuring points 8. When the optical binocular camera is selected, the distance sensor may not be configured, and the existing binocular distance measurement algorithm is directly used Realize the distance measurement of the temperature measuring point 8; when the TOF sensor or 3D structured light sensor is selected as the imaging unit, since the acquired 3D facial image includes depth information, it is not necessary to configure the distance sensor, and can directly estimate the mobile terminal device The distance between 1 and the temperature measuring point 8. The principles of binocular distance measurement, TOF sensor distance measurement, and 3D structured light sensor distance measurement refer to Embodiment 1, and will not be repeated here.

In a specific application scenario, the mobile terminal device 1 can be an existing mobile terminal device, such as a mobile phone, a tablet computer, etc., in terms of hardware, the temperature detection unit 3 is integrated into the existing mobile terminal device, and then the mobile terminal is used The existing optical camera and binocular camera in the equipment, or add TOF sensor, 3D structured light camera, etc. as the imaging unit 2, use the memory chip (RAM\ROM) in the mobile terminal device as the storage unit 4, and use the mobile terminal device The screen is used as the display unit 7 to display information such as the face image and the detected temperature value. In terms of software, the processing unit 5, the judgment unit 6 and other functional modules are integrated in the processing chip system (CPU, GPU) of the mobile terminal device, and the mobile terminal The system of the terminal equipment integrates existing or developed and designed intelligent face recognition, three-dimensional reconstruction and other related algorithms to realize the image recognition of the face and face, thereby realizing the scheme of the present invention. It is also possible to develop a separate temperature detection terminal device according to the scheme of the present invention, which integrates the imaging unit 2, the temperature detection unit 3, the display unit 5, the recognition processing unit 4, and the image recognition algorithm to implement the scheme of the present invention. Application requirements are developed and designed, so I won’t repeat them here.

Fig. 4 is a reference diagram for use of a mobile terminal device according to the second embodiment of the present invention.

Referring to Figure 4, the screen of the mobile terminal device 1 is used to display relevant information, prompting the distance of moving the measured object, so that the relative distance between the mobile terminal device 1 and the measured object is within the focal length of the imaging unit 2 or the temperature detection unit 3 Between depths of field, for example, adjust the position of the mobile terminal device 1 to be close to the human face, or adjust the position of the human face to be close to the mobile terminal device 1. For example, when a person is close to the mobile terminal device 1, when the relative distance between the person and the mobile terminal device 1 is too close, a reminder message is displayed on the display screen of the mobile terminal device 1 to remind the person to stay away from the mobile terminal device 1, or when the person and the mobile terminal device 1 When the relative distance of the terminal device 1 is too far, a reminder message is displayed on the display screen of the mobile terminal device 1 to remind the person to approach the mobile terminal device 1. When the human face is recognized, the second coordinate of the temperature measuring point 8 is calculated, and the temperature measuring point 8 is accurately recognized, and the temperature measuring point 8 is aligned with the temperature detecting unit 3 and the temperature measuring point 8 is in the temperature detecting unit When the depth of field is within the range of 3, the detection value of the temperature measuring point 8 (forehead) can be recorded in the mobile terminal device 1 and displayed to the user to achieve accurate body temperature measurement. If the temperature measurement point is not accurately identified, misaligned (the second coordinate of the temperature measurement point 8 and the first coordinate of the temperature detection unit 3 do not overlap or basically overlap in the reference coordinate system), the temperature detection unit 3 yuan and the temperature measurement point If the distance between 8 is not within the preset range, any one of the items is not satisfied, the measured temperature is invalid temperature value, and the measured temperature value may not be measured or displayed.

Example three

A non-transitory computer-readable storage medium according to an embodiment of the present invention stores computer instructions for causing a computer to execute the body temperature measurement method of a mobile terminal device of the first embodiment.

According to the computer-readable storage medium of the embodiment of the present invention, non-transitory computer-readable instructions are stored thereon. When the non-transitory computer-readable instruction is executed by the processor, all or part of the steps of the method in the first embodiment of the present invention are executed.

The aforementioned computer-readable storage media include, but are not limited to: optical storage media (for example: CD-ROM and DVD), magneto-optical storage media (for example: MO), magnetic storage media (for example: magnetic tape or mobile hard disk), Rewrite non-volatile memory media (for example: memory card) and media with built-in ROM (for example: ROM cartridge).

In summary, the present invention collects facial images in real time through the imaging unit, and at the same time uses the coordinate system of the imaging unit as the reference coordinate system. Given the first coordinates of the temperature detection unit in the reference coordinate system, it is based on the collected facial images. Identify the second coordinate of the temperature measurement point in the reference coordinate system, by judging whether the second coordinate and the first coordinate coincide or basically coincide in the reference coordinate system, when it is judged to be coincident or substantially coincident, the detection value of the temperature detection unit is Effective temperature value, so as to realize the accurate measurement of the face and face temperature measurement points on the mobile terminal device, and can realize the convenient and easy-to-use accurate body temperature detection of the face and face.

The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Without departing from the scope and spirit of the described embodiments, many modifications and changes are obvious to those of ordinary skill in the art.

Claims

A method for measuring body temperature of a mobile terminal device, the mobile terminal device including a temperature detection unit and an imaging unit, characterized in that the method includes:

Setting a reference coordinate system, the reference coordinate system including an X axis and a Y axis;

The coordinates of the temperature detection unit in the reference coordinate system are set as the first coordinates, and the temperature detection unit is used to detect the temperature of the human face in real time along a first direction, the first direction being perpendicular to the X Axis and the plane on which the Y axis is located;

Acquiring the image of the human face in real time along the first direction by using the imaging unit;

Determining that the coordinates of the image of the temperature measurement point of the human face in the reference coordinate system are the second coordinates;

Judging whether the second coordinate coincides or substantially coincides with the first coordinate; and judging whether the distance between the mobile terminal device and the temperature measurement point along the first direction is within a preset range;

If yes, output the temperature signal detected by the temperature detection unit.
The method for measuring body temperature of a mobile terminal device according to claim 1, wherein the preset range is a depth of field of the temperature detection unit.
The method for measuring body temperature of a mobile terminal device according to claim 2, wherein the depth of field ranges from 0 cm to 50 cm.
The method for measuring body temperature of a mobile terminal device according to claim 1, wherein the first direction is parallel to the optical axis of the imaging unit.
The method for measuring body temperature of a mobile terminal device according to claim 1, wherein before outputting the temperature signal, the method further comprises:

Judging whether the temperature signal is a valid temperature signal;

When the second coordinate and the first coordinate coincide or substantially coincide, and the distance between the mobile terminal and the temperature measurement point in the first direction is within a preset range, the detected temperature signal is Effective temperature signal; otherwise, it is an invalid temperature signal.
The method for measuring body temperature of a mobile terminal device according to claim 1, wherein the determining that the coordinates of the image of the temperature measurement point of the human face in the reference coordinate system are the second coordinates, comprises:

Determining the image of the feature point of the face according to the image of the face, where the feature point and the temperature measurement point have a specific geometric position relationship;

Determining the coordinates of the image of the feature point in the reference coordinate system;

The second coordinate of the temperature measurement point image is calculated according to the coordinates of the image of the characteristic point in the reference coordinate system.
The method for measuring body temperature of a mobile terminal device according to claim 6, wherein the temperature measurement point is the central area of the forehead;

The characteristic points are two eyes, and each eye is a characteristic point;

Or, the characteristic points are both ears, and each ear is a characteristic point;

Alternatively, the feature points include at least two points of the contour of the human face.
The method for measuring body temperature of a mobile terminal device according to claim 6, wherein the temperature measurement point is an ear canal; and the characteristic point includes at least two points of the ear contour.
The method for measuring body temperature of a mobile terminal device according to claim 1, wherein the determining that the coordinates of the image of the temperature measurement point of the human face in the reference coordinate system are the second coordinates, comprises:

Identify the temperature measurement point image of the face image based on the characteristics of the temperature measurement point, and the characteristics of the temperature measurement point include at least one of a shape and an image feature;

Based on the recognized image of the temperature measurement point, the second coordinate of the temperature measurement point image in the reference coordinate system is determined.
The method for measuring body temperature of a mobile terminal device according to claim 1, wherein the reference coordinate system comprises an image coordinate system, a camera coordinate system, or a pixel coordinate system of the imaging unit.
The method for measuring body temperature of a mobile terminal device according to claim 1, wherein setting the position of the temperature detection unit in the reference coordinate system as the first coordinate comprises:

According to the position of the temperature detection unit in the mobile terminal, the first coordinates of the temperature detection unit in the reference coordinate system are stored in advance.
The method for measuring body temperature of a mobile terminal device according to claim 1, wherein determining whether the distance between the mobile terminal and the temperature measurement point along the first direction is within a preset range comprises:

Acquiring the distance between the temperature measurement point of the face and the mobile terminal in real time;

Comparing the acquired distance with the preset range;

It is determined whether the measured distance is within the preset range.
The method for measuring body temperature of a mobile terminal device according to claim 12, wherein acquiring the distance between the temperature measurement point of the human face and the mobile terminal in real time comprises:

Estimate the distance between the temperature detection unit and the temperature measurement point in the first direction according to the image of the characteristic point; or,

Use a preset distance sensor in the mobile terminal device to detect the distance between the temperature measurement point and the mobile terminal device;

or,

The image is a three-dimensional image;

The depth information of the human face is acquired, and the distance between the temperature detection unit and the temperature measurement point is estimated based on the depth information.
The method for measuring body temperature of a mobile terminal device according to claim 1, wherein the temperature detection unit is an infrared temperature sensor.
A mobile terminal device, characterized in that it comprises:

The imaging unit is used to obtain an image of a human face along the first direction;

A temperature detection unit, configured to detect the temperature of the human face along the first direction;

The storage unit stores: related information of a reference coordinate system, the related information of the reference coordinate system includes: X axis and Y axis, and the first direction is perpendicular to the plane where the X axis and the Y axis are located; The coordinates of the temperature detection unit in the reference coordinate system are the first coordinates;

A processing unit, determining that the coordinates of the image of the temperature measurement point of the face in the reference coordinate system are the second coordinates;

A judging unit for judging whether the second coordinate and the first coordinate are coincident or substantially coincident; and judging whether the distance between the mobile terminal device and the temperature measurement point along the first direction is within a preset range within;

The display unit is configured to display the temperature value detected by the temperature inspection unit when the judgment result of the judgment unit is yes.
The mobile terminal device according to claim 15, wherein the judging unit comprises:

The first sub-judging unit is used to judge whether the second coordinate and the first coordinate are coincident or substantially coincident;

The second sub-judging unit is used to judge whether the distance between the mobile terminal device and the temperature measurement point along the first direction is within a preset range;

The output unit outputs a judgment signal. When the judgment results of the first sub judgment unit and the second sub judgment unit are both yes, the judgment signal is an effective temperature signal; When one of the judgment results of the second sub judgment unit is no, the judgment signal is an invalid temperature signal.
The mobile terminal device according to claim 15, wherein the processing unit comprises:

The first determining subunit is configured to determine the image of the feature point of the face and face according to the image of the face and face acquired by the imaging unit, and the storage unit also stores the features of the feature point and the temperature measurement point. Specific geometric position relationship;

The second determining subunit is configured to determine the coordinates of the image of the characteristic point in the reference coordinate system according to the image of the characteristic point determined by the first determining unit;

The third determining subunit is used to obtain the geometric position relationship, and calculate the temperature measurement point image according to the coordinates of the feature point image in the reference coordinate system and the geometric position relationship The second coordinate.
The mobile terminal device of claim 17, wherein the temperature measurement point is a central area of the forehead;

The characteristic points are two eyes, and each eye is a characteristic point;

Or, the characteristic points are both ears, and each ear is a characteristic point;

Alternatively, the feature points include at least two points of the contour of the human face.
The mobile terminal device according to claim 17, wherein the temperature measurement point is an ear canal; and the characteristic point includes at least two points of the ear contour.
The mobile terminal device according to claim 15, further comprising:

The temperature measurement point recognition unit is used to recognize the temperature measurement point image in the face image according to the characteristics of the temperature measurement point, and based on the recognized image of the temperature measurement point, determine that the temperature measurement point image is in the The second coordinate in the reference coordinate system;

The characteristics of the temperature measurement point include at least one of a shape and an image characteristic.
The mobile terminal device according to claim 15, wherein the first direction is parallel to the optical axis of the imaging unit.
The mobile terminal device according to claim 15, wherein the preset range is the depth of field of the temperature detection unit.
The mobile terminal device of claim 22, wherein the depth of field ranges from 0 cm to 50 cm.
The mobile terminal device according to claim 15, wherein the reference coordinate system comprises an image coordinate system of the imaging unit.
The mobile terminal device according to claim 15, wherein the first coordinates are determined based on the position of the temperature detection unit in the mobile terminal device and stored in the storage unit in advance.
The mobile terminal device according to claim 17, further comprising a distance obtaining unit, configured to estimate the distance between the temperature detection unit and the temperature measurement point along the first direction according to the image of the characteristic point;

or,

Use a preset distance sensor in the mobile terminal device to detect the distance between the temperature measurement point and the mobile terminal device;

or,

When the image acquired by the imaging unit is a three-dimensional image, the distance between the temperature detection unit and the temperature measurement point is estimated based on the acquired depth information of the human face.
The mobile terminal device of claim 26, wherein the distance sensor is one of an infrared distance sensor, a TOF sensor, and a lidar.
28. The mobile terminal device of claim 15, wherein the imaging unit comprises at least one of an optical monocular camera, an optical binocular camera, a TOF sensor, and a 3D structured light sensor.
The mobile terminal device according to claim 15, wherein the temperature detection unit is an infrared temperature sensor.
A non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to make a computer execute the mobile terminal device according to any one of claims 1-14. Body temperature measurement method.