CN110309777B

CN110309777B - Face recognition AR glasses based on LCOS framework and face recognition method thereof

Info

Publication number: CN110309777B
Application number: CN201910583832.3A
Authority: CN
Inventors: 定世宇
Original assignee: Wuhan Splendid Optronics Tech Co Ltd
Current assignee: Wuhan Splendid Optronics Tech Co Ltd
Priority date: 2019-07-01
Filing date: 2019-07-01
Publication date: 2021-04-30
Anticipated expiration: 2039-07-01
Also published as: CN110309777A

Abstract

The invention discloses face recognition AR glasses based on an LCOS framework and a face recognition method thereof, wherein the face recognition AR glasses comprise an integrated glasses frame 1, a glasses main body 2 positioned at a glasses frame at one side of the glasses frame 1 and glasses legs 3 positioned at the other side of the glasses frame 1, the lower ends of the glasses legs 3 are fixedly connected with nose pads 4, the lower ends of the nose pads 4 are provided with two gaskets 5, a camera hole 6 is formed in the glasses main body 2, the glasses main body 2 comprises a camera module, a data processing module, a communication module and a display module, and the display module, the camera module and the data processing module are all connected with the communication module; the invention enables the police to wear the glasses in daily patrol to quickly and accurately find the target criminal suspect in a public place with large and dense pedestrian traffic, increases the efficiency of public security case handling, enables a command center to timely master the field situation, quickly deploys the arresting scheme and greatly improves the urban safety.

Description

Face recognition AR glasses based on LCOS framework and face recognition method thereof

Technical Field

The invention belongs to the technical field of AR. In particular to face recognition AR glasses based on an LCOS framework and a face recognition method thereof.

Background

Augmented Reality (AR) is a technology for calculating the position and angle of a camera image in real time and adding a corresponding image, and the technology aims to sleeve a virtual world on a screen in the real world and perform interaction. With the development of the technology in these years, various AR glasses begin to be widely used in the field of public vision, and in the field of safe cities, the information of escaping people can be quickly and accurately identified by combining the public security department with the face identification technology in patrol of railway station squares.

Although many of existing prism-type AR glasses also adopt an LCOS architecture, the imaging optical path thereof adopts a double-cemented lens in combination with a single lens, and the structure is very complicated in such an optical path design. The method has the advantages that security threats such as data tampering and attack are easily caused to the police face recognition AR glasses network, images are lost, the images cannot be updated in time, and the accuracy of scene depth information shooting is low.

Disclosure of Invention

The invention aims to overcome the defects and provides face recognition AR glasses based on an LCOS framework and a face recognition method thereof.

The utility model provides a face identification AR glasses based on LCOS framework, includes the mirror holder 1 of integral type, be located the glasses main part 2 of 1 side of mirror holder frame department and be located the mirror leg 3 of 1 opposite side of mirror holder, the lower extreme fixedly connected with nose of mirror leg 3 holds in the palm 4, the lower extreme that the nose held in the palm 4 is equipped with two gaskets 5, the hole of making a video recording 6 has been seted up on glasses main part 2, glasses main part 2 includes camera module, data processing module, communication module and display module, camera module, data processing module all link to each other with communication module.

The display module includes: the LCOS display device comprises an LCOS display chip 201, a polarization beam splitter prism PBS202 and a free-form surface prism 203 which are sequentially arranged along a first axis, and an Led lighting device 204 which is arranged on a second axis perpendicular to the first axis and close to the polarization beam splitter prism PBS 202.

According to the face recognition AR glasses based on the LCOS framework, the face recognition method comprises the following steps:

the camera module collects face data such as face images and videos in real time, and sends the collected face data to the data processing module and the control center through the communication module;

the data processing module processes the received face data and sends the face data to the control center through the communication module;

a face database of the criminal evasion suspect in the public security system is prestored in the control center, the control center compares the processed face data with the prestored face database to judge the real identity of the identified person, and if the identified person belongs to the criminal evasion suspect, identity identification data is generated and sent to the LCOS display chip 201 in the display module;

a light source emitted by the Led lighting device 204 irradiates the polarization beam splitter PBS202 to generate linearly polarized light, so as to realize lighting of the LCOS display chip 201, virtual image light emitted by the LCOS display chip 201 enters the free-form surface prism 203 after being transmitted by the polarization beam splitter PBS202, is refracted and then combined with ambient light entering the free-form surface prism 203 after being transmitted, and then is transmitted to human eyes from a light-emitting surface of the free-form surface prism 203, so that a face image of a suspect escaping from crimes can be displayed in front of the eyes of an operator;

the control center receives and stores the face data such as the face image and the video collected by the camera module in real time, and the face data is used as evidence and scenes for the police.

The glasses can be worn by public security policemen in daily patrol to quickly and accurately find target criminal suspects in public places with large and dense people flow, the public security case handling efficiency is increased, images of all glasses can be guaranteed to be updated in a timely and uniform mode, even if some equipment is attacked or broken down, the images can still be guaranteed to be distributed reliably, the picture precision is not affected by complex scenes, the real-time requirement of an augmented reality system is met, a command center can timely master the field condition, the capture scheme is quickly deployed, and the urban safety is greatly improved.

Drawings

Fig. 1 is a perspective view of the present invention.

FIG. 2 is a schematic diagram of an optical path in a display module according to the present invention.

FIG. 3 is a schematic diagram of the main body and the control center of the eyeglasses of the present invention.

Fig. 4 is a partial side view of the outer appearance of the eyeglass body of the present invention.

Detailed Description

The invention is further illustrated by the following specific examples:

the utility model provides a face identification AR glasses based on LCOS framework, includes the mirror holder 1 of integral type, be located the glasses main part 2 of 1 side of mirror holder frame department and be located the mirror leg 3 of 1 opposite side of mirror holder, the lower extreme fixedly connected with nose of mirror leg 3 holds in the palm 4, the lower extreme that the nose held in the palm 4 is equipped with two gaskets 5, the hole of making a video recording 6 has been seted up on glasses main part 2, glasses main part 2 includes camera module, data processing module, communication module and display module, camera module, data processing module all link to each other with communication module. Wherein the camera module can be mounted in the camera hole 6.

The display module includes: the LCOS display device comprises an LCOS display chip 201, a polarization beam splitter prism PBS202 and a free-form surface prism 203 which are sequentially arranged along a first axis, and an Led lighting device 204 which is arranged on a second axis perpendicular to the first axis and close to the polarization beam splitter prism PBS 202. The first axis and the second axis are perpendicular to the light incident surface of the polarization beam splitter PBS202 and penetrate through the geometric center point of the polarization beam splitter PBS202, the polarization beam splitter PBS202 and the Led lighting device 204 are matched to generate linearly polarized light, so that the LCOS display chip 201 is illuminated, virtual image light emitted by the LCOS display chip 201 enters the free-form surface prism 203 after being transmitted by the polarization beam splitter PBS202, and is combined with ambient light from the environment light which enters the free-form surface prism 203 after being transmitted and then is transmitted to human eyes from the light emitting surface of the free-form surface prism 203. The free-form surface prism 203 is installed inside the glasses main body 2, corresponds to the position of human eyes, and a small part of the lower end of the free-form surface prism can not be shielded by the glasses main body 2 and can receive the direct radiation of the ambient light, so that the human eyes can see the virtual image and the real ambient image after being overlapped on the light-emitting side of the first light-emitting surface Se 1.

The LCOS framework not only perfectly meets the requirements of a compact and light-weight system, but also has the characteristics of heat dissipation function, low energy consumption, cost performance and the like, and is specially designed for AR application such as glasses, head-mounted display and the like which need the requirements of durability, compactness, light weight and the like. With the ever-increasing demand for wearable AR microdisplays in the areas of safe cities and industry, this area will become an emerging market that is constantly evolving.

The control center compares the processed face data with a prestored face database to judge the real identity of the identified person, and when a plurality of policemen wear the AR glasses to execute tasks in a monitoring area, the control center sends identity identification data to all AR glasses; the total number of the AR glasses in the monitoring area is 2n +1, wherein n is the number of the AR glasses with abnormal communication modules, the transmission of the identification data in the monitoring area is possibly influenced, and the remaining n +1 is normal AR glasses; the first AR glasses forward the first identity identification data received by the first AR glasses to other AR glasses in the monitoring area, namely the first AR glasses forward the first identity identification data to other 2n AR glasses, and the other 2n AR glasses forward the received identity identification data; after the remaining AR glasses receive the first identification data forwarded by the first AR glasses, the AR glasses match the first identification data with the received identification data information forwarded by the other 2n-1 AR glasses, and if the information contents are the same, the matching is successful; if the matching is successful, continuously forwarding the first identity identification data to other 2n-1 AR glasses, and receiving a set of all identity identification data information forwarded by other 2n-1 AR glasses; if the number of the identification data in the set of the identification data information received by a certain AR glasses exceeds n, identifying the AR glasses, forwarding the identification of the AR glasses to other AR glasses, and enabling the AR glasses to receive information which is forwarded by other 2n-1 AR glasses and whether the information is provided with the identification or not; when the number of the AR glasses with the identification exceeds n, each AR glass updates the identification data through the data processing module; and when the number of the AR glasses with the identification does not exceed n, each AR glass does not update the identification data.

The identification data may be one or more or a group.

The communication module is interfered, attacked by a network or failed in self firmware.

When the first AR glasses forward the first identity identification data received by the first AR glasses to other AR glasses in the monitoring area, calculating a period T from the time when the first AR glasses forward the first identity identification data received by the first AR glasses to other AR glasses in the monitoring area to the time when all the AR glasses are traversed, wherein the failure-free forwarding probability of the identity identification data in the period T is as follows:

wherein, p is the probability of single normal forwarding of the identification data, m is the number of AR glasses worn by the police in a monitoring area, and n is the number of AR glasses with abnormal communication modules;

and the control center judges whether the actual forwarding probability is less than pT after one period T is finished, if so, the communication module in the first AR glasses is abnormal, and the communication module is replaced or encrypted.

The camera module collects the same scene twice, and the front image and the rear image form a group, namely a first image p1 and a second image p2, so as to generate a three-dimensional point data set; converting the three-dimensional point data set into a third map P3, assuming that P (x, y, z) is a point in the three-dimensional point data set, calculating its coordinates corresponding to point P (a, b, sj) in the third map P3:

where a and b are the abscissa and ordinate of each pixel in the third graph p3, sj is depth data, lx and ly are the focal length of the camera module on the X, Y axis, o_xOy is the center of the aperture of the camera module, and s is a depth scaling factor;

creating an original image with the same size as the second image p2, traversing each pixel point in the original image, searching a depth value sj of the pixel point corresponding to the third image p3, and if sj is larger than zero, marking the pixel area as A1; if sj is equal to zero, label the pixel area as A2; for pixel points in an A1 area, copying a corresponding depth value in a third image p3 to an original image, for pixel points in an A2 area, copying a corresponding pixel value in a second image p2 to the original image to obtain a fourth image p4, setting len as the length between a certain pixel and a narrow edge on the edge of a missing part in the fourth image p4, storing len values of all pixels on the edge to the narrow edge of a queue in an ascending mode, sequentially processing, assuming that a starting point is q, processing four adjacent domain points of a point q first, adding the points to the narrow edge of the queue, and sequentially processing each pixel stored in the narrow edge of the queue until filling is completed.

Claims

1. The utility model provides a face identification AR glasses based on LCOS framework, includes mirror holder (1) of integral type, be located glasses main part (2) of mirror holder (1) one side frame department and be located mirror leg (3) of mirror holder (1) opposite side, the lower extreme fixedly connected with nose of mirror leg (3) holds in the palm (4), the lower extreme that nose held in the palm (4) is equipped with two gaskets (5), camera hole (6), its characterized in that have been seted up on glasses main part (2): the glasses main body (2) comprises a camera module, a data processing module, a communication module and a display module, and the display module, the camera module and the data processing module are all connected with the communication module; the display module includes: the LCOS display device comprises an LCOS display chip (201), a polarization beam splitter prism PBS (202) and a free-form surface prism (203) which are sequentially arranged along a first axis, and an Led lighting device (204) which is arranged on a second axis perpendicular to the first axis and close to the polarization beam splitter prism PBS (202); the face recognition method comprises the following steps:

the camera module collects face data such as face images in real time, and sends the collected face data to the data processing module and the control center through the communication module;

the method comprises the steps that a face database of a criminal evasion suspect in a public security system is prestored in a control center, the control center compares processed face data with the prestored face database to judge the real identity of an identified person, and if the identified person belongs to the criminal evasion suspect, identity identification data are generated and sent to an LCOS display chip (201) in a display module;

a light source emitted by an Led lighting device (204) irradiates a polarization beam splitter prism PBS (202) to generate linearly polarized light, so that illumination on an LCOS display chip (201) is realized, virtual image light emitted by the LCOS display chip (201) enters a free-form surface prism (203) after being transmitted by the polarization beam splitter prism PBS (202), is refracted and then is combined with ambient light entering the free-form surface prism (203) after being transmitted, and then is transmitted to human eyes from a light-emitting surface of the free-form surface prism (203), so that a human face image of a person escaping from a crime suspect can be displayed in front of the eyes of an operator;

the control center receives and stores face data such as face images and videos collected by the camera module in real time and the face data serves as evidences and scenes for policemen to use; the control center compares the processed face data with a prestored face database to judge the real identity of the identified person, and when a plurality of policemen wear the AR glasses to execute tasks in a monitoring area, the control center sends identity identification data to all AR glasses; the total number of the AR glasses in the monitoring area is 2n +1, wherein n is the number of the AR glasses with abnormal communication modules, the transmission of the identification data in the monitoring area is possibly influenced, and the remaining n +1 is normal AR glasses; the first AR glasses forward the first identity identification data received by the first AR glasses to other AR glasses in the monitoring area, namely the first AR glasses forward the first identity identification data to other 2n AR glasses, and the other 2n AR glasses forward the received identity identification data; after the remaining AR glasses receive the first identification data forwarded by the first AR glasses, the AR glasses match the first identification data with the received identification data information forwarded by the other 2n-1 AR glasses, and if the information contents are the same, the matching is successful; if the matching is successful, continuously forwarding the first identity identification data to other 2n-1 AR glasses, and receiving a set of all identity identification data information forwarded by other 2n-1 AR glasses; if the number of the identification data in the set of the identification data information received by a certain AR glasses exceeds n, identifying the AR glasses, forwarding the identification of the AR glasses to other AR glasses, and enabling the AR glasses to receive information which is forwarded by other 2n-1 AR glasses and whether the information is provided with the identification or not; when the number of the AR glasses with the identification exceeds n, each AR glass updates the identification data through the data processing module; and when the number of the AR glasses with the identification does not exceed n, each AR glass does not update the identification data.

2. A face recognition method based on the LCOS-based face recognition AR glasses, as claimed in claim 1, wherein the identification data may be one or more pieces or a group.

3. The face recognition method of claim 2, wherein the communication module abnormality is an interference, a network attack, or a firmware failure.

4. The face recognition method according to claim 3, wherein when the first AR glasses forward the first identity recognition data received by the first AR glasses to other AR glasses in the monitoring area, a period T is calculated from the time when the first AR glasses forward the first identity recognition data received by the first AR glasses to other AR glasses in the monitoring area to the time when all the AR glasses are traversed, and the forwarding probability of the identity recognition data in the period T is zero:

5. The face recognition method of claim 4, wherein the camera module collects the same scene twice, and the two images are grouped into a first image p1 and a second image p2, so as to generate a three-dimensional point data set; converting the three-dimensional point data set into a third map P3, assuming that P (x, y, z) is a point in the three-dimensional point data set, calculating its coordinates corresponding to point P (a, b, sj) in the third map P3: