WO2018035736A1

WO2018035736A1 - Display method and device for intelligent glasses

Info

Publication number: WO2018035736A1
Application number: PCT/CN2016/096489
Authority: WO
Inventors: 付楠; 朱艳春; 余绍德; 陈昳丽; 张志成; 谢耀钦; 王磊
Original assignee: 中国科学院深圳先进技术研究院
Priority date: 2016-08-24
Filing date: 2016-08-24
Publication date: 2018-03-01

Abstract

A display method and device for intelligent glasses. The method comprises: adjusting and demarcating a display area on a display screen of the intelligent glasses, so that the display area is the same as an outside area seen by a user through the intelligent glasses (S401); adjusting the fixed position of an optical instrument on the intelligent glasses, so that first image data obtained by the optical instrument and second image data seen by the user through the intelligent glasses are the same in size (S402); and adjusting the display position of the first image data in the display area according to the second image data, so that the first image data and the second image data are overlapped and the first image data is displayed (S403).

Description

Display method and device for smart glasses

Technical field

The invention relates to the field of smart wear, in particular to a display method and device for smart glasses.

Background technique

The current smart glasses can be divided into occlusion type and transmission type from the display principle. The typical representative of the occlusion type is the Oculus series. This device obscures the natural field of view and covers the field of view with the displayed image. This device completely obscures the normal field of view without energy. The typical representative of the transmissive type is googleglass and Epson Moverio BT-200 smart glasses and Microsoft's hololens. The device itself does not obscure the natural field of view. The image will only overlap with some images of the natural field of view, and in the absence of energy. The normal field of view will not be blocked. The technical means of this paper are mainly aimed at transmissive smart glasses, and the display of this type of smart glasses does not completely cover the natural field of view. The current mainstream method is to completely display the image captured by the camera on the lens display. As shown in Fig. 1, the solid line is the image seen by the human eye through the lens in the real scene, and the dotted line is the image on the lens display. The image on the lens display is only the image captured by the camera, so that the virtual image in the field of view will interfere with the real image and affect the user experience.

A disadvantage of the prior art is that it appears visually that the displayed image will overlap with the real image, as shown in Figure 1, and there is no correspondence in this overlap. The human eye sees an image like a picture-in-picture with repeated content. In this way, the image information will be mixed, which affects people's acquisition of normal information. At the same time, since the display window only occupies a small part of the field of view, the amount of information displayed and the content are limited and the content is not clear enough.

Summary of the invention

The object of the present invention is to correlate the displayed content with the content of the actual visual field, so as not to obscure the content of the natural visual field and supplement the corresponding information in the natural visual field, as shown in the display mode of FIG. 2, the displayed content. It can be completely coincident with the actual image actually observed, so that the displayed information is more intuitive and can accurately correspond to the real object. At the same time, the information we add on the virtual object can also correspond to the real object more accurately, and improve the accuracy of the displayed information. Since the ratio of the virtual image to the real image is 1:1, it can be better. The details of the image are guaranteed.

In order to solve the above problems, the present invention specifically provides a display method for smart glasses, the method comprising: adjusting and calibrating a display area on a display screen of the smart glasses, so that the display area and a user pass through the smart glasses The outer area is consistent; the fixed position of the optical instrument on the smart glasses is adjusted, so that the first image data acquired by the optical instrument is consistent with the size of the second image data that the user sees through the smart glasses; Adjusting a display position of the first image data in the display area according to the second image data, overlapping the first image data with the second image data, and displaying the first image data.

In the above display method of the smart glasses, preferably, the adjusting and calibrating the display area on the display screen of the smart glasses comprises: by using a calibration plate that observes a predetermined threshold outside the display screen, using a checkerboard calibration method or a target The shape calibration method calibrates the display area on the display screen.

In the above display method of the smart glasses, preferably, after the displaying the first image data, the method further comprises: monitoring an observation distance between the image collection end of the optical instrument and the collected physical object, when the observation distance is less than When the threshold is predetermined, the first image data displayed on the display screen is enlarged or reduced according to a predetermined ratio.

In the above display method of the smart glasses, preferably, the overlapping the first image data with the second image data comprises: observing and adjusting a display position of the first image data by a user to make the first image The data overlaps with the second image data; or, the human eye vision is simulated by a sampling optical device set in advance, and the second image data obtained by the sampling optical device is matched and overlapped with the first image data.

In the above display method of the smart glasses, preferably, the simulating the human eye vision by the preset sampling optical device and matching the second image data obtained by the sampling optical device with the first image data includes: fixing the Second image data, the first image data is laterally moved or longitudinally moved in a predetermined pixel unit, and the similarity between the moving first image data and the second image data is obtained by comparison a degree value; obtaining the similarity matrix by the plurality of lateral movements or longitudinally moving the first image data; and determining, according to the similarity matrix, determining a lateral movement or a longitudinal movement of the first image data Moving the distance; moving the first image data according to the moving distance.

Preferably, after the displaying the first image data, the method further includes: acquiring an observation distance between the user's glasses and the display screen, and adjusting the predetermined parameter according to the observation distance. The display of the size and/or display position of the first image data is output on the display screen.

In the above display method of the smart glasses, preferably, after the displaying the first image data, the method further includes: monitoring a control command input by the user, and adjusting the first image data according to the control command to the display screen The output is displayed.

In the above display method of the smart glasses, preferably, the monitoring control command input by the user, after adjusting the first image data according to the control command, outputting the display on the display screen includes: according to the control instruction, Monitoring the displacement condition of the smart glasses, and expanding or reducing the first image data according to the displacement condition, and outputting the display on the display screen.

In the above display method of the smart glasses, preferably, the monitoring the control command input by the user, and the outputting the display on the display screen after adjusting the first image data according to the control command further comprises: monitoring the first Whether the calibration image data is included in the image data, and if the calibration image data is included, the monitoring image including the calibration image data in the first image data is intercepted; and the monitoring image is compared with the model image in the database according to the control instruction. Obtaining a calibration model corresponding to the calibration image data; adding the calibration model to the first image data, and adjusting a size and a display position of the calibration model; the first to be superimposed with the calibration model Image data is displayed on the display screen.

In the above display method of the smart glasses, preferably, the distance between the glasses of the user and the display screen is greater than or equal to 0.5 cm.

The present invention also provides a display device for smart glasses, the display device comprising a calibration module, a model adjustment module and a displacement adjustment module; the calibration module is configured to adjust and calibrate a display area on a display screen of the smart glasses, so that The display area is consistent with an external area that the user sees through the smart glasses; the model adjustment module is configured to adjust the first obtained by the optical instrument according to different fixed positions of the optical instrument on the smart glasses. Image data, such that the first image data is consistent with a size of the second image data that the user sees through the smart glasses; the displacement adjustment module is configured to adjust the first image data according to the second image data The display position in the display area overlaps the first image data with the second image data and displays the first image data.

In the above display device of the smart glasses, preferably, the display device further includes a distance determining module, the distance determining module is configured to acquire an observation distance between the user's glasses and the display screen, and collect the optical instrument Obtaining an observation distance between the image acquisition end and the collected physical object, or determining an observation distance between the image acquisition end of the optical instrument and the collected physical object according to the first image data.

The beneficial technical effect of the present invention is that the display method of the smart glasses can display information corresponding to the actual scene on the smart glasses in a proportionally corresponding position, so that it is not easy to block the visual field, and no information confusion can be generated, and more details can be displayed. Ensure the correspondence and accuracy of the information display; at the same time, with the cooperation of the ranging function, it can provide better compensation, ensure the use effect at close distance, and according to the corresponding relationship, the user can obtain more real scene and illusion through the smart glasses. A new experience of collection.

The above and other objects, features, and advantages of the present invention will become more apparent and understood by the appended claims appended claims

DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below. Obviously, the drawings in the following description are only It is a certain embodiment of the present invention, and other drawings can be obtained from those skilled in the art without any creative work.

1 is a schematic diagram of a display screen of a smart glasses in the prior art;

2 is a schematic diagram of a screen displayed by a display method of smart glasses according to the present invention;

3 is a schematic diagram showing the effect of the display method of the smart glasses provided by the present invention;

4 is a schematic flow chart of a display method of smart glasses according to the present invention;

FIG. 5 is a schematic diagram showing the principle of a display method of smart glasses according to a preferred embodiment of the present invention.

detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

Referring to FIG. 2 and FIG. 4, the present invention provides a display mode of smart glasses, which can visually observe the natural field of view and the ratio of objects seen through virtual imaging is 1:1, and simultaneously The position of the image also coincides with the image position of the natural field of view to ensure accurate display of the detail information; the display mode mainly includes the following implementation steps: S401 adjusts and calibrates the display area on the display screen of the smart glasses, so that the display area Aligning with an external area that the user sees through the smart glasses; S402 adjusts a fixed position of the optical instrument on the smart glasses, so that the first image data acquired by the optical instrument and the user see through the smart glasses The second image data arrives at the same size; S403 adjusts the display position of the first image data in the display area according to the second image data, so that the first image data overlaps with the second image data and displays The first image data is described.

The main purpose of the above step S401 is to calibrate the size of the display area of the display screen, specifically by observing the board calibration plate from the predetermined threshold outside the display screen, and calibrating the display on the display screen by the checkerboard calibration method or the target calibration method. Display area; for example: to perform the calibration of the display area, the calibration method uses a checkerboard, where the checkerboard The size of the fixed plate should be determined according to the size of the display window of the display. The size can be less than 20 times the size of the display window, as close as possible to 20 times. If the size exceeds 20 times, the calibration plate may be incompletely displayed. If the calibration plate is too small, the calibration error will be increased. Observe the checkerboard of the checkerboard through the display screen, and pull away the calibration plate at the same time. The farther the distance is, the better, at least 20 times the distance e of the human eye from the display screen. Under normal circumstances, we think that e=1cm, then the distance should be more than 20cm. Therefore, the error is guaranteed to be less than 5%. At the same time, if the calibration plate is too far, the calibration plate cannot fill the display screen, and the calibration purpose cannot be achieved. It is better to completely display the entire calibration plate in the transparent display screen, thereby effectively reducing Error rate.

In the above step S402, the size of the display content is mainly calibrated, so that the size ratio of the display content is consistent with the personnel observation ratio, thereby facilitating the coincidence of the later images, so that the size of the displayed object and the human eye seen by the user through the display screen are The normal observation size is the same; in the process, the human eye can be directly calibrated, or the camera can be used to collect data instead of the human eye for calibration; the former method, that is, the human eye directly performs calibration, specifically by displaying the first image data. On the display screen, while the user observes, the image display scale and size are adjusted autonomously. The process is mainly determined by the human eye and then adjusted by the external control device or other means to adjust the first image data displayed on the display screen. The latter method for calibrating the data collected by the camera in place of the human eye is mainly for illuminating the position of the human eye through the display screen for image acquisition, and the collected image can be regarded as an image seen by the human eye, and the size of the display content is adjusted. The method of consistent ratio is to use the virtual content without displaying it. The camera is sampling. This camera is called a sampling camera. It is used for calibration. It records the size of the sampled content (how many pixels can be used), and then occludes the display to ensure that the sampling camera can only collect The content of the display (the content does not include any real world of transmission) and display the virtual image captured by the camera (device camera) on the display screen, while gradually zooming in or out the image, the sampling camera records the process of transformation, and The process data is compared with the previously collected physical content, and the zoom-in and zoom-out parameters corresponding to the image with the closest number of pixels are recorded, and it is considered that the image is consistent with the physical size of the human eye, and the pixel variation range is plus or minus 20 The % interval is reserved for subsequent user fine-tuning. The above two methods can be selected according to the actual situation. Of course, the ratio of the size of the display content is not only the above two methods, but the above method is only a clearer description of the present invention when the captured image is consistent with the image seen by the human eye. The process plan adopted does not impose any restrictions on its specific implementation process.

In a preferred embodiment of the present invention, the step S403 may further include: superimposing the first image data and the second image data by the user viewing and adjusting the display position of the first image data; or by setting in advance The sampling optical device simulates human eye vision and matches the second image data obtained by the sampling optical device with the first image data. The method for simulating a human eye by using an optical device further includes: fixing the second image data, moving the first image data laterally or vertically in a predetermined pixel unit, and recording the moved first image data and The similarity value between the two obtained by the comparison of the second image data; by multiple lateral movements or longitudinal movements And moving the first image data to obtain the similarity value to form a similarity matrix; according to the similarity matrix, obtaining a moving distance for determining a lateral movement or a longitudinal movement of the first image data; and moving the movement according to the moving distance The first image data is described. Specifically, in actual use, the similarity value obtained by first comparing the second image with the first image is used to form a similarity matrix. For example, only one value is obtained for each movement, so that the movement in multiple horizontal and vertical directions is performed. Multiple values can be obtained, and then the value constitutes a matrix. Finally, the normal distribution (one-dimensional) and the Gaussian distribution (two-dimensional) are used as constraints, and the extreme value of the Gaussian distribution function of the matrix is solved as the final extreme result. And determining, by the extreme value result, a moving distance of the first image data to move laterally or vertically.

In the above embodiment, the image rearrangement is performed, that is, the image display position is adjusted, and the flow is similar to the calibration process of step S402. The image can be collected by the camera to simulate the position of the human eye through the display screen, and the image is collected. It can be thought of as an image seen by the human eye. At this time, the projected image is compared with the sampled real image. The coverage similarity value of the projected image in the real image satisfies the Gaussian distribution (where the similarity can be the value of the area overlap or the pixel value is interpolated, etc.) The present invention will not be exemplified herein. In this case, the corresponding extreme value of the Gaussian distribution function can be obtained. It is considered that the extreme point is a point overlapping the actual position, and the overall display image around the point is 10% in width. The vertical height is 10% as a range for fine adjustment by the user.

In a preferred embodiment of the present invention, after the displaying the first image data, the first image data may be further calibrated, so that the user can display the data more when using the smart glasses to observe the close objects. It is clear that the present invention also provides a processing method for monitoring an observation distance between an image acquisition end of the optical instrument and a collected physical object, and when the observation distance is less than a predetermined threshold, zooming in or out on the display screen according to a predetermined ratio. The first image data displayed thereon; wherein the predetermined threshold value may be set according to an actual smart glasses device condition, and may generally be set to a distance of 20 times from the display screen of the human eye, when the human eye or the optical instrument observes a close distance The scene is often accompanied by defocusing. In this case, adjusting the display data of the optical instrument according to the preset threshold range can effectively help the user to observe the close physical object more clearly; for example, when the physical object is actually observed, when the smart glasses detect the physical object When the optical instrument acquisition end is below a threshold, the image can be compensated according to the formula c=a(1+e/b). The distance from the human eye to the imaging is a, the distance from the human eye to the real object is b, the distance from the camera to the imaging is c, the distance from the camera to the real object is d, the distance from the human eye to the display screen is e; e is the default constant. That is to say, the size of the displayed image is only related to b, then we can adjust the size of the displayed image according to the measured distance b, and automatically enlarge the image when b is reduced, so as to ensure the accuracy of the image display. For example, the device itself has a ranging function, which means that the b value can be obtained here. It is also known if the distance measuring device on the eye side of the device is the e value. Then c=a(1+e/b)(1+e/b) can get the specific value m, that is, c=am at this time, that is, the ratio of the image on the display screen to the actual image is m, assuming The scale value obtained by the previous calibration is n, then the required scaling value is m/n, that is, the existing image is reduced by n times and then expanded by m times. The actual operation is usually performed by directly using the specific value of m/n. If the human eye side distance e is unknown, the default e is a constant (this value does not change in general), and the standard calibration n = (1 + e / b) can get the e value. The core application of the ranging of b is that the ratio of this zoom-in and zoom can vary with the distance of the object being observed, thereby avoiding distortion.

It should be noted that, in general, the virtual image is completely coincident with the image seen by the human eye through the crystal. However, according to the principle based on the formula c=a(1+e/b), the matching of the image can only be achieved when b is large enough and e is small enough; therefore, if the smart glasses do not have the ranging function themselves, then e is limited. A certain effective range (0.5cm-2cm), that is, the distance between the human eye and the display screen needs to be at least 0.5 cm or more to ensure the display effect.

In the above embodiment, the image data captured by the optical instrument is displayed to be consistent with the human eye observation scene, and the image data to be captured is required to be processed, and the distance between the human eye and the display screen is involved, the camera and the physical object are Distances and the like, in order to more clearly illustrate the flow and principle of the present invention, the principle of the present invention is further explained by the following FIG. 3; in the process of using smart glasses, the human eye sees the physical image through the wafer, if To achieve an image size of 1:1, and the positions are coincident, the image that is bound to be displayed needs to be consistent with the lens image. That is to say, the image of the object observed by the human eye through the lens should be consistent with the image captured by the camera and displayed on the display. Of course, the image captured by the camera is an inverted real image, and here is equivalent to the erect virtual image in front; In most cases, the image captured by the camera is not consistent with the object seen by the human eye. Therefore, in order to ensure that the ratio of the display image to the actual object observed by the human eye is consistent, the image to be displayed is further corrected; The distance from the human eye 302 to the imaging 3021 needs to be set to a, the distance from the human eye 302 to the physical object 301 is b, the distance from the camera 303 to the imaging 3022 is c, and the distance from the camera 303 to the physical object 301 is d; then, in order to make the camera image size Consistent with the human eye observation lens imaging size, the formula a/b=c/d must be satisfied. Here, the zooming c is variable by the focus distance and the image. b and d satisfy the relationship d=b+e, assuming that the camera 303 is in line with the human eye 302 and the object 301, then e is the distance between b and d. So the formula can be changed to a/b=c/(b+e). The formula c = a (1 + e / b) is obtained. In this formula, a is a constant, because the example of human eye and lens imaging can be considered to be constant; e is also constant, because the distance between camera 303 and human eye 301 can also be considered constant during wearing smart glasses. Of course, the general camera should be in front of the human eye, that is, the example of d should be smaller than the example of b, that is, e<0 here. Thus we know that the variable c has only a certain relationship with b and satisfies the formula c=a(1+e/b), and a, e is constant and known. This formula also shows that if the observed object is infinity, the e/b is close to 0, c=a, so the image captured by the camera should be consistent with the human eye to observe the lens imaging. In other words, if e is small enough and b is large enough, we think c=a, which is the size of the camera image. It can be considered that the distance between the human eye and the object is not considered. In general, the distance of e is less than or equal to 1cm, which means that if the object is at 20cm, the error should be around 5%. If the object has an error of 2% at 50 cm, the error above 100 cm is less than 1%. Of course, we can use the formula to get the exact size of the generated virtual image. Therefore, through the above method, after the actual distance measurement or other ranging method of the smart glasses can be used to understand the b value of the corresponding parameter, it can be based on the above formula. The image is adaptively adjusted to ensure the accuracy of the display of c; the ranging function can also be limited to when the distance of the object is close to a certain extent and then compensated using the formula algorithm, and in other cases, c=a.

Of course, in the above embodiment, the value of c can be changed by adjusting the parameters of the camera, but the c value cannot be directly set in the actual work, and the size of the displayed image cannot be directly set. Therefore, the present invention can By changing the parameters of the camera or the size of the displayed image to achieve the above purpose, based on the purpose, a certain correction method is needed to initialize the relationship between the displayed content and the content captured by the camera; in order to ensure the accuracy of the correction, the target can be The object (calibration plate, where the calibration plate is generally a black and white grid plate similar to a chess board) is placed 1 meter away, and the number of cells on the calibration plate is observed through the display wafer, and the calibration plate is moved so that it is obtained in the display window. The number of grids is exactly an integer (because there is no distortion in the image itself, so choose horizontal or vertical integers); keep the position of the smart glasses and the calibration plate unchanged, open the camera, display the shooting content, and adjust the display content to With the same number of squares, we can think that the proportion of the image displayed at this time is observed with the human eye. The image ratio is 1 to 1; on this basis, the displayed image is panned or horizontally moved to align its position, or the target calibration plate can be re-calibrated so that the displayed content coincides with the human eye.

Due to the difference of the user, the distance of a is likely to change, and the relative position of the human eye and the display wafer is not the same as the position of the calibration due to the wearing relationship, so if necessary, the user is required to wear Performing a second adjustment, that is, further image calibration; the display method of the smart glasses provided by the present invention further includes: obtaining an observation distance between the user's glasses and the display screen, and adjusting the first parameter according to the observation distance according to the predetermined parameter A display is output on the display screen after the size and/or display position of an image data. In this way, the same observation experience is effectively given to different users through the solution, and the first image data is purposefully adjusted based on different distances from different screens when the smart glasses are worn by different users, so that the accurate and real images are second. The image data is completely overlapped.

In the above embodiment, the predetermined parameter may be pre-tested and recorded with different observation distances, and the first image data needs to be adjusted, so that when the subsequent user actually wears the smart glasses, the first is adjusted according to the observed distance. The image data is obtained in a manner that is not described in detail herein; of course, the first image data can be adjusted by the user. In a preferred embodiment of the present invention, a control command input by the user is monitored. The control command outputs the display on the display screen after adjusting the first image data; that is, the user can autonomously adjust the first image data according to a specific feeling or a parameter that is known in advance; for example, the smart glasses are set. When the distance e between the human eye and the display screen is set to 1 cm, and the distance between the human eye of a certain user and the display screen is 2 cm, the first image data can be adjusted by the parameter amount corresponding to 2 cm. The purpose of overlapping the first image data with the second image data; if the user cannot know the parameter amount, the distance between the human eye and the display screen cannot be accurately determined. At this time, the present invention further provides monitoring the smart according to the control instruction. In the displacement condition of the glasses, the first image data is enlarged or reduced according to the displacement condition, and then displayed on the display screen. For example, a gyroscope is used as a translation input device, and the gyroscope is generally integrated in the smart glasses to collect the posture of the human head. When the control command is received when calibration is required, the current gyroscope is first recorded. Instrument data, after the user observes the real image and the virtual image in the display wafer, rotate the smart glasses in a certain direction; for example, if the virtual image does not coincide with the real image, and the virtual image is above the real image, then the The virtual image moves down as a whole. At this time, the user needs to do the operation, so that the gyroscope will record the head direction deviating from the horizontal direction, and feedback the information to move the overall display content downward. The operations in other directions are similar. After calibration, the image captured by the camera and displayed on the display can achieve the effect of Figure 2, the information is displayed more accurately, and the details of the display content are retained. It will be apparent to those skilled in the art that the gyroscope is only a preferred technical solution for the user to adjust the first image data to explain the user more clearly. The present invention does not limit the user's self-adjustment of the first image data. Equipment and methods.

Based on the above physical object and image corresponding display, the present invention further provides a preferred embodiment for simply explaining the application scenario and real-time flow of the display method. After the first image data and the second image data are calibrated, The image displayed by the smart glasses is consistent with the real situation. At this time, the smart glasses further monitor whether the first image data includes calibration image data, and if the calibration image data is included, the first image data is included to include the calibration image. Monitoring the picture of the data; comparing the monitoring picture with the model picture in the database according to the control instruction, obtaining a calibration model corresponding to the calibration image data; adding the calibration model to the first image data, and Adjusting the size and display position of the calibration model; and displaying the first image data superimposed with the calibration model on the display screen. For example, the user may write a predetermined identifier, such as a text, a symbol or a pattern, on the external physical object or other marking method; when the smart glasses monitor the first image data to include the identification information, the image is intercepted and The cloud database compares and determines the type of the object that is calibrated by the user. The method for determining the image recognition can be implemented by using the prior art. The present invention is not described in detail herein; after determining the user calibration object, the database is searched. The model data of the model of the object is confirmed by the user, and the model data is superimposed to the first image data display for the user to refer to when the user confirms; in the process, the model data can be monitored according to the detection device according to the detection device in the early stage. The internal structure of the object is constructed, and can also be constructed according to a specific drawing or description in a computer or the like. By this method, the user can superimpose the model data to be referenced into the real display interface. For reference judgment, for example, during the operation of the doctor, the designated symbol can be calibrated on the patient's body. When the smart glasses monitor the symbol, the multi-class model of the corresponding human body is obtained from the database, and the doctor selects the human artery fluoroscopy model or skeleton. An image such as a perspective model. At this time, the smart glasses adjust the model according to the physical proportion and superimpose and display the image in the first image data, and the doctor can accurately determine the physical condition of the user by using the display content, and provide more accurate operations such as surgery; In other words, when the construction workers are performing renovations on the house, in order to ensure that the construction does not affect the safety of the house, damage to the load-bearing wall, internal lines, etc., it is necessary to study the relevant drawings of the house; For example, the interior model of the house superimposed by the smart glasses can accurately and timely understand the situation of the house during construction, which greatly saves the construction personnel's time and ensures the personal safety of the construction workers during the construction process; The invention can be used in many scenes of life, the invention is no longer here An illustration.

Referring to FIG. 5, in a preferred embodiment of the present invention, a schematic diagram of a principle presented to a user when using the smart glasses display method provided by the present invention is given. In FIG. 5, the user is in the initial stage, and is normal. The observed scene is a palm, and then the smart glasses are worn to further understand the bone condition of the palm. At this time, the smart glasses superimpose the palm bone perspective obtained by CT shooting in the previous stage to the palm display. What can be seen is the perspective view of the palm of the hand taken by the CT, and the original palm is covered and no longer displayed; it is worth noting that the display method of the smart glasses provided by the present invention only superimposes the display content that needs to be changed to In the original object, it does not change or interfere with other objects; for example, when the object in front of the user is the whole arm, and the user only needs to understand the bone condition of the palm, then the display content is covered in the original palm position to display the bone situation map. Without making any changes to the arm; or, when the house is being renovated, to show the line in the wall more clearly, Wall was cut position as seen by the user comprising the glasses trend line displayed line situation, while other portions, such as local wall outside the line remains undisturbed, normal display.

It will be known to those skilled in the art that the image observed by one eye is a planar image, and only the binocular observation can obtain the depth of field. Therefore, when the user wears the smart glasses, in order to ensure that the image observed by the user is consistent with the real image, the present invention uses at least two optical devices. After obtaining different image data respectively, the processing is performed, and then the output is displayed on the left and right display screens of the smart glasses respectively, so that the later users can obtain a more realistic experience when viewing the smart glasses. At the same time, the display screen of the smart glasses provided by the present invention is a transparent display panel capable of displaying a picture, so that the user can observe the external actual situation through the smart glasses when using or not using the smart glasses.

The display method of the smart glasses provided by the invention can display the information corresponding to the actual scene on the smart glasses in an equal proportion corresponding position, so that the visual field is not easily blocked, the information confusion is not generated, more details can be displayed, and the correspondence of the information display is ensured. And accurate; at the same time, with the cooperation of the ranging function, it can provide better compensation and ensure close proximity Out-of-time usage and the new experience of giving users more real and illusory collections through smart glasses based on this correspondence.

In the above embodiment, the display device provided by the present invention may further include a distance determining module, configured to acquire an observation distance between the user's glasses and the display screen, and acquire an image of the optical instrument. Obtaining an observation distance between the terminal and the collected physical object, or determining, according to the first image data, an observation distance between the image acquisition end of the optical instrument and the collected physical object; the distance determination module accurately acquiring various distance parameters The smart glasses can be adjusted to more accurately and effectively adjust the first image data according to the parameters.

Those skilled in the art will appreciate that embodiments of the present invention can be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or a combination of software and hardware. Moreover, the invention can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.

The present invention has been described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (system), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or FIG. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine for the execution of instructions for execution by a processor of a computer or other programmable data processing device. Means for implementing the functions specified in one or more of the flow or in a block or blocks of the flow chart.

The computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device. The apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.

These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device. The instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

The principles and embodiments of the present invention have been described in connection with the specific embodiments of the present invention. The description of the above embodiments is only for the understanding of the method of the present invention and the core idea thereof. At the same time, for those skilled in the art, according to the present invention The present invention is not limited by the scope of the present invention.

Claims

A display method for smart glasses, characterized in that the method comprises:

Adjusting and calibrating a display area on a display screen of the smart glasses such that the display area is consistent with an outer area that the user sees through the smart glasses;

Adjusting a fixed position of the optical instrument on the smart glasses, so that the first image data acquired by the optical instrument is consistent with the size of the second image data that the user sees through the smart glasses;

Adjusting, according to the second image data, a display position of the first image data in the display area, overlapping the first image data with the second image data, and displaying the first image data.
The display method of the smart glasses according to claim 1, wherein the adjusting and calibrating the display area on the display screen of the smart glasses comprises: by observing a calibration plate that is outside a predetermined threshold from the display screen, The checkerboard calibration method or the target calibration method calibrates the display area on the display screen.
The display method of the smart glasses according to claim 1, wherein the displaying the first image data further comprises: monitoring an observation distance between the image collection end of the optical instrument and the collected physical object, when When the observation distance is less than a predetermined threshold, the first image data displayed on the display screen is enlarged or reduced according to a predetermined ratio.
The method for displaying smart glasses according to claim 1, wherein the overlapping the first image data with the second image data comprises:

The first image data and the second image data are overlapped by the user viewing and adjusting the display position of the first image data;

Or, the human eye vision is simulated by a sampling optical device set in advance, and the second image data obtained by the sampling optical device is matched and overlapped with the first image data.
The display method of the smart glasses according to claim 4, wherein the simulation of the human eye vision by the sampling optical device set in advance and the matching of the second image data obtained by the sampling optical device with the first image data contain:

Fixing the second image data, moving the first image data laterally or vertically in a predetermined pixel unit, and recording the two obtained by comparing the moved first image data and the second image data Similarity value between

Obtaining the similarity value to form a similarity matrix by moving the first image data a plurality of times laterally or longitudinally;

Determining, according to the similarity matrix, a moving distance that determines a lateral movement or a longitudinal movement of the first image data;

The first image data is moved according to the moving distance.
The display method of the smart glasses according to claim 1, wherein the displaying the first image data further comprises: acquiring an observation distance between the glasses of the user and the display screen, according to the observation distance The display is output on the display screen after adjusting the size and/or display position of the first image data according to a predetermined parameter.
The display method of the smart glasses according to claim 1, wherein the displaying the first image data further comprises: monitoring a control command input by the user, and adjusting the first image data according to the control command The display is output on the display screen.
The display method of the smart glasses according to claim 7, wherein the monitoring the user-entered control command, adjusting the first image data according to the control command, and outputting the display on the display screen comprises: The control instruction monitors a displacement condition of the smart glasses, and enlarges or reduces the first image data according to the displacement condition, and then outputs the display on the display screen.
The display method of the smart glasses according to claim 7, wherein the monitoring the control command input by the user, adjusting the first image data according to the control command, and outputting the display on the display screen further comprises:

Monitoring whether the first image data includes calibration image data, and if the calibration image data is included, intercepting the monitoring image that includes the calibration image data in the first image data;

And comparing the monitoring picture with the model picture in the database according to the control instruction, to obtain a calibration model corresponding to the calibration image data;

Adding the calibration model to the first image data, and adjusting a size and a display position of the calibration model;

The first image data superimposed with the calibration model is outputted on the display screen.
The display method of the smart glasses according to claim 1, wherein a distance between the glasses of the user and the display screen is greater than or equal to 0.5 cm.
A display device for smart glasses, characterized in that the display device comprises a calibration module, a model adjustment module and a displacement adjustment module;

The calibration module is configured to adjust and calibrate a display area on a display screen of the smart glasses, such that the display area is consistent with an external area that the user sees through the smart glasses;

The model adjustment module is configured to adjust the first image data obtained by the optical instrument according to different fixed positions of the optical instrument on the smart glasses, so that the first image data and the user see through the smart glasses The second image data arrives at the same size;

The displacement adjustment module is configured to adjust a display position of the first image data in the display area according to the second image data, overlap the first image data with the second image data, and display the first Image data.
The display device of the smart glasses according to claim 11, wherein the display device further comprises a distance determining module, wherein the distance determining module is configured to acquire an observation distance between the glasses of the user and the display screen, and Obtaining an observation distance between the image acquisition end of the optical instrument and the collected physical object, or determining an observation distance between the image collection end of the optical instrument and the collected physical object according to the first image data.