US20180150998A1 - Light field light source orientation method for augmented reality and virtual reality and front-end device - Google Patents
Light field light source orientation method for augmented reality and virtual reality and front-end device Download PDFInfo
- Publication number
- US20180150998A1 US20180150998A1 US15/381,137 US201615381137A US2018150998A1 US 20180150998 A1 US20180150998 A1 US 20180150998A1 US 201615381137 A US201615381137 A US 201615381137A US 2018150998 A1 US2018150998 A1 US 2018150998A1
- Authority
- US
- United States
- Prior art keywords
- light source
- end device
- brightness
- data
- certain object
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
- G06F3/147—Digital output to display device ; Cooperation and interconnection of the display device with other functional units using display panels
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/06—Ray-tracing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/50—Lighting effects
- G06T15/506—Illumination models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
- G09G5/02—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the way in which colour is displayed
- G09G5/026—Control of mixing and/or overlay of colours in general
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
- G09G5/10—Intensity circuits
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2215/00—Indexing scheme for image rendering
- G06T2215/16—Using real world measurements to influence rendering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2219/00—Indexing scheme for manipulating 3D models or images for computer graphics
- G06T2219/20—Indexing scheme for editing of 3D models
- G06T2219/2012—Colour editing, changing, or manipulating; Use of colour codes
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G2320/00—Control of display operating conditions
- G09G2320/02—Improving the quality of display appearance
- G09G2320/0242—Compensation of deficiencies in the appearance of colours
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G2320/00—Control of display operating conditions
- G09G2320/06—Adjustment of display parameters
- G09G2320/0626—Adjustment of display parameters for control of overall brightness
- G09G2320/0646—Modulation of illumination source brightness and image signal correlated to each other
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G2320/00—Control of display operating conditions
- G09G2320/06—Adjustment of display parameters
- G09G2320/0666—Adjustment of display parameters for control of colour parameters, e.g. colour temperature
Definitions
- the present invention relates to the technical field of computer simulation, in particular to a light field light source orientation method and system for augmented reality and virtual reality.
- augmented reality at the present stage a virtual object and a light field of a reality environment lack of direct interaction and corresponding adjustment to achieve the reality of the object, including change of an object shadow and integration degree of an object surface. Interaction with the reality environment can only be conducted through two modes: a hardware photo sensor or GPS and position data.
- the augmented reality at the present stage has the following disadvantages:
- the main purpose of the present invention is to provide a light field light source orientation method for augmented reality and virtual reality and a front-end device.
- the present invention can collect surrounding environmental factors, such as the light source direction, so as to project a computer-generated object into a reality environment to possess a shadow consistent with that in the reality, so that an augmented reality environment is more realistic and restores a real effect of the object.
- the present invention adopts the following technical solution:
- the present invention provides a light field light source orientation method for augmented reality and virtual reality, comprising steps:
- A. identifying a target mark as follows: collecting image data through a front-end device and identifying the collected image data;
- tracking the target mark as follows: tracking the position of a certain object in the image data by the front-end device; obtaining the surface shape and feature of the certain object, and locking a pixel color disc of the certain object through combined establishment of algorithm binding and engine matching;
- D. analyzing a grey zone on the certain object as follows: analyzing, by the front-end device, a BLOB of the certain object, judging a refraction angle, length and brightness of the grey zone of the certain object according to the BLOB, and determining a light source direction in a place where the front-end device is located, a distance between the front-end device and a light source, and the intensity of the light source according to the refraction angle, the length and the brightness of the grey zone;
- E. pushing the data to a light field of a virtual object as follows: pushing the data into the light field of the virtual object after converting the data into identifiable data of the front-end device according to the data value of the pixel color disc of the certain object, the light source direction, the distance between the front-end device and the light source and the intensity of the light source; and
- F. compensating and adjusting the imaging of the virtual object adjusting the virtual object and a shadow of the virtual object through the data value of the pixel color disc of the certain object, the light source direction, the distance between the front-end device and the light source and the intensity of the light source, so that a virtual scenario is more realistic and clear.
- the data value of the pixel color disc comprises: hue-color essential attributes, saturation-color purity of 0-100% value and lightness and brightness of 0-100% value.
- step D is as follows: converting the refraction angle, the length and the brightness into identifiable codes of the front-end device; determining the light source direction in the place where the front-end device is located according to identification codes of the refraction angle; determining the distance between the front-end device and the light source according to identification codes of the length of the grey zone; and determining the intensity of the light source according to identification codes of the brightness of the grey zone.
- the front-end device is a mobile phone or helmet glasses.
- the method also comprises steps of adjusting brightness, hue and saturation of the virtual object through the data value of the pixel color disc of the certain object, the light source direction, the distance between the front-end device and the light source and the intensity of the light source; adjusting the refraction angle, the length and the brightness of the shadow of the virtual object according to the light source direction, the distance between the front-end device and the light source and the intensity of the light source; and the virtual scenario is more realistic and clear.
- Embodiments of the present invention also provide a front-end device, comprising:
- a target mark identifying unit used for collecting image data and identifying the collected image data
- a target mark tracking unit used for tracking the position of a certain object in the image data, obtaining the surface shape and feature of the certain object, and locking a pixel color disc of the certain object through combined establishment of algorithm binding and engine matching;
- a pixel color disc analyzing unit used for collecting the locked pixel color disc of the certain object, and analyzing and obtaining a data value of the pixel color disc
- a grey zone analyzing unit used for analyzing a BLOB of the certain object, judging a refraction angle, length and brightness of the grey zone of the certain object according to the BLOB, and determining a light source direction in a place where the front-end device is located, a distance between the front-end device and a light source, and the intensity of the light source according to the refraction angle, the length and the brightness of the grey zone;
- a data pushing unit used for pushing the data into the light field of the virtual object after converting the data into identifiable data of the front-end device according to the data value of the pixel color disc of the certain object, the light source direction, the distance between the front-end device and the light source and the intensity of the light source;
- a compensation and adjustment unit used for adjusting the virtual object and a shadow of the virtual object through the data value of the pixel color disc of the certain object, the light source direction, the distance between the front-end device and the light source and the intensity of the light source, so that a virtual scenario is more realistic and clear.
- the grey zone analyzing unit is used for converting the refraction angle, the length and the brightness into identifiable codes of the front-end device; determining the light source direction in the place where the front-end device is located according to identification codes of the refraction angle; determining the distance between the front-end device and the light source according to identification codes of the length of the grey zone (shadow); and determining the intensity of the light source according to identification codes of the brightness of the grey zone.
- the data value of the pixel color disc comprises: hue-color essential attributes, saturation-color purity of 0-100% value and lightness and brightness of 0-100% value.
- the compensation unit is used for adjusting brightness, hue and saturation of the virtual object through the data value of the pixel color disc of the certain object, the light source direction, the distance between the front-end device and the light source and the intensity of the light source, and adjusting the refraction angle, the length and the brightness of the shadow of the virtual object according to the light source direction, the distance between the front-end device and the light source and the intensity of the light source; and the virtual scenario is more realistic and clear.
- the front-end device comprises a mobile phone or helmet glasses.
- the method and the device provided by the present invention solve the problems of too high configuration of front-end data collection hardware, high cost of data transmission device and poor user experience of a helmet device of a display end; solve the problems that the virtual object cannot be integrated into the light field of the reality environment, the shadow cannot be accurately tracked, a light ray angle is inaccurate and the object display is floating and is in the air; solve the problem of serious color difference caused by an influence of the light field on the color; and solve the problem of high manufacture cost because the virtual object is only limited to a digitized image and a projected real object.
- the object projected from a virtual augmented environment is consistent with a reality shadow direction, the reflective refraction angle and the brightness are adjusted, and the reality of the projected object in the reality environment is increased.
- the virtual object can be integrated into the light field of the reality environment in real time, so that naked eye experience of a user is closer to the reality.
- FIG. 1 is a flow chart of a method provided by the embodiments of the present invention.
- FIG. 2 is a structural schematic diagram of a device provided by the embodiments of the present invention.
- the embodiments of the present invention provide a light field light source orientation method for augmented reality and virtual reality. As shown in FIG. 1 , the method comprises steps:
- A. identifying a target mark as follows: collecting image data through a front-end device and identifying the collected image data; in the present embodiment, more specifically, the front-end device is a mobile phone or helmet glasses;
- tracking the target mark as follows: tracking the position of a certain object in the image data by the front-end device; the certain object is an object in the image; for example, a bird or a tree on a mountain in a landscape painting can act as a certain object; subsequent description contents in the present embodiment are based on the certain object; obtaining the surface shape and feature of the certain object, and locking a pixel color disc of the certain object through combined establishment of algorithm binding and engine matching;
- the data value of the pixel color disc comprises: hue-color essential attributes (H), saturation-color purity of 0-100% value (S) and lightness (V) and brightness (L) of 0-100% value.
- step D is as follows: converting the refraction angle, the length and the brightness into identifiable codes of the front-end device; determining the light source direction in the place where the front-end device is located according to identification codes of the refraction angle; determining the distance between the front-end device and the light source according to identification codes of the length of the grey zone (shadow); and determining the intensity of the light source according to identification codes of the brightness of the grey zone;
- E. pushing the data to a light field of a virtual object as follows: pushing the data into the light field of the virtual object (an image formed by projecting the certain object) after converting the data into identifiable data of the front-end device according to the data value of the pixel color disc of the certain object, the light source direction, the distance between the front-end device and the light source and the intensity of the light source; and
- F. compensating and adjusting the imaging of the virtual object adjusting the virtual object and a shadow of the virtual object through the data value of the pixel color disc of the certain object, the light source direction, the distance between the front-end device and the light source and the intensity of the light source, so that a virtual scenario is more realistic and clear.
- the step F is as follows: adjusting brightness, hue and saturation of the virtual object through the data value of the pixel color disc of the certain object, the light source direction, the distance between the front-end device and the light source and the intensity of the light source; adjusting the refraction angle, the length and the brightness of the shadow of the virtual object according to the light source direction, the distance between the front-end device and the light source and the intensity of the light source; and the virtual scenario is more realistic and clear.
- the method provided by the above embodiment solves the problems of too high configuration of front-end data collection hardware, high cost of data transmission device and poor user experience of a helmet device of a display end; solve the problems that the virtual object cannot be integrated into the light field of the reality environment, the shadow cannot be accurately tracked, a light ray angle is inaccurate and the object display is floating and is in the air; solve the problem of serious color difference caused by an influence of the light field on the color; and solve the problem of high manufacture cost because the virtual object is only limited to a digitized image and a projected real object.
- the object projected from a virtual augmented environment is consistent with a reality shadow direction, the reflective refraction angle and the brightness are adjusted, and the reality of the projected object in the reality environment is increased.
- the virtual object can be integrated into the light field of the reality environment in real time, so that naked eye experience of a user is closer to the reality.
- Embodiments of the present invention also provide a front-end device. As shown in FIG. 2 , the front-end device comprises:
- a target mark identifying unit used for collecting image data and identifying the collected image data
- a target mark tracking unit used for tracking the position of a certain object in the image data, obtaining the surface shape and feature of the certain object, and locking a pixel color disc of the certain object through combined establishment of algorithm binding and engine matching;
- a pixel color disc analyzing unit used for collecting the locked pixel color disc of the certain object, and analyzing and obtaining a data value of the pixel color disc
- a grey zone analyzing unit used for analyzing a BLOB (binary large object) of the certain object, judging a refraction angle, length and brightness of the grey zone of the certain object according to the BLOB, and determining a light source direction in a place where the front-end device is located, a distance between the front-end device and a light source, and the intensity of the light source according to the refraction angle, the length and the brightness of the grey zone;
- a data pushing unit used for pushing the data into the light field of the virtual object (an image formed by projecting the certain object) after converting the data into identifiable data of the front-end device according to the data value of the pixel color disc of the certain object, the light source direction, the distance between the front-end device and the light source and the intensity of the light source;
- a compensation and adjustment unit used for adjusting the virtual object and a shadow of the virtual object through the data value of the pixel color disc of the certain object, the light source direction, the distance between the front-end device and the light source and the intensity of the light source, so that a virtual scenario is more realistic and clear.
- the grey zone analyzing unit is used for converting the refraction angle, the length and the brightness into identifiable codes of the front-end device; determining the light source direction in the place where the front-end device is located according to identification codes of the refraction angle; determining the distance between the front-end device and the light source according to identification codes of the length of the grey zone (shadow); and determining the intensity of the light source according to identification codes of the brightness of the grey zone.
- the data value of the pixel color disc comprises: hue-color essential attributes (H), saturation-color purity of 0-100% value (S) and lightness (V) and brightness (L) of 0-100% value.
- the compensation unit is used for adjusting brightness, hue and saturation of the virtual object through the data value of the pixel color disc of the certain object, the light source direction, the distance between the front-end device and the light source and the intensity of the light source, and adjusting the refraction angle, the length and the brightness of the shadow of the virtual object according to the light source direction, the distance between the front-end device and the light source and the intensity of the light source; and the virtual scenario is more realistic and clear.
- the front-end device comprises a mobile phone or helmet glasses.
- the device provided by the above embodiment solves the problems of too high configuration of front-end data collection hardware, high cost of data transmission device and poor user experience of a helmet device of a display end; solve the problems that the virtual object cannot be integrated into the light field of the reality environment, the shadow cannot be accurately tracked, a light ray angle is inaccurate and the object display is floating and is in the air; solve the problem of serious color difference caused by an influence of the light field on the color; and solve the problem of high manufacture cost because the virtual object is only limited to a digitized image and a projected real object.
- the object projected from a virtual augmented environment is consistent with a reality shadow direction, the reflective refraction angle and the brightness are adjusted, and the reality of the projected object in the reality environment is increased.
- the virtual object can be integrated into the light field of the reality environment in real time, so that naked eye experience of a user is closer to the reality.
Abstract
Description
- The present application claims the benefit of Chinese patent application No. 201611086214.0 filed on Nov. 29, 2016. All the above are hereby incorporated by reference.
- The present invention relates to the technical field of computer simulation, in particular to a light field light source orientation method and system for augmented reality and virtual reality.
- In augmented reality at the present stage, a virtual object and a light field of a reality environment lack of direct interaction and corresponding adjustment to achieve the reality of the object, including change of an object shadow and integration degree of an object surface. Interaction with the reality environment can only be conducted through two modes: a hardware photo sensor or GPS and position data. The augmented reality at the present stage has the following disadvantages:
- Early 3D data collection of the virtual object has extremely high requirements; and too large data needs to be processed, causing low reaction speed. For a device of a user, such as an AR helmet, a mobile-end device has high hardware demands and high cost. Manufacture of contents is complex, has many conditional limitations and is low in efficiency. Device configuration is enhanced depending on hardware solutions, causing overlarge device and poor user experience.
- The main purpose of the present invention is to provide a light field light source orientation method for augmented reality and virtual reality and a front-end device. The present invention can collect surrounding environmental factors, such as the light source direction, so as to project a computer-generated object into a reality environment to possess a shadow consistent with that in the reality, so that an augmented reality environment is more realistic and restores a real effect of the object.
- To solve the technical problem, the present invention adopts the following technical solution:
- The present invention provides a light field light source orientation method for augmented reality and virtual reality, comprising steps:
- A. identifying a target mark as follows: collecting image data through a front-end device and identifying the collected image data;
- B. tracking the target mark as follows: tracking the position of a certain object in the image data by the front-end device; obtaining the surface shape and feature of the certain object, and locking a pixel color disc of the certain object through combined establishment of algorithm binding and engine matching;
- C. analyzing the pixel color disc on the certain object as follows: collecting the locked pixel color disc of the certain object, and analyzing and obtaining a data value of the pixel color disc;
- D. analyzing a grey zone on the certain object as follows: analyzing, by the front-end device, a BLOB of the certain object, judging a refraction angle, length and brightness of the grey zone of the certain object according to the BLOB, and determining a light source direction in a place where the front-end device is located, a distance between the front-end device and a light source, and the intensity of the light source according to the refraction angle, the length and the brightness of the grey zone;
- E. pushing the data to a light field of a virtual object as follows: pushing the data into the light field of the virtual object after converting the data into identifiable data of the front-end device according to the data value of the pixel color disc of the certain object, the light source direction, the distance between the front-end device and the light source and the intensity of the light source; and
- F. compensating and adjusting the imaging of the virtual object: adjusting the virtual object and a shadow of the virtual object through the data value of the pixel color disc of the certain object, the light source direction, the distance between the front-end device and the light source and the intensity of the light source, so that a virtual scenario is more realistic and clear.
- Preferably, the data value of the pixel color disc comprises: hue-color essential attributes, saturation-color purity of 0-100% value and lightness and brightness of 0-100% value.
- Preferably, step D is as follows: converting the refraction angle, the length and the brightness into identifiable codes of the front-end device; determining the light source direction in the place where the front-end device is located according to identification codes of the refraction angle; determining the distance between the front-end device and the light source according to identification codes of the length of the grey zone; and determining the intensity of the light source according to identification codes of the brightness of the grey zone.
- Preferably, the front-end device is a mobile phone or helmet glasses.
- Preferably, the method also comprises steps of adjusting brightness, hue and saturation of the virtual object through the data value of the pixel color disc of the certain object, the light source direction, the distance between the front-end device and the light source and the intensity of the light source; adjusting the refraction angle, the length and the brightness of the shadow of the virtual object according to the light source direction, the distance between the front-end device and the light source and the intensity of the light source; and the virtual scenario is more realistic and clear.
- Embodiments of the present invention also provide a front-end device, comprising:
- a target mark identifying unit used for collecting image data and identifying the collected image data;
- a target mark tracking unit used for tracking the position of a certain object in the image data, obtaining the surface shape and feature of the certain object, and locking a pixel color disc of the certain object through combined establishment of algorithm binding and engine matching;
- a pixel color disc analyzing unit used for collecting the locked pixel color disc of the certain object, and analyzing and obtaining a data value of the pixel color disc;
- a grey zone analyzing unit used for analyzing a BLOB of the certain object, judging a refraction angle, length and brightness of the grey zone of the certain object according to the BLOB, and determining a light source direction in a place where the front-end device is located, a distance between the front-end device and a light source, and the intensity of the light source according to the refraction angle, the length and the brightness of the grey zone;
- a data pushing unit used for pushing the data into the light field of the virtual object after converting the data into identifiable data of the front-end device according to the data value of the pixel color disc of the certain object, the light source direction, the distance between the front-end device and the light source and the intensity of the light source; and
- a compensation and adjustment unit used for adjusting the virtual object and a shadow of the virtual object through the data value of the pixel color disc of the certain object, the light source direction, the distance between the front-end device and the light source and the intensity of the light source, so that a virtual scenario is more realistic and clear.
- Preferably, the grey zone analyzing unit is used for converting the refraction angle, the length and the brightness into identifiable codes of the front-end device; determining the light source direction in the place where the front-end device is located according to identification codes of the refraction angle; determining the distance between the front-end device and the light source according to identification codes of the length of the grey zone (shadow); and determining the intensity of the light source according to identification codes of the brightness of the grey zone.
- Preferably, the data value of the pixel color disc comprises: hue-color essential attributes, saturation-color purity of 0-100% value and lightness and brightness of 0-100% value.
- Preferably, the compensation unit is used for adjusting brightness, hue and saturation of the virtual object through the data value of the pixel color disc of the certain object, the light source direction, the distance between the front-end device and the light source and the intensity of the light source, and adjusting the refraction angle, the length and the brightness of the shadow of the virtual object according to the light source direction, the distance between the front-end device and the light source and the intensity of the light source; and the virtual scenario is more realistic and clear.
- Preferably, the front-end device comprises a mobile phone or helmet glasses.
- Through implementation of the technical solution of the present invention, the following beneficial effects are generated: the method and the device provided by the present invention solve the problems of too high configuration of front-end data collection hardware, high cost of data transmission device and poor user experience of a helmet device of a display end; solve the problems that the virtual object cannot be integrated into the light field of the reality environment, the shadow cannot be accurately tracked, a light ray angle is inaccurate and the object display is floating and is in the air; solve the problem of serious color difference caused by an influence of the light field on the color; and solve the problem of high manufacture cost because the virtual object is only limited to a digitized image and a projected real object. By detecting the light source direction of the reality environment, the object projected from a virtual augmented environment is consistent with a reality shadow direction, the reflective refraction angle and the brightness are adjusted, and the reality of the projected object in the reality environment is increased. Under the condition of completing light field reduction of the virtual object without a strong front-end data collection device, complicated early manufacture and subsequent terminal display hardware with high cost, the virtual object can be integrated into the light field of the reality environment in real time, so that naked eye experience of a user is closer to the reality.
-
FIG. 1 is a flow chart of a method provided by the embodiments of the present invention; and -
FIG. 2 is a structural schematic diagram of a device provided by the embodiments of the present invention. - Realization of the purpose, functional characteristics and advantages of the present invention will be further described in combination with the embodiments and with reference to the drawings.
- To make the purpose, the technical solution and the advantages of the present invention more clear, the present invention will be further described below in detail in combination with the drawings and the embodiments. It should be understood that specific embodiments described herein are only used for explaining the present invention, not used for limiting the present invention.
- The embodiments of the present invention provide a light field light source orientation method for augmented reality and virtual reality. As shown in
FIG. 1 , the method comprises steps: - A. identifying a target mark as follows: collecting image data through a front-end device and identifying the collected image data; in the present embodiment, more specifically, the front-end device is a mobile phone or helmet glasses;
- B. tracking the target mark as follows: tracking the position of a certain object in the image data by the front-end device; the certain object is an object in the image; for example, a bird or a tree on a mountain in a landscape painting can act as a certain object; subsequent description contents in the present embodiment are based on the certain object; obtaining the surface shape and feature of the certain object, and locking a pixel color disc of the certain object through combined establishment of algorithm binding and engine matching;
- C. analyzing the pixel color disc on the certain object as follows: collecting the locked pixel color disc of the certain object, and analyzing and obtaining a data value of the pixel color disc; in the present embodiment, more specifically, the data value of the pixel color disc comprises: hue-color essential attributes (H), saturation-color purity of 0-100% value (S) and lightness (V) and brightness (L) of 0-100% value.
- D. analyzing a grey zone on the certain object as follows: analyzing, by the front-end device, a BLOB (binary large object) of the certain object, judging a refraction angle, length and brightness of the grey zone of the certain object according to the BLOB, and determining a light source direction in a place where the front-end device is located, a distance between the front-end device and a light source, and the intensity of the light source according to the refraction angle, the length and the brightness of the grey zone; in the present embodiment, more specifically, step D is as follows: converting the refraction angle, the length and the brightness into identifiable codes of the front-end device; determining the light source direction in the place where the front-end device is located according to identification codes of the refraction angle; determining the distance between the front-end device and the light source according to identification codes of the length of the grey zone (shadow); and determining the intensity of the light source according to identification codes of the brightness of the grey zone;
- E. pushing the data to a light field of a virtual object as follows: pushing the data into the light field of the virtual object (an image formed by projecting the certain object) after converting the data into identifiable data of the front-end device according to the data value of the pixel color disc of the certain object, the light source direction, the distance between the front-end device and the light source and the intensity of the light source; and
- F. compensating and adjusting the imaging of the virtual object: adjusting the virtual object and a shadow of the virtual object through the data value of the pixel color disc of the certain object, the light source direction, the distance between the front-end device and the light source and the intensity of the light source, so that a virtual scenario is more realistic and clear. In the present embodiment, more specifically, the step F is as follows: adjusting brightness, hue and saturation of the virtual object through the data value of the pixel color disc of the certain object, the light source direction, the distance between the front-end device and the light source and the intensity of the light source; adjusting the refraction angle, the length and the brightness of the shadow of the virtual object according to the light source direction, the distance between the front-end device and the light source and the intensity of the light source; and the virtual scenario is more realistic and clear.
- The method provided by the above embodiment solves the problems of too high configuration of front-end data collection hardware, high cost of data transmission device and poor user experience of a helmet device of a display end; solve the problems that the virtual object cannot be integrated into the light field of the reality environment, the shadow cannot be accurately tracked, a light ray angle is inaccurate and the object display is floating and is in the air; solve the problem of serious color difference caused by an influence of the light field on the color; and solve the problem of high manufacture cost because the virtual object is only limited to a digitized image and a projected real object. By detecting the light source direction of the reality environment, the object projected from a virtual augmented environment is consistent with a reality shadow direction, the reflective refraction angle and the brightness are adjusted, and the reality of the projected object in the reality environment is increased. Under the condition of completing light field reduction of the virtual object without a strong front-end data collection device, complicated early manufacture and subsequent terminal display hardware with high cost, the virtual object can be integrated into the light field of the reality environment in real time, so that naked eye experience of a user is closer to the reality.
- Embodiments of the present invention also provide a front-end device. As shown in
FIG. 2 , the front-end device comprises: - a target mark identifying unit used for collecting image data and identifying the collected image data;
- a target mark tracking unit used for tracking the position of a certain object in the image data, obtaining the surface shape and feature of the certain object, and locking a pixel color disc of the certain object through combined establishment of algorithm binding and engine matching;
- a pixel color disc analyzing unit used for collecting the locked pixel color disc of the certain object, and analyzing and obtaining a data value of the pixel color disc;
- a grey zone analyzing unit used for analyzing a BLOB (binary large object) of the certain object, judging a refraction angle, length and brightness of the grey zone of the certain object according to the BLOB, and determining a light source direction in a place where the front-end device is located, a distance between the front-end device and a light source, and the intensity of the light source according to the refraction angle, the length and the brightness of the grey zone;
- a data pushing unit used for pushing the data into the light field of the virtual object (an image formed by projecting the certain object) after converting the data into identifiable data of the front-end device according to the data value of the pixel color disc of the certain object, the light source direction, the distance between the front-end device and the light source and the intensity of the light source; and
- a compensation and adjustment unit used for adjusting the virtual object and a shadow of the virtual object through the data value of the pixel color disc of the certain object, the light source direction, the distance between the front-end device and the light source and the intensity of the light source, so that a virtual scenario is more realistic and clear.
- In the above embodiment, more specifically, the grey zone analyzing unit is used for converting the refraction angle, the length and the brightness into identifiable codes of the front-end device; determining the light source direction in the place where the front-end device is located according to identification codes of the refraction angle; determining the distance between the front-end device and the light source according to identification codes of the length of the grey zone (shadow); and determining the intensity of the light source according to identification codes of the brightness of the grey zone.
- In the above embodiment, more specifically, the data value of the pixel color disc comprises: hue-color essential attributes (H), saturation-color purity of 0-100% value (S) and lightness (V) and brightness (L) of 0-100% value.
- In the above embodiment, more specifically, the compensation unit is used for adjusting brightness, hue and saturation of the virtual object through the data value of the pixel color disc of the certain object, the light source direction, the distance between the front-end device and the light source and the intensity of the light source, and adjusting the refraction angle, the length and the brightness of the shadow of the virtual object according to the light source direction, the distance between the front-end device and the light source and the intensity of the light source; and the virtual scenario is more realistic and clear.
- In the above embodiment, more specifically, the front-end device comprises a mobile phone or helmet glasses.
- The device provided by the above embodiment solves the problems of too high configuration of front-end data collection hardware, high cost of data transmission device and poor user experience of a helmet device of a display end; solve the problems that the virtual object cannot be integrated into the light field of the reality environment, the shadow cannot be accurately tracked, a light ray angle is inaccurate and the object display is floating and is in the air; solve the problem of serious color difference caused by an influence of the light field on the color; and solve the problem of high manufacture cost because the virtual object is only limited to a digitized image and a projected real object. By detecting the light source direction of the reality environment, the object projected from a virtual augmented environment is consistent with a reality shadow direction, the reflective refraction angle and the brightness are adjusted, and the reality of the projected object in the reality environment is increased. Under the condition of completing light field reduction of the virtual object without a strong front-end data collection device, complicated early manufacture and subsequent terminal display hardware with high cost, the virtual object can be integrated into the light field of the reality environment in real time, so that naked eye experience of a user is closer to the reality.
- The above only describes preferred embodiments of the present invention and is not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and the principle of the present invention shall be contained within the protection scope of the present invention.
Claims (10)
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201611086214.0A CN106530406A (en) | 2016-11-29 | 2016-11-29 | Light field source orientation method for augmented and virtual reality and front-end equipment |
CN201611086214.0 | 2016-11-29 | ||
CN201611086214 | 2016-11-29 |
Publications (2)
Publication Number | Publication Date |
---|---|
US20180150998A1 true US20180150998A1 (en) | 2018-05-31 |
US10002464B1 US10002464B1 (en) | 2018-06-19 |
Family
ID=58354252
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/381,137 Expired - Fee Related US10002464B1 (en) | 2016-11-29 | 2016-12-16 | Light field light source orientation method for augmented reality and virtual reality and front-end device |
Country Status (2)
Country | Link |
---|---|
US (1) | US10002464B1 (en) |
CN (1) | CN106530406A (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112040596A (en) * | 2020-08-18 | 2020-12-04 | 张雪媛 | Virtual space light control method, computer readable storage medium and system |
CN112313605A (en) * | 2018-10-03 | 2021-02-02 | 谷歌有限责任公司 | Object placement and manipulation in augmented reality environments |
US10930049B2 (en) | 2018-08-27 | 2021-02-23 | Apple Inc. | Rendering virtual objects with realistic surface properties that match the environment |
US11156896B2 (en) | 2019-10-31 | 2021-10-26 | Samsung Electronics Co., Ltd. | Augmented reality device |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108933954A (en) * | 2017-05-22 | 2018-12-04 | 中兴通讯股份有限公司 | Method of video image processing, set-top box and computer readable storage medium |
CN107492144B (en) * | 2017-07-12 | 2020-07-24 | 联想(北京)有限公司 | Light and shadow processing method and electronic equipment |
CN107845132B (en) * | 2017-11-03 | 2021-03-02 | 太平洋未来科技(深圳)有限公司 | Rendering method and device for color effect of virtual object |
Family Cites Families (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7869631B2 (en) * | 2006-12-11 | 2011-01-11 | Arcsoft, Inc. | Automatic skin color model face detection and mean-shift face tracking |
US8477246B2 (en) * | 2008-07-11 | 2013-07-02 | The Board Of Trustees Of The Leland Stanford Junior University | Systems, methods and devices for augmenting video content |
US8964298B2 (en) * | 2010-02-28 | 2015-02-24 | Microsoft Corporation | Video display modification based on sensor input for a see-through near-to-eye display |
US20130278631A1 (en) * | 2010-02-28 | 2013-10-24 | Osterhout Group, Inc. | 3d positioning of augmented reality information |
CN102426695A (en) * | 2011-09-30 | 2012-04-25 | 北京航空航天大学 | Virtual-real illumination fusion method of single image scene |
US9269022B2 (en) * | 2013-04-11 | 2016-02-23 | Digimarc Corporation | Methods for object recognition and related arrangements |
US9754416B2 (en) * | 2014-12-23 | 2017-09-05 | Intel Corporation | Systems and methods for contextually augmented video creation and sharing |
US9646419B2 (en) * | 2015-01-14 | 2017-05-09 | International Business Machines Corporation | Augmented reality device display of image recognition analysis matches |
-
2016
- 2016-11-29 CN CN201611086214.0A patent/CN106530406A/en active Pending
- 2016-12-16 US US15/381,137 patent/US10002464B1/en not_active Expired - Fee Related
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10930049B2 (en) | 2018-08-27 | 2021-02-23 | Apple Inc. | Rendering virtual objects with realistic surface properties that match the environment |
CN112313605A (en) * | 2018-10-03 | 2021-02-02 | 谷歌有限责任公司 | Object placement and manipulation in augmented reality environments |
US11156896B2 (en) | 2019-10-31 | 2021-10-26 | Samsung Electronics Co., Ltd. | Augmented reality device |
US11586091B2 (en) | 2019-10-31 | 2023-02-21 | Samsung Electronics Co., Ltd. | Augmented reality device |
CN112040596A (en) * | 2020-08-18 | 2020-12-04 | 张雪媛 | Virtual space light control method, computer readable storage medium and system |
Also Published As
Publication number | Publication date |
---|---|
CN106530406A (en) | 2017-03-22 |
US10002464B1 (en) | 2018-06-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10002464B1 (en) | Light field light source orientation method for augmented reality and virtual reality and front-end device | |
US11604076B2 (en) | Vision augmented navigation | |
US9846927B2 (en) | Systems and methods for haziness detection | |
US9488483B2 (en) | Localization using road markings | |
US20180081434A1 (en) | Eye and Head Tracking | |
US11600075B2 (en) | Nighttime sensing | |
CN103886107B (en) | Robot localization and map structuring system based on ceiling image information | |
CN104766270B (en) | One kind is based on fish-eye actual situation illumination fusion method | |
US10976163B2 (en) | Robust vision-inertial pedestrian tracking with heading auto-alignment | |
US20150161762A1 (en) | Information processing apparatus, information processing method, and program | |
CN116681870A (en) | Overlay contrast control in augmented reality displays | |
US10628964B2 (en) | Methods and devices for extended reality device training data creation | |
CN106908064B (en) | Indoor night vision navigation method based on Kinect2 sensor | |
US20150332471A1 (en) | User hand detecting device for detecting user's hand region and method thereof | |
US11024040B2 (en) | Dynamic object tracking | |
CN114037625A (en) | Image recovery method based on physical scattering model | |
US10540809B2 (en) | Methods and apparatus for tracking a light source in an environment surrounding a device | |
CN107330436B (en) | Scale criterion-based panoramic image SIFT optimization method | |
KR101935853B1 (en) | Night Vision System using LiDAR(light detection and ranging) and RADAR(Radio Detecting And Ranging) | |
Li et al. | Dynamic Adjustment and Distinguishing Method for Vehicle Headlight Based on Data Access of a Thermal Camera | |
US20240153245A1 (en) | Hybrid system for feature detection and descriptor generation | |
Wu et al. | Construction of Smart Glasses Assisted Road Sign Recognition System | |
Wagner et al. | Visual odometry for the lunar analogue rover “artemis” | |
WO2024097469A1 (en) | Hybrid system for feature detection and descriptor generation | |
Mao | Study on image dehazing with the self-adjustment of the haze degree |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: PACIFIC FUTURE LIMITED, HONG KONG Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:THEW, IAN ROY;LEE, KIEN YI;REEL/FRAME:040654/0045 Effective date: 20161215 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
FEPP | Fee payment procedure |
Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY |
|
LAPS | Lapse for failure to pay maintenance fees |
Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY |
|
STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
FP | Lapsed due to failure to pay maintenance fee |
Effective date: 20220619 |