WO2020056689A1 - 一种ar成像方法、装置及电子设备 - Google Patents

一种ar成像方法、装置及电子设备 Download PDF

Info

Publication number
WO2020056689A1
WO2020056689A1 PCT/CN2018/106784 CN2018106784W WO2020056689A1 WO 2020056689 A1 WO2020056689 A1 WO 2020056689A1 CN 2018106784 W CN2018106784 W CN 2018106784W WO 2020056689 A1 WO2020056689 A1 WO 2020056689A1
Authority
WO
WIPO (PCT)
Prior art keywords
sub
light
light intensity
angle
image
Prior art date
Application number
PCT/CN2018/106784
Other languages
English (en)
French (fr)
Inventor
菲永奥利维尔
李建亿
Original Assignee
太平洋未来科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 太平洋未来科技（深圳）有限公司 filed Critical 太平洋未来科技（深圳）有限公司
Priority to PCT/CN2018/106784 priority Critical patent/WO2020056689A1/zh
Priority to CN201811110297.1A priority patent/CN109214351B/zh
Publication of WO2020056689A1 publication Critical patent/WO2020056689A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G06V40/165—Detection; Localisation; Normalisation using facial parts and geometric relationships
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds

Definitions

  • the present invention relates to the field of augmented reality technology, and in particular, to an AR imaging method, device, and electronic device.
  • Existing light detection algorithms generally rely on specific sensors and hardware, or make assumptions about the position and time of a light source (such as the sun or the moon) based on environmental information in the scene.
  • the estimation of light information mainly focuses on light intensity and hue. It only calculates the average brightness of the entire real scene image, not the actual ambient light direction, which has nothing to do with the actual position of the light source in the real scene. It does not provide sufficient information for the shadow of the virtual object.
  • the AR imaging method, device, and electronic device provided by the embodiments of the present invention are used to solve at least the foregoing problems in related technologies.
  • An embodiment of the present invention provides an AR imaging method, including:
  • Detecting a face image captured by a mobile terminal extracting a sub-image of a nasal area in the face image; determining a light intensity weighting center of the sub-image based on light, and comparing the light intensity weighting center with the face image Compare the weighted centers to obtain the estimated light angle; obtain the light angle of the real scene according to the estimated light angle and the rotation angle of the mobile terminal, where the real scene is the real world currently shot by the mobile terminal; Generating a shadow image of the virtual object according to a light angle of the real scene and a preset position of the virtual object in the real scene.
  • determining the light intensity weighting center of the sub-image based on the light, and comparing the light intensity weighting center with the weight center of the face image to obtain a light estimation angle includes: dividing the sub image into For several sub-regions, determine the sub-light intensity weighting center of each of the sub-regions; compare each of the sub-light intensity weighting centers with the weight center of the face image to obtain the sub-ray estimation angle of each of the sub-regions Calculating the sub-light intensity of each of the sub-regions, and determining the weight of the sub-light estimation angle of the sub-region according to the sub-light intensity of the sub-region; and weighting the sub-light estimation angle and the sub-light estimation angle according to the sub-light To calculate the estimated angle of the light.
  • the method further comprises: calculating a distance value between the light intensity weighted center and the weighted center of the face image, and determining a confidence factor based on the distance value, the confidence factor reflecting the nose area Contrast between adjacent areas of the middle nose tip.
  • the method further comprises: calculating a sub-distance value between each of the sub-light intensity weighting centers and the weight center of the face image, and determining a sub-confidence factor based on the sub-distance values;
  • the confidence factor and the weight of the sub-ray estimation angle determine a confidence factor, and the confidence factor reflects the contrast between adjacent regions of the nose tip in the nose region.
  • the method further comprises: extracting an eye socket area in the face image, and comparing the light intensity of the eye socket; when the difference in light intensity of the eye socket is greater than or equal to a preset threshold, according to the eye socket's The light intensity determines the head tilt angle of the human face; and the light estimation angle is adjusted according to the human face tilt angle.
  • an AR imaging device including: an extraction module for detecting a face image captured by a mobile terminal, extracting a sub-image of a nasal region in the face image; a comparison module, It is used for determining the light intensity weighting center of the sub-image based on the light, comparing the light intensity weighting center with the weight center of the face image to obtain a light estimation angle; and an acquiring module for estimating the angle based on the light estimation.
  • a generating module is configured to calculate the light angle of the real scene and the virtual object at A preset position in the real scene generates a shadow image of the virtual object.
  • the comparison module includes: a decomposition unit for dividing the sub-image into a plurality of sub-regions, and determining a sub-light intensity weighting center of each of the sub-regions; a comparison unit for dividing each of the sub-light intensity The weighted center is compared with the weighted center of the face image to obtain an estimated sub-ray angle of each of the sub-regions; a determining unit is configured to calculate a sub-light intensity of each of the sub-regions, and according to the sub-lights of the sub-regions The intensity determines a weight of the sub-ray estimation angle of the sub-region; a calculation unit is configured to calculate the ray estimation angle according to each of the sub-ray estimation angle and the weight of the sub-ray estimation angle.
  • the comparison module is further configured to calculate a distance value between the weighted center of the light intensity and the weighted center of the face image, and determine a confidence factor based on the distance value, the confidence factor reflecting the nose Contrast between the adjacent areas of the nasal tip in the middle area.
  • the comparison module is further configured to calculate a sub-distance value between each of the sub-light intensity weighting centers and the weight center of the face image, and determine a sub-confidence factor based on the sub-distance values;
  • the sub-confidence factor and the weight of the sub-ray estimation angle determine a confidence factor, and the confidence factor reflects the contrast between adjacent regions of the nose tip in the nose region.
  • the comparison module further includes an adjustment unit, configured to extract an eye socket region in the face image and compare the light intensity of the eye socket; when the difference in light intensity of the eye socket is greater than or equal to
  • the threshold is preset, the head tilt angle of the human face is determined according to the light intensity of the eye socket; and the light estimation angle is adjusted according to the human face tilt angle.
  • Another aspect of the embodiments of the present invention provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein,
  • the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the AR imaging method described above.
  • the electronic device further includes an image acquisition module including a lens, an auto-focusing voice coil motor, a mechanical image stabilizer, and an image sensor, and the lens is fixed on the auto-focusing voice coil motor.
  • the lens is used to acquire an image
  • the image sensor transmits the image acquired by the lens to the recognition module
  • the autofocus voice coil motor is mounted on the mechanical image stabilizer
  • the processing module is based on the inside of the lens
  • the feedback of the lens shake detected by the gyroscope drives the action of the mechanical image stabilizer to achieve lens shake compensation.
  • the mechanical anti-shake device includes a movable plate, a movable frame, an elastic restoring mechanism, a base plate, and a compensation mechanism; a central portion of the movable plate is provided with a through hole through which the lens passes, and the auto-focusing voice coil motor Installed on the movable plate, the movable plate is installed in the movable frame, and the opposite sides of the movable plate are slidingly fitted with the inner walls of the opposite sides of the movable frame so that the movable plate can be moved along Reciprocating sliding in the first direction; the size of the movable frame is smaller than that of the substrate, and two opposite sides of the movable frame are connected to the substrate through two elastic restoring mechanisms, respectively.
  • the compensation mechanism includes a drive shaft, gears, A gear track and a limit track, the drive shaft is mounted on the base plate, the drive shaft is connected with the gear drive;
  • the gear track is provided on the movable plate, and the gear is mounted In the gear track, when the gear rotates, the gear track enables the movable plate to generate a displacement in a first direction and a displacement in a second direction, wherein the first direction is perpendicular to the second direction;
  • the limit track is disposed on the movable plate or the base plate, and the limit track is used to prevent the gear from detaching from the gear track.
  • a side of the movable plate is provided with a waist-shaped hole, and the waist-shaped hole is provided with a plurality of teeth that mesh with the gear along a circumferential direction thereof, and the waist-shaped hole and the plurality of teeth together constitute the A gear track, wherein the gear is located in the waist-shaped hole and meshes with the teeth; the limit track is disposed on the base plate, and a bottom of the movable plate is provided with a limit position within the limit track Piece, the limit track makes the movement track of the limit piece in a waist shape.
  • the limiting member is a protrusion provided on the bottom surface of the movable plate.
  • the gear track includes a plurality of cylindrical protrusions provided on the movable plate, the plurality of cylindrical protrusions are evenly spaced along the second direction, and the gear is in phase with the plurality of protrusions.
  • the limit track is a first arc-shaped stopper and a second arc-shaped stopper provided on the movable plate, and the first arc-shaped stopper and the second arc-shaped stopper are respectively It is arranged on two opposite sides of the gear track in the first direction, and the first arc-shaped stopper, the second arc-shaped stopper, and a plurality of the protrusions cooperate to make the moving track of the movable plate Waist-shaped.
  • the elastic recovery mechanism includes a telescopic spring.
  • the image acquisition module includes a mobile phone and a bracket for mounting the mobile phone.
  • the bracket includes a mobile phone mounting base and a retractable support rod;
  • the mobile phone mounting base includes a retractable connection plate and a folding plate group installed at opposite ends of the connection plate, and one end of the support rod is connected to the connection The middle portions of the plates are connected by a damping hinge;
  • the folded plate group includes a first plate body, a second plate body, and a third plate body, wherein one of two opposite ends of the first plate body is in phase with the connecting plate.
  • the other end of the opposite ends of the first plate body is hinged to one of the opposite ends of the second plate body; the other end of the opposite ends of the second plate body is connected to the third plate One end of the two opposite ends of the body is hinged; the second plate body is provided with an opening for the corner of the mobile phone to be inserted; when the mobile phone mount is used to install the mobile phone, the first plate body, the second plate body, and the first plate body
  • the three plates are folded in a right triangle state, the second plate is a hypotenuse of a right triangle, and the first plate and the third plate are right angles of a right triangle, wherein the third plate is One side of the side is attached to one side of the connecting plate , One of the opposite ends of the third plate member and the other end opposite ends of the first plate against body.
  • a first connection portion is provided on one side surface of the third plate body, and a first connection portion that is matched with the first connection portion is provided on a side surface where the connection plate is in contact with the third plate body.
  • a second connection portion is provided at one end of the opposite ends of the first plate body, and a second connection is provided at the other end of the opposite ends of the third plate body to cooperate with the second connection portion.
  • the other end of the support rod is detachably connected to a base.
  • the AR imaging method, device and electronic equipment provided by the embodiments of the present invention can obtain real-world light information only by using an existing dual-camera mobile terminal, and provide a more realistic effect for AR. And the cost is reduced; the anti-shake hardware structure of the mobile phone camera and the mobile phone selfie stand further enhance the shooting effect, which is more conducive to subsequent image or video processing.
  • FIG. 1 is a flowchart of an AR imaging method according to an embodiment of the present invention
  • FIG. 2 is a flowchart of an AR imaging method according to an embodiment of the present invention.
  • FIG. 3 is a flowchart of an AR imaging method according to an embodiment of the present invention.
  • FIG. 4 is a structural diagram of an AR imaging device according to an embodiment of the present invention.
  • FIG. 5 is a structural diagram of an AR imaging device according to an embodiment of the present invention.
  • FIG. 6 is a schematic diagram of a hardware structure of an electronic device that executes an AR imaging method provided by a method embodiment of the present invention
  • FIG. 7 is a schematic structural diagram of an image acquisition module according to an embodiment of the present invention.
  • FIG. 8 is a schematic structural diagram of a first mechanical vibration stabilizer provided by an embodiment of the present invention.
  • FIG. 9 is a schematic structural diagram of a bottom surface of a first movable board according to an embodiment of the present invention.
  • FIG. 10 is a schematic structural diagram of a second mechanical image stabilizer provided by an embodiment of the present invention.
  • FIG. 11 is a schematic structural diagram of a bottom surface of a second movable board according to an embodiment of the present invention.
  • FIG. 12 is a structural diagram of a stent provided by an embodiment of the present invention.
  • FIG. 13 is a schematic diagram of a state of a stent provided by an embodiment of the present invention.
  • FIG. 14 is a schematic view of another state of a stent provided by an embodiment of the present invention.
  • FIG. 15 is a structural state diagram when the mounting base and the mobile phone are connected according to an embodiment of the present invention.
  • FIG. 1 is a flowchart of an AR imaging method according to an embodiment of the present invention. As shown in FIG. 1, an AR imaging method provided by an embodiment of the present invention includes:
  • S101 Detect a face image captured by a mobile terminal, and extract a sub-image of a nasal region in the face image.
  • the front camera of the mobile terminal can be used to determine the real The lighting conditions of the world's current scene. Since the user is roughly facing the screen of the mobile terminal, it is only necessary to mirror the angle of the ambient light detected in the face image to the virtual object that needs to be added to the current scene in the real world.
  • a snapshot image of the current scene is taken by a front camera, and when it is detected that the snapshot image includes a user's face image, a face well known to those skilled in the art is used.
  • Detection algorithms for example, cost functions based on color, shape, and symmetry published by Eli Saber et al.
  • any CNN-based face detection algorithm, etc. from this image
  • the facial features are extracted in the facial features. Because the nose is protruding from the face in the facial features, the brightness of the nose area is more likely to be affected by ambient light. Therefore, this embodiment extracts the nose from the facial features to obtain the nose in the facial image. The sub-image of the region.
  • the corresponding light intensity weighting center is determined according to the image moment of the sub-image.
  • An image moment is a set of moments calculated from digital graphics. It usually describes the global features of the image and provides a lot of information about the different types of geometric features of the image, such as size, position, orientation, and shape. For example, a The order moment is related to the shape, the second order moment shows the degree of expansion of the curve around the straight line average value, and the third order moment is a measure of the symmetry of the average value. From the second and third order moments, a group of 7 constants can be derived. Order moments and invariant order moments are statistical characteristics of images, which can be used to classify images based on this, which are common knowledge in the art, and the present invention will not repeat them here.
  • the weighting center is the collection center of the image
  • the coordinate position of the weighting center of the image is the direction of the light intensity of the sub-picture.
  • the coordinate positions are compared.
  • the direction from the geometric center to the light weighted center is the direction of the ambient light.
  • we can establish the coordinate system by selecting the coordinate origin and obtain the angle between this vector and the X axis for the current scene.
  • the light estimation angle can also be calculated by other non-proprietary algorithms, which is not limited in the present invention. It should be noted that, in the embodiment of the present invention, the ambient light will be considered to be unidirectional and uniform.
  • the light estimation angle may be obtained through the following steps.
  • S1021 Divide the sub-image into a plurality of sub-regions, and determine a sub-light intensity weighting center of each of the sub-regions.
  • S1022 Compare the weighted center of each sub-light intensity with the weighted center of the face image to obtain an estimated sub-ray angle of each of the sub-regions.
  • S1023 Calculate the sub-light intensity of each of the sub-regions, and determine a weight of the sub-light estimation angle of the sub-region according to the sub-light intensity of the sub-region.
  • the sub-images can be divided into four equal parts to obtain four sub-regions, and the sub-light intensity weighting center of each sub-region and the sub-ray estimation angle of each sub-region are determined according to the above method.
  • the light intensity corresponding to the sub-picture is obtained according to the light and dark contrast information therein.
  • the sub-light intensity of each sub-region is used as the weight of the sub-light estimation angle of the sub-region; finally ,
  • the sub-ray estimation angles of the four sub-areas are calculated by adding and averaging according to their corresponding weights to obtain the average ray estimation angle.
  • the following manner may also be used to determine whether the calculated light estimation angle is accurate.
  • a distance value between the weighted center of light intensity and the weighted center of the face image is calculated, and a confidence factor is determined based on the distance value, and the confidence factor reflects the tip of the nose in the nose region Contrast between adjacent areas.
  • this distance value By measuring the distance between the weighted center of the face image and the weighted center of the light intensity, we can obtain a distance value that increases as the contrast between the shaded and illuminated parts of the nose area increases. In a diffused light environment with no specific light direction, the distance value will be small or zero. Therefore, this distance value provides a confidence factor related to the contrast in the nasal region between the adjacent regions of the nose tip. Basically, there are specific light sources when shooting faces. Therefore, the larger the distance value is, the greater the confidence is, the higher the accuracy of the light estimation angle is, and the more it can be determined that this angle represents the ambient light. direction.
  • a sub-distance value between each of the sub-light intensity weighting centers and the weight center of the face image is calculated, and a sub-confidence factor is determined based on the sub-distance value; according to each of the sub-confidence factors And the weight of the sub-ray estimation angle to determine a confidence factor that reflects the contrast between adjacent regions of the nose tip in the nose region.
  • step S102 calculates the light estimation angle by decomposing the nasal region into several sub-images
  • the confidence factor may be determined through the second embodiment, thereby improving the accuracy of the confidence factor.
  • the method includes:
  • the light intensities of the two eye sockets of the face should be the same or have a small difference.
  • the difference in the light intensity of the two eye sockets is greater than or equal to a preset threshold, it is explained that the shooting is performed in step S101.
  • the face image the user's head is deflected.
  • a tilt angle model can be obtained by performing machine model learning training in advance, and inputting the difference in light intensity into the tilt angle model to obtain the head tilt angle of the face.
  • the head tilt angle It refers to the angle when the 90-degree position is maintained relative to the head.
  • the quotient of the inclination angle and the right angle can be used as a correction coefficient, and the light estimation angle is adjusted based on the correction coefficient, so as to solve the effect of head deflection on the result.
  • S103 Obtain a light angle of a real scene according to the estimated angle of the light and the rotation angle of the mobile terminal, where the real scene is the real world currently shot by the mobile terminal.
  • the face image of the user is captured by the front-end camera, and the AR video is viewed by the back-end camera, it is necessary to perform a mirroring operation on the above-mentioned light estimation angle to implement image-oriented switching.
  • the user since the user may rotate the mobile terminal while watching the AR video, it is necessary to construct the light angle of the ambient light in the real scene according to the light estimation angle and the rotation angle of the mobile terminal.
  • the light estimation angle may be rotated according to the rotation angle of the mobile terminal, so as to obtain the light angle of the real scene.
  • S104 Generate a shadow image of the virtual object according to a light angle of the real scene and a preset position of the virtual object in the real scene.
  • the shadow position of the virtual object in the target picture may be determined according to the light angle of the real scene and the preset position of the virtual object.
  • determine the shadow shape at the shadow position Generates a shadow image of a virtual object based on the shadow position and shape.
  • the embodiment of the present invention only the existing dual-camera mobile terminal can be used to obtain light information in the real world, which provides a more realistic effect for AR and reduces the cost.
  • the embodiment of the present invention It can be applied to indoor and outdoor conditions during the day or night, and it does not need to know the real world time and use GPS positioning to obtain the local position, which has low requirements on the performance of mobile terminal equipment.
  • FIG. 4 is a structural diagram of an AR imaging device according to an embodiment of the present invention. As shown in FIG. 3, the device specifically includes: an extraction module 100, a comparison module 200, an acquisition module 300, and a generation module 400. among them,
  • An extraction module 100 is configured to detect a face image captured by a mobile terminal and extract a sub-image of a nasal area in the face image; a comparison module 200 is configured to determine a light intensity weighting center of the sub-image based on light, and Comparing the light intensity weighting center with the weighting center of the face image to obtain a light estimation angle; an acquisition module 300 is configured to obtain a light angle of a real scene according to the light estimation angle and the rotation angle of the mobile terminal Wherein the real scene is the real world currently captured by the mobile terminal; a generating module 400 is configured to generate the virtual scene according to a light angle of the real scene and a preset position of a virtual object in the real scene The shadow image of the object.
  • the AR imaging device provided by the embodiment of the present invention is specifically configured to execute the method provided by the embodiment shown in FIG. 1, and its implementation principles, methods, and functional uses are similar to the embodiment shown in FIG. 1, and details are not described herein again.
  • FIG. 5 is a structural diagram of an AR imaging device according to an embodiment of the present invention. As shown in FIG. 5, the device specifically includes: an extraction module 100, a comparison module 200, an acquisition module 300, and a generation module 400. among them,
  • An extraction module 100 is configured to detect a face image captured by a mobile terminal and extract a sub-image of a nasal area in the face image; a comparison module 200 is configured to determine a light intensity weighting center of the sub-image based on light, and Comparing the light intensity weighting center with the weighting center of the face image to obtain a light estimation angle; an acquisition module 300 is configured to obtain a light angle of a real scene according to the light estimation angle and the rotation angle of the mobile terminal Wherein the real scene is the real world currently captured by the mobile terminal; a generating module 400 is configured to generate the virtual scene according to a light angle of the real scene and a preset position of a virtual object in the real scene The shadow image of the object.
  • the comparison module 200 includes a decomposition unit 210, a comparison unit 220, a determination unit 230, and a calculation unit 240. among them,
  • a decomposition unit 210 is configured to divide the sub-image into a plurality of sub-regions and determine a sub-light intensity weighting center of each of the sub-regions; a comparison unit 220 is configured to compare each of the sub-light intensity weighting centers with the face The weighted centers of the images are compared to obtain a sub-light estimation angle determining unit 230 of each of the sub-regions, which is used to calculate the sub-light intensity of each of the sub-regions, and determine the sub-regions based on the sub-light intensity of the sub-regions.
  • the weight of the sub-ray estimation angle; a calculation unit 240 is configured to calculate the ray estimation angle according to each of the sub-ray estimation angle and the weight of the sub-ray estimation angle.
  • the comparison module 200 further includes an adjustment unit 250 for extracting an eye socket region in the face image and comparing the light intensity of the eye socket; when the difference in light intensity of the eye socket is greater than or equal to
  • the threshold is set, the head tilt angle of the human face is determined according to the light intensity of the eye socket; and the light estimation angle is adjusted according to the human face tilt angle.
  • the comparison module 200 is further configured to calculate a distance value between the light intensity weighted center and the weighted center of the face image, and determine a confidence factor based on the distance value, the confidence factor reflecting the nose Contrast between the adjacent areas of the nasal tip in the middle area.
  • the comparison module 200 is further configured to calculate a sub-distance value between each of the sub-light intensity weighting centers and the weight center of the face image, and determine a sub-confidence factor based on the sub-distance values;
  • the sub-confidence factor and the weight of the sub-ray estimation angle determine a confidence factor, and the confidence factor reflects the contrast between adjacent regions of the nose tip in the nose region.
  • the comparison module 200 further includes an adjustment unit 250 for extracting an eye socket region in the face image and comparing the light intensity of the eye socket; when the light intensity of the eye socket is greater than or equal to a preset At the threshold, the head tilt angle of the human face is determined according to the light intensity of the eye socket; and the light estimation angle is adjusted according to the human face tilt angle.
  • an adjustment unit 250 for extracting an eye socket region in the face image and comparing the light intensity of the eye socket; when the light intensity of the eye socket is greater than or equal to a preset At the threshold, the head tilt angle of the human face is determined according to the light intensity of the eye socket; and the light estimation angle is adjusted according to the human face tilt angle.
  • the AR imaging device provided by the embodiment of the present invention is specifically configured to execute the method provided by the embodiment shown in FIG. 1 to FIG. 3, and its implementation principles, methods, and functional uses are similar to the embodiment shown in FIG. 1-3, and here No longer.
  • the above-mentioned AR imaging device may be used as one of the software or hardware functional units and independently provided in the above-mentioned electronic device, or may be used as one of the functional modules integrated in the processor to perform AR imaging according to the embodiments of the present invention. method.
  • the electronic device provided by the embodiment of the present invention includes one or more processors 610 and a memory 620.
  • One processor 610 is taken as an example in FIG. 6.
  • the apparatus for performing the AR imaging method may further include an input device 630 and an output device 630. among them,
  • the processor 610, the memory 620, the input device 630, and the output device 640 may be connected through a bus or other methods. In FIG. 6, the connection through the bus is taken as an example.
  • the memory 620 is a non-volatile computer-readable storage medium and can be used to store non-volatile software programs, non-volatile computer executable programs, and modules, as corresponding to the AR imaging method in the embodiment of the present invention.
  • the processor 610 executes various functional applications and data processing of the server by running non-volatile software programs, instructions, and modules stored in the memory 620, that is, implementing the AR imaging method.
  • the memory 620 may include a storage program area and a storage data area, where the storage program area may store an operating system and an application program required for at least one function; the storage data area may store the use of an AR imaging device created according to an embodiment of the present invention. Data, etc.
  • the memory 620 may include a high-speed random access memory 620, and may further include a non-volatile memory 620, such as at least one magnetic disk memory 620, a flash memory device, or other non-volatile solid-state memory 620.
  • the memory 620 may optionally include a memory 620 remotely disposed with respect to the processor 66, and these remote memories 620 may be connected to the AR imaging device through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
  • the input device 630 may receive inputted numeric or character information, and generate key signal inputs related to user settings and function control of the AR imaging device.
  • the input device 630 may include a device such as a pressing module.
  • the one or more modules are stored in the memory 620, and when executed by the one or more processors 610, execute the AR imaging method.
  • the electronic devices in the embodiments of the present invention exist in various forms, including but not limited to:
  • Mobile communication equipment This type of equipment is characterized by mobile communication functions, and its main goal is to provide voice and data communication.
  • Such terminals include: smart phones (such as iPhone), multimedia phones, feature phones, and low-end phones.
  • Ultra-mobile personal computer equipment This type of equipment belongs to the category of personal computers, has computing and processing functions, and generally has the characteristics of mobile Internet access.
  • Such terminals include: PDA, MID and UMPC devices, such as iPad.
  • Portable entertainment equipment This type of equipment can display and play multimedia content.
  • Such devices include: audio and video players (such as iPod), handheld game consoles, e-books, as well as smart toys and portable car navigation devices.
  • the electronic device includes an image acquisition module.
  • the image acquisition module of this embodiment includes a lens 1000, an autofocus voice coil motor 2000, a mechanical image stabilizer 3000, and an image sensor 4000.
  • the lens 1000 is fixed on the auto-focusing voice coil motor 2000, and the lens 1000 is fixed on the auto-focusing voice coil motor 2000.
  • the lens 1000 is used to acquire an image, and the image sensor 4000 sets the lens 1000
  • the acquired image is transmitted to the recognition module, the auto-focusing voice coil motor 2000 is mounted on the mechanical image stabilizer 3000, and the processing module drives the camera based on the feedback of the lens 1000 shake detected by the gyroscope in the lens 1000.
  • the operation of the mechanical image stabilizer 3000 is described to realize the shake compensation of the lens 1000.
  • the lens 1000 needs to be driven in at least two directions, which means that multiple coils need to be arranged. It brings certain challenges to the miniaturization of the overall structure, and is easily affected by external magnetic fields, which affects the anti-shake effect. Therefore, the Chinese patent published as CN106131435A provides a miniature optical anti-shake camera module, which realizes memory through temperature changes.
  • the alloy wire is stretched and shortened to pull the auto-focusing voice coil motor 2000 to achieve the shake compensation of the lens 1000.
  • the control chip of the micro memory alloy optical anti-shake actuator can control the change of the driving signal to change the memory alloy wire.
  • the temperature is used to control the elongation and shortening of the memory alloy wire, and the position and moving distance of the actuator are calculated based on the resistance of the memory alloy wire.
  • the micro memory alloy optical image stabilization actuator moves to the specified position, the resistance of the memory alloy wire at this time is fed back. By comparing the deviation of this resistance value and the target value, the movement on the micro memory alloy optical image stabilization actuator can be corrected. deviation.
  • the structure of the above technical solution alone cannot achieve accurate compensation for the lens 1000 in the case of multiple shakes, which is due to the heating of the shape memory alloy It takes a certain time to cool down and cool down.
  • the above technical solution can achieve the compensation of the lens 1000 for the shake in the first direction, but when the subsequent shake in the second direction occurs, due to the memory alloy It is too late to deform in an instant, so it is easy to cause compensation in a timely manner, and it is impossible to accurately achieve 1000-shake compensation for a lens that has multiple shakes and continuous shakes in different directions, so it is necessary to improve its structure.
  • this embodiment improves the optical image stabilizer and design it as a mechanical image stabilizer 3000.
  • the specific structure is as follows:
  • the mechanical image stabilizer 3000 of this embodiment includes a movable plate 3100, a movable frame 3200, an elastic restoring mechanism 3300, a base plate 3400, and a compensation mechanism 3500.
  • the movable plate 3100 and the base plate 3400 are provided in the middle of the plate for the lens to pass through.
  • the auto-focusing voice coil motor is installed on the movable plate 3100, and the movable plate 3100 is installed in the movable frame 3200.
  • the movable plate 3100 of this embodiment The width in the left-to-right direction is substantially the same as the internal width of the movable frame 3200, so that the opposite sides (left and right sides) of the movable plate 3100 and the inner walls of the opposite sides (left and right sides) of the movable frame 3200 slide to fit.
  • the movable plate 3100 can slide back and forth along the first direction within the movable frame 3200.
  • the first direction in this embodiment is the vertical direction in the figure.
  • the size of the movable frame 3200 in this embodiment is smaller than the size of the substrate 3400, and two opposite sides of the movable frame 3200 are connected to the substrate 3400 through two elastic recovery mechanisms 3300, respectively.
  • the elastic restoring mechanism 3300 is a telescopic spring or other elastic member, and it should be noted that the elastic restoring mechanism 3300 of this embodiment only allows it to expand and retract in the left-right direction in the figure (that is, the second direction described below). The ability to move along the first direction cannot be designed.
  • the purpose of designing the elastic recovery mechanism 3300 is also to facilitate the movable frame 3200 to reset the movable plate 3100 after the movable frame 3200 compensates for displacement.
  • the specific action process of this embodiment will be described below. The process is described in detail.
  • the compensation mechanism 3500 of this embodiment drives the movable plate 3100 and the lens on the movable plate 3100 under the driving of the processing module (which may be an action instruction sent by the processing module) to implement lens shake compensation.
  • the compensation mechanism 3500 in this embodiment includes a driving shaft 3510, a gear 3520, a gear track 3530, and a limit track 3540.
  • the driving shaft 3510 is mounted on the substrate 3400, and specifically is mounted on the substrate 3400. On the surface, the drive shaft 3510 is drivingly connected to the gear 3520.
  • the drive shaft 3510 can be driven by a structure such as a micro motor (not shown), and the micro motor is controlled by the processing module described above; the gear track 3530 is provided On the movable plate 3100, the gear 3520 is installed in the gear track 3530 and moves in a preset direction of the gear track 3530.
  • the gear track 3530 makes the movement
  • the plate 3100 can generate a displacement in a first direction and a displacement in a second direction, wherein the first direction is perpendicular to the second direction; the limit track 3540 is provided on the movable plate 3100 or the base plate 3400 In the above, the limit track 3540 is used to prevent the gear 3520 from leaving the gear track 3530.
  • gear track 3530 and the limit track 3540 of this embodiment have the following two structural forms:
  • a waist-shaped hole 3550 is provided on the lower side of the movable plate 3100 in this embodiment, and a plurality of waist-shaped holes 3550 are provided along the circumferential direction (that is, the surrounding direction of the waist-shaped hole 3550).
  • a tooth 3560 meshing with the gear 3520, the waist-shaped hole 3550 and a plurality of the teeth 3560 together form the gear track 3530, and the gear 3520 is located in the waist-shaped hole 3550 and communicates with the tooth 3560.
  • the meshing makes the gear 3520 move the gear track 3530 when it rotates, and then directly drives the movement of the movable plate 3100.
  • this embodiment describes A limiting rail 3540 is provided on the base plate 3400, and a bottom of the movable plate 3100 is provided with a limiting member 3570 installed in the limiting rail 3540.
  • the limiting rail 3540 enables the limiting member 3570 to be located therein.
  • the movement trajectory is waist-shaped, that is, the movement trajectory of the limiter 3570 in the current track is the same as the movement trajectory of the movable plate 3100.
  • the limiter 3570 of this embodiment is provided on the movable plate 3100. Bulge on the bottom.
  • the gear track 3530 of this embodiment may also be composed of a plurality of cylindrical protrusions 3580 provided on the movable plate 3100, and a plurality of the cylindrical protrusions 3580 along the
  • the gears 3520 are arranged at regular intervals in the second direction, and the gears 3520 are engaged with a plurality of the protrusions; and the limit track 3540 is a first arc-shaped stopper 3590 and a second arc provided on the movable plate 3100.
  • Shape limiting member 3600, the first arc-shaped limiting member 3590 and the second arc-shaped limiting member 3600 are respectively disposed on opposite sides of the gear rail 3530 in the first direction to prevent movement on the movable plate 3100
  • the gear 3520 is located on one side of the gear rail 3530.
  • the gear 3520 is easily separated from the gear rail 3530 formed by the cylindrical protrusion 3580.
  • the first arc-shaped stopper 3590 or the second arc-shaped stopper 3600 can Plays a guiding role, so that the movable plate 3100 can move in a preset direction of the gear track 3530, that is, the first arc-shaped limiting member 3590, the second arc-shaped limiting member 3600, and a plurality of the protrusions cooperate
  • the motion trajectory of the movable board 3100 is waist-shaped.
  • the following describes the working process of the mechanical image stabilizer 3000 of this embodiment in detail with reference to the above structure.
  • the two shake directions are opposite, and the movable plate 3100 needs to be compensated once in the first direction. And then motion compensation once in the second direction.
  • the gyroscope feeds back the detected lens 1000 shake direction and distance to the processing module in advance, and the processing module calculates the required moving distance of the movable plate 3100, so that the driving shaft 3510 drives The gear 3520 rotates.
  • the gear 3520 cooperates with the gear track 3530 and the limit track 3540, and the processing module wirelessly sends a driving signal, thereby driving the movable plate 3100 to move to the compensation position in the first direction.
  • the processing module wirelessly sends a driving signal, thereby driving the movable plate 3100 to move to the compensation position in the first direction.
  • the movable plate is driven again by the driving shaft 3510. 3100 reset.
  • the elastic recovery mechanism 3300 also provides a reset force for resetting the movable plate 3100, which is more convenient for the movable plate 3100 to return to the initial position.
  • the processing method is the same as the compensation steps in the first direction described above.
  • the mechanical compensator provided in this embodiment not only does not receive interference from external magnetic fields and has a good anti-shake effect, but also can accurately compensate the lens 1000 in the event of multiple shakes, and the compensation is timely and accurate.
  • the mechanical anti-shake device using this embodiment is not only simple in structure, but also requires small installation space for each component, which facilitates the integration of the entire anti-shake device and has higher compensation accuracy.
  • the electronic device in this embodiment includes a mobile phone and a bracket for mounting the mobile phone.
  • the purpose of including a bracket for an electronic device is to support and fix the electronic device due to the uncertainty of the image acquisition environment.
  • the bracket 5000 in this embodiment includes a mobile phone mounting base 5100 and a retractable supporting rod 5200.
  • the middle portion of the supporting rod 5200 and the mobile phone mounting base 5100 passes through a damping hinge.
  • the applicant found that the mobile phone mount 5100 combined with the support pole 5200 occupies a large space. Even if the support pole 5200 is retractable, the mobile phone mount 5100 cannot undergo structural changes and the volume will not be further reduced. Putting it in a pocket or a small bag causes the inconvenience of carrying the stent 5000. Therefore, in this embodiment, a second step improvement is made to the stent 5000, so that the overall storability of the stent 5000 is further improved.
  • the mobile phone mounting base 5100 of this embodiment includes a retractable connecting plate 5110 and a folding plate group 5120 installed at opposite ends of the connecting plate 5110.
  • the support rod 5200 and the connecting plate 5110 The middle part is connected by a damping hinge;
  • the folding plate group 5120 includes a first plate body 5121, a second plate body 5122, and a third plate body 5123, wherein one of the two opposite ends of the first plate body 5121 is connected to the first plate body 5121.
  • the connecting plate 5110 is hinged, the other end of the opposite ends of the first plate body 5121 is hinged with one of the opposite ends of the second plate body 5122, and the opposite ends of the second plate body 5122 The other end is hinged to one of opposite ends of the third plate body 5123; the second plate body 5122 is provided with an opening 5130 for a corner of the mobile phone to be inserted.
  • the first plate 5121, the second plate 5122, and the third plate 5123 are folded into a right triangle state, and the second plate 5122 is a hypotenuse of a right-angled triangle, and the first plate body 5121 and the third plate 5123 are right-angled sides of a right triangle, wherein one side surface of the third plate body 5123 and one of the connection plate 5110 are The side is attached side by side, and the other end of the opposite ends of the third plate body 5123 and the one of the opposite ends of the first plate body 5121 are against each other.
  • This structure can make the three folding plates in a self-locking state, and When the two lower corners of the mobile phone are inserted into the two openings 5130 on both sides, the lower sides of the mobile phone 6000 are located in two right-angled triangles.
  • the mobile phone 6000 can be completed through the joint work of the mobile phone, the connecting plate 5110, and the folding plate group 5120.
  • the triangle state cannot be opened under external force.
  • the triangle state of 5120 pieces of folding plate group can only be released after the mobile phone is pulled out from the opening 5130.
  • the connecting plate 5110 When the mobile phone mounting base 5100 is not in working state, the connecting plate 5110 is reduced to a minimum length, and the folding plate group 5120 and the connecting plate 5110 are folded to each other.
  • the user can fold the mobile phone mounting base 5100 to a minimum volume. Due to the scalability of the lever 5200, the entire bracket 5000 can be accommodated in the smallest state, which improves the collection of the bracket 5000. The user can even put the bracket 5000 directly into the pocket or small handbag, which is very convenient.
  • a first connection portion is also provided on one side of the third plate body 5123, and a side surface where the connection plate 5110 is in contact with the third plate body 5123 is provided with the first connection portion.
  • a first mating portion that mates with the connecting portion.
  • the first connecting portion of this embodiment is a convex strip or protrusion (not shown in the figure), and the first matching portion is a card slot (not shown in the figure) opened on the connecting plate 5110.
  • This structure not only improves the stability when the 5120 pieces of the folding plate group are in a triangular state, but also facilitates the connection of the 5120 pieces of the folding plate group and the connecting plate 5110 when the mobile phone mounting base 5100 needs to be folded to a minimum state.
  • a second connection portion is also provided at one end of the opposite ends of the first plate body 5121, and the other end of the opposite ends of the third plate body 5123 is provided with the second connection portion.
  • the second connection portion may be a protrusion (not shown in the figure), and the second mating portion is an opening 5130 or a card slot (not shown in the figure) that cooperates with the protrusion.
  • a base (not shown in the figure) can be detachably connected to the other end of the support rod 5200.
  • the support rod 5200 can be stretched to A certain length, put the bracket 5000 on a plane through the base, and then place the mobile phone in the mobile phone mount 5100 to complete the fixing of the mobile phone; and the detachable connection of the support rod 5200 and the base can make the two can be carried separately, further The accommodating and carrying convenience of the bracket 5000 are improved.
  • the device embodiments described above are only schematic, and the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, which may be located in One place, or can be distributed to multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the objective of the solution of this embodiment. Those of ordinary skill in the art can understand and implement without creative labor.
  • An embodiment of the present invention provides a non-transitory computer-readable storage storage medium, where the computer storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by an electronic device, the electronic device is caused
  • the AR imaging method in any of the above method embodiments is performed on the above.
  • An embodiment of the present invention provides a computer program product, wherein the computer program product includes a computer program stored on a non-transitory computer-readable storage medium, the computer program includes program instructions, and when the program instructions When executed by an electronic device, the electronic device is caused to execute the AR imaging method in any of the method embodiments described above.
  • each embodiment can be implemented by means of software plus a necessary universal hardware platform, and of course, also by hardware.
  • the above-mentioned technical solution in essence or a part that contributes to the existing technology may be embodied in the form of a software product, and the computer software product may be stored in a computer-readable storage medium, the computer-readable record A medium includes any mechanism for storing or transmitting information in a form readable by a computer (eg, a computer).
  • machine-readable media include read-only memory (ROM), random-access memory (RAM), magnetic disk storage media, optical storage media, flash storage media, electrical, optical, acoustic, or other forms of propagation signals (e.g., carrier waves , Infrared signals, digital signals, etc.), the computer software product includes a number of instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute various embodiments or certain parts of the embodiments Methods.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Multimedia (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Geometry (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Studio Devices (AREA)

Abstract

一种AR成像方法、装置及电子设备，所述方法包括：检测移动终端拍摄到的人脸图像，提取所述人脸图像中鼻部区域的子图像（S101）；基于光线确定所述子图像的光强加权中心，将所述光强加权中心与所述人脸图像的加权中心进行比较，得到光线估计角度（S102）；根据所述光线估计角度和所述移动终端的旋转角度，得到真实场景的光线角度（S103），其中，所述真实场景为所述移动终端当前拍摄的真实世界；根据所述真实场景的光线角度和虚拟对象在所述真实场景中的预设位置，生成所述虚拟对象的阴影图像（S104）。所述方法不需要额外的硬件，只需使用现有的双摄像头的移动终端即可获取真实世界中的光线信息，为AR提供更逼真的效果，且降低了成本，对移动终端设备的性能要求低。

Description

一种AR成像方法、装置及电子设备 技术领域
本发明涉及增强现实技术领域，尤其涉及一种AR成像方法、装置及电子设备。
背景技术
计算机视觉特别是在增强现实应用领域中的一个常见问题是，对影响虚拟对象成像的光线强度和方向的检测。现有的光线检测算法一般依赖于特定的传感器和硬件，或者根据场景中的环境信息对光源(例如太阳或月亮)的位置和时间对光源进行假设。在增强现实应用领域，对光线信息的估计主要关注光强和色调，它只是计算整个真实场景图像的平均亮度，而不是实际的环境光方向，这与真实场景光源的实际位置没有关系。并没有为虚拟对象的阴影提供足够的信息保障。
此外，手机上开始越来越多的使用AR技术，真实场景的获取也越来越多的使用手机摄像装置。为了提高手机获取视频的图像质量，以及避免用户长时间手持手机观看视频导致体力不支，往往需要改进手机摄像头的防抖性能和稳定支撑机构。现有技术是通过软件防抖技术对抖动进行预先处理，但是硬件本身并未有明显改进，这不能从根本上解除抖动的影响，也为后续AR图像处理增加了困难。
发明内容
本发明实施例提供的AR成像方法、装置及电子设备，用以至少解决相关技术中的上述问题。
本发明实施例一方面提供了一种AR成像方法，包括：
检测移动终端拍摄到的人脸图像，提取所述人脸图像中鼻部区域的子图像；基于光线确定所述子图像的光强加权中心，将所述光强加权中心与所述人脸图像的加权中心进行比较，得到光线估计角度；根据所述光线估计角度和所述移动终端的旋转角度，得到真实场景的光线角度，其中，所述真实场景为所述移动终端当前拍摄的真实世界；根据所述真实场景的光线角度和虚拟对象在所述真实场景中的预设位置，生成所述虚拟对象的阴影图像。
进一步地，所述基于光线确定所述子图像的光强加权中心，将所述光强加权中心与所述人脸图像的加权中心进行比较，得到光线估计角度，包括：将所述子图像分为若干个子区域，确定各所述子区域的子光强加权中心；将各所述子光强加权中心与 所述人脸图像的加权中心进行比较，得到各所述子区域的子光线估计角度计算各所述子区域的子光照强度，根据所述子区域的子光照强度确定所述子区域的子光线估计角度的权重；根据各所述子光线估计角度及所述子光线估计角度的权重，计算得到所述光线估计角度。
进一步地，所述方法还包括：计算所述光强加权中心与所述人脸图像的加权中心之间的距离值，基于所述距离值确定置信因子，所述置信因子反映所述鼻部区域中鼻尖相邻区域之间的对比度。
进一步地，所述方法还包括：计算各所述子光强加权中心与所述人脸图像的加权中心之间的子距离值，基于所述子距离值确定子置信因子；根据各所述子置信因子及所述子光线估计角度的权重，确定置信因子，所述置信因子反映所述鼻部区域中鼻尖相邻区域之间的对比度。
进一步地，所述方法还包括：提取所述人脸图像中的眼窝区域，比较所述眼窝的光照强度；当所述眼窝的光照强度差值大于或等于预设阈值时，根据所述眼窝的光照强度确定所述人脸的头部倾斜角度；根据所述人脸倾斜角度所述光线估计角度进行调整。
本发明实施例的另一方面提供了一种AR成像装置，包括：提取模块，用于检测移动终端拍摄到的人脸图像，提取所述人脸图像中鼻部区域的子图像；比较模块，用于基于光线确定所述子图像的光强加权中心，将所述光强加权中心与所述人脸图像的加权中心进行比较，得到光线估计角度；获取模块，用于根据所述光线估计角度和所述移动终端的旋转角度，得到真实场景的光线角度，其中，所述真实场景为所述移动终端当前拍摄的真实世界；生成模块，用于根据所述真实场景的光线角度和虚拟对象在所述真实场景中的预设位置，生成所述虚拟对象的阴影图像。
进一步地，所述比较模块包括：分解单元，用于将所述子图像分为若干个子区域，确定各所述子区域的子光强加权中心；比较单元，用于将各所述子光强加权中心与所述人脸图像的加权中心进行比较，得到各所述子区域的子光线估计角度；确定单元，用于计算各所述子区域的子光照强度，根据所述子区域的子光照强度确定所述子区域的子光线估计角度的权重；计算单元，用于根据各所述子光线估计角度及所述子光线估计角度的权重，计算得到所述光线估计角度。
进一步地，所述比较模块还用于，计算所述光强加权中心与所述人脸图像的加权中心之间的距离值，基于所述距离值确定置信因子，所述置信因子反映所述鼻部区域中鼻尖相邻区域之间的对比度。
进一步地，所述比较模块还用于，计算各所述子光强加权中心与所述人脸图像的加权中心之间的子距离值，基于所述子距离值确定子置信因子；根据各所述子置信因子及所述子光线估计角度的权重，确定置信因子，所述置信因子反映所述鼻部区域中鼻尖相邻区域之间的对比度。
进一步地，所述比较模块还包括调整单元，所述调整单元用于，提取所述人脸图像中的眼窝区域，比较所述眼窝的光照强度；当所述眼窝的光照强度差值大于或等于预设阈值时，根据所述眼窝的光照强度确定所述人脸的头部倾斜角度；根据所述人脸倾斜角度所述光线估计角度进行调整。
本发明实施例的又一方面提供一种电子设备，包括：至少一个处理器；以及，与所述至少一个处理器通信连接的存储器；其中，
所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行上述的AR成像方法。
进一步的，所述电子设备还包括图像获取模块，所述图像获取模块包括镜头、自动聚焦音圈马达、机械防抖器以及图像传感器，所述镜头固装在所述自动聚焦音圈马达上，所述镜头用于获取图像，所述图像传感器将所述镜头获取的图像传输至所述识别模块，所述自动聚焦音圈马达安装在所述机械防抖器上，所述处理模块根据镜头内的陀螺仪检测到的镜头抖动的反馈驱动所述机械防抖器的动作，实现镜头的抖动补偿。
进一步的，所述机械防抖器包括活动板、活动框、弹性回复机构、基板以及补偿机构；所述活动板的中部设有供所述镜头穿过的通孔，所述自动聚焦音圈马达安装在所述活动板上，所述活动板安装在所述活动框内，所述活动板的相对两侧与所述活动框相对两侧的内壁滑动配合，以使所述活动板可沿着第一方向往复滑动；所述活动框的尺寸小于所述基板的尺寸，所述活动框的相对两侧分别通过两个弹性回复机构与所述基板相连接，所述基板的中部也设有所述镜头穿过的通孔；所述补偿机构在所述处理模块的驱动下带动所述活动板和活动板上的镜头动作，以实现镜头的抖动补偿；所述补偿机构包括驱动轴、齿轮、齿轮轨道以及限位轨道，所述驱动轴安装在所述基板上，所述驱动轴与所述齿轮传动连接；所述齿轮轨道设置在所述活动板上，所述齿轮安装在所述齿轮轨道内，所述齿轮转动时通过所述齿轮轨道使得所述活动板能够产生向第一方向的位移和第二方向的位移，其中所述第一方向与所述第二方向垂直；所述限位轨道设置在所述活动板上或所述基板上，所述限位轨道用于防止所述齿轮脱离所述齿轮轨道。
进一步的，所述活动板一侧设有一腰型孔，所述腰型孔沿其周向设有多个与所述齿轮相啮合的齿，所述腰型孔和多个所述齿共同构成所述齿轮轨道，所述齿轮位于所述腰型孔内并与所述齿相啮合；所述限位轨道设置在所述基板上，所述活动板底部设有位于所述限位轨道内的限位件，所述限位轨道使所述限位件在其中的运动轨迹呈腰型。
进一步的，所述限位件为设置在所述活动板底面上的凸起。
进一步的，所述齿轮轨道包括设置在所述活动板上的多个圆柱凸起，多个所述圆柱凸起沿所述第二方向均匀间隔布设，所述齿轮与多个所述凸起相啮合；所述限位轨道为设置在所述活动板上的第一弧形限位件和第二弧形限位件，所述第一弧形限位件和第二弧形限位件分别设置在所述齿轮轨道沿第一方向上的相对两侧，所述第一弧形限位件和第二弧形限位件及多个所述凸起相配合使得所述活动板的运动轨迹呈腰型。
进一步的，所述弹性回复机构包括伸缩弹簧。
进一步的，所述图像获取模块包括手机和用于安装所述手机的支架。
进一步的，所述支架包括手机安装座和可伸缩的支撑杆；所述手机安装座包括可伸缩的连接板和安装于连接板相对两端的折叠板组，所述支撑杆的一端与所述连接板中部通过阻尼铰链相连接；所述折叠板组包括第一板体、第二板体及第三板体，其中，所述第一板体的相对两端中的一端与所述连接板相铰接，所述第一板体的相对两端中的另一端与所述第二板体的相对两端中的一端相铰接；所述第二板体相对两端的另一端与所述第三板体相对两端中的一端相铰接；所述第二板体设有供手机边角插入的开口；所述手机安装座用于安装手机时，所述第一板体、第二板体和第三板体折叠呈直角三角形状态，所述第二板体为直角三角形的斜边，所述第一板体和所述第三板体为直角三角形的直角边，其中，所述第三板体的一个侧面与所述连接板的一个侧面并排贴合，所述第三板体相对两端中的另一端与所述第一板体相对两端中的一端相抵。
进一步的，所述第三板体的一个侧面设有第一连接部，所述连接板与所述第三板体相贴合的侧面设有与所述第一连接部相配合的第一配合部，所述支架手机安装座用于安装手机时，所述第一连接部和所述第一配合部卡合连接。
进一步的，所述第一板体相对两端中的一端设有第二连接部，所述第三板体相对两端中的另一端设有与所述第二连接部相配合的第二配合部，所述支架手机安装座用于安装手机时，所述第二连接部和所述第二配合部卡合连接。
进一步的，所述支撑杆的另一端可拆卸连接有底座。
由以上技术方案可见，本发明实施例提供的AR成像方法、装置及电子设备，只需使用现有的双摄像头的移动终端即可获取真实世界中的光线信息，为AR提供更逼真的效果，且降低了成本；手机摄像头的防抖动硬件结构和手机自拍支架也进一步增强了拍摄效果，更利于后续的图像或视频处理。
附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明实施例中记载的一些实施例，对于本领域普通技术人员来讲，还可以根据这些附图获得其他的附图。
图1为本发明一个实施例提供的AR成像方法流程图；
图2为本发明一个实施例提供的AR成像方法流程图；
图3为本发明一个实施例提供的AR成像方法流程图；
图4为本发明一个实施例提供的AR成像装置结构图；
图5为本发明一个实施例提供的AR成像装置结构图；
图6为执行本发明方法实施例提供的AR成像方法的电子设备的硬件结构示意图；
图7为本发明一个实施例提供的图像获取模块的结构示意图；
图8为本发明一个实施例提供的第一种机械防抖器的结构示意图；
图9为本发明一个实施例提供的第一种活动板的底面结构示意图；
图10为本发明一个实施例提供的第二种机械防抖器的结构示意图；
图11为本发明一个实施例提供的第二种活动板的底面结构示意图；
图12为本发明一个实施例提供的支架的结构图；
图13为本发明一个实施例提供的支架的一个状态示意图；
图14为本发明一个实施例提供的支架的另一个状态示意图；
图15为本发明一个实施例提供的安装座与手机相连接时的结构状态图。
具体实施方式
为了使本领域的人员更好地理解本发明实施例中的技术方案，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅是本发明实施例一部分实施例，而不是全部的实施例。基于本发明实 施例中的实施例，本领域普通技术人员所获得的所有其他实施例，都应当属于本发明实施例保护的范围。
下面结合附图，对本发明的一些实施方式作详细说明。在不冲突的情况下，下述的实施例及实施例中的特征可以相互结合。
图1为本发明实施例提供的AR成像方法流程图。如图1所示，本发明实施例提供的AR成像方法，包括：
S101，检测移动终端拍摄到的人脸图像，提取所述人脸图像中鼻部区域的子图像。
实施例中，我们提供了一种低成本、便捷和准确的方式，在用户戴着AR眼镜、利用移动终端的后置摄像头进行AR视频的观看时，可以通过移动终端的前置摄像头来确定真实世界当前场景的光线条件。由于用户大致面对着移动终端的屏幕，因此，只需将在人脸图像中检测到的环境光的光线角度镜像到需要添加到真实世界当前场景中的虚拟对象即可。
具体地，当需要检测当前真实场景的环境光时，通过前置摄像头来拍摄当前场景的快照图像，当检测到该快照图像中包括用户的人脸图像时，通过本领域技术人员熟知的人脸检测算法(例如，Eli Saber等发表的基于颜色、形状和对称性的成本函数进行正面人脸检测和面部特征提取；Karin Sobottka等发表的一种新的自动人脸分割、人脸特征提取和跟踪方法；Phillip Ian Wilson等发表的基于分类器的人脸特征检测方法；Alan L.Yuille等发表的使用可变形模板从人脸中提取特征)者任何基于CNN的人脸检测算法等，从该图像中提取人脸特征，由于五官中鼻子是凸出于人脸的，鼻部区域的明暗更容易受到环境光的影响，因此本实施例提取人脸特征中的鼻子，得到人脸图像中鼻部区域的子图像。
S102，基于光线确定所述子图像的光强加权中心，将所述光强加权中心与所述人脸图像的加权中心进行比较，得到光线估计角度。
在本步骤中，根据子图像的图像矩来确定其对应的光强加权中心。图像矩是一个从数字图形中计算出来的矩集，通常描述了该图像的全局特征，并提供大量的关于该图像不同类型的几何特征信息，比如大小、位置、方向及形状等，例如，一阶矩与形状有关、二阶矩显示了曲线围绕直线平均值的扩展程度、三阶矩则是关于平均值的对称性测量，由二阶矩和三阶矩可以导出一组共有7个不变阶矩，不变阶矩是图像统计特征，其可以据此对图像进行分类操作，属于本领域的公知常识，本发明在此不再赘述。
可选地，在确定了子图片的光强加权中心之后，将其与进行比较(该加权中心即为图像的集合中心)，将图像的加权中心的坐标位置与子图片的光强加权中心的坐标位置进行比较，从几何中心到光强加权中心的方向即是环境光的光线方向，同时，我们可以通过选择坐标原点来建立坐标系，得到该向量与X轴的夹角，作当前场景的环境光的光线估计角度。此外，也可以通过其他非专有算法来对光线估计角度进行计算，本发明在此不做限制。需要说明的是，在本发明实施例中，环境光将被认为是单向的和均匀的。
作为本发明实施例的一些可选实施方式，如图2所示，可以通过如下步骤得到光线估计角度。
S1021，将所述子图像分为若干个子区域，确定各所述子区域的子光强加权中心。
S1022，将各所述子光强加权中心与所述人脸图像的加权中心进行比较，得到各所述子区域的子光线估计角度。
S1023，计算各所述子区域的子光照强度，根据所述子区域的子光照强度确定所述子区域的子光线估计角度的权重。
S1024，根据各所述子光线估计角度及所述子光线估计角度的权重，计算得到所述光线估计角度。
具体地，首先，可以将子图像平均分成四份，得到四个子区域，按照上述方法确定每个子区域的子光强加权中心、以及每个子区域的子光线估计角度；其次，对于每一张子图片，根据其中的明暗对比度信息等获得该子图片对应的光线强度，在得到每个子区域的子光照强度后，将每个子区域的子光照强度作为该子区域的子光线估计角度的权重；最后，对四个子区域的子光线估计角度根据其分别对应的权重进行加和求平均值的计算，得到平均光线估计角度。
作为本发明实施例的可选实施方式，还可以用过如下方式确定计算得到的光线估计角度是否准确。
作为第一种实施方式，计算所述光强加权中心与所述人脸图像的加权中心之间的距离值，基于所述距离值确定置信因子，所述置信因子反映所述鼻部区域中鼻尖相邻区域之间的对比度。
通过测量人脸图像加权中心和光照强度加权中心之间的距离，我们可以获得一个距离值，该值随着鼻部区域阴影部分和发光部分之间的对比度的增加而增加。在没有特定光方向周围漫射光环境中，该距离值将很小或为零。因此，这个距离值提供了鼻部区域中与鼻尖相邻区域之间的对比度相关的置信度因素。在人脸进行拍摄时基 本都是存在特定光源的，因此该距离值越大，说明置信度越大，光线估计角度的准确性也就越高，也就越能确定这个角度代表环境光的光线方向。
作为第二种实施方式，计算各所述子光强加权中心与所述人脸图像的加权中心之间的子距离值，基于所述子距离值确定子置信因子；根据各所述子置信因子及所述子光线估计角度的权重，确定置信因子，所述置信因子反映所述鼻部区域中鼻尖相邻区域之间的对比度。
当步骤S102通过对鼻部区域分解为若干个子图像进行光线估计角度的计算时，可以通过第二种实施方式进行置信因子的确定，从而提高置信因子的准确度。
作为本发明实施例的可选实施方式，我们在关注人脸鼻部区域的同时，还可以根据需要，特别是在检测到低置信度的情况下，通过关注用户的眼窝来增加结果的置信度和准确性。
具体地，如图3所示，所述方法包括：
S1021’，提取所述人脸图像中的眼窝区域，比较所述眼窝的光照强度。
S1022’，当所述眼窝的光照强度差值大于或等于预设阈值时，根据所述眼窝的光照强度确定所述人脸的头部倾斜角度。
S1023’，根据所述人脸倾斜角度所述光线估计角度进行调整。
具体地，如果头部没有发生偏转时，人脸两个眼窝的光照强度应该是相同或者相差很小的，当两个眼窝的光照强度差值大于或等于预设阈值时，说明在步骤S101拍摄到的人脸图像时，用户的头部是存在偏转的。可选地，可以通过预先进行机器模型学习训练来得到倾斜角度模型，将光照强度差值输入值该倾斜角度模型中，得到人脸的头部倾斜角度，需要说明的是，该头部倾斜角度指的是相对于头部与水平保持90度位置时的角度。可以将该倾斜角度与直角的商作为修正系数，基于该修正系数对光线估计角度进行调整，从而解决来自头部偏转对结果产生的影响。
S103，根据所述光线估计角度和所述移动终端的旋转角度，得到真实场景的光线角度，其中，所述真实场景为所述移动终端当前拍摄的真实世界。
在本步骤中，由于用户的人脸图像时前端摄像头拍摄的，而AR视频是通过后端摄像头进行观看的，因此需要对上述光线估算角度进行镜像操作，以实现面向图像的切换。此外，由于用户在观看AR视频的过程中，可能会对移动终端进行旋转，因此，需要根据光线估计角度和所述移动终端的旋转角度来构建真实场景下的环境光的光线角度。可选地，可以根据移动终端的旋转角度来旋转光线估计角度，从而得到真实场景的光线角度。
S104，根据所述真实场景的光线角度和虚拟对象在所述真实场景中的预设位置，生成所述虚拟对象的阴影图像。
在本步骤中，首先可以根据真实场景的光线角度和虚拟对象的预设位置，确定虚拟对象在目标图片中的阴影位置。其次，根据虚拟对象的形状、确定在阴影位置的阴影形状。根据阴影位置和阴影形状，生成虚拟对象的阴影图像。用户在佩戴AR眼镜观看AR视频时，可以看到虚拟对象的阴影效果，即基于当前真实场景光线得到的虚拟对象的逼真反射效果。
本发明实施例提供的AR成像方法，只需使用现有的双摄像头的移动终端即可获取真实世界中的光线信息，为AR提供更逼真的效果，且降低了成本；此外，本发明实施例可以应用在白天或晚上的室内外条件下，也不需要知道真实世界的时间及利用GPS定位获取本地位置，对移动终端设备的性能要求低。
图4为本发明实施例提供的AR成像装置结构图。如图3所示，该装置具体包括：提取模块100、比较模块200、获取模块300和生成模块400。其中，
提取模块100，用于检测移动终端拍摄到的人脸图像，提取所述人脸图像中鼻部区域的子图像；比较模块200，用于基于光线确定所述子图像的光强加权中心，将所述光强加权中心与所述人脸图像的加权中心进行比较，得到光线估计角度；获取模块300，用于根据所述光线估计角度和所述移动终端的旋转角度，得到真实场景的光线角度，其中，所述真实场景为所述移动终端当前拍摄的真实世界；生成模块400，用于根据所述真实场景的光线角度和虚拟对象在所述真实场景中的预设位置，生成所述虚拟对象的阴影图像。
本发明实施例提供的AR成像装置具体用于执行图1所示实施例提供的所述方法，其实现原理、方法和功能用途等与图1所示实施例类似，在此不再赘述。
图5为本发明实施例提供的AR成像装置结构图。如图5所示，该装置具体包括：提取模块100、比较模块200、获取模块300和生成模块400。其中，
提取模块100，用于检测移动终端拍摄到的人脸图像，提取所述人脸图像中鼻部区域的子图像；比较模块200，用于基于光线确定所述子图像的光强加权中心，将所述光强加权中心与所述人脸图像的加权中心进行比较，得到光线估计角度；获取模块300，用于根据所述光线估计角度和所述移动终端的旋转角度，得到真实场景的光线角度，其中，所述真实场景为所述移动终端当前拍摄的真实世界；生成模块400，用于根据所述真实场景的光线角度和虚拟对象在所述真实场景中的预设位置，生成所述虚拟对象的阴影图像。
具体地，比较模块200包括分解单元210、比较单元220、确定单元230和计算单元240。其中，
分解单元210，用于将所述子图像分为若干个子区域，确定各所述子区域的子光强加权中心；比较单元220，用于将各所述子光强加权中心与所述人脸图像的加权中心进行比较，得到各所述子区域的子光线估计角度确定单元230，用于计算各所述子区域的子光照强度，根据所述子区域的子光照强度确定所述子区域的子光线估计角度的权重；计算单元240，用于根据各所述子光线估计角度及所述子光线估计角度的权重，计算得到所述光线估计角度。
具体地，比较模块200还包括调整单元250，调整单元250用于，提取所述人脸图像中的眼窝区域，比较所述眼窝的光照强度；当所述眼窝的光照强度差值大于或等于预设阈值时，根据所述眼窝的光照强度确定所述人脸的头部倾斜角度；根据所述人脸倾斜角度所述光线估计角度进行调整。
可选地，比较模块200还用于，计算所述光强加权中心与所述人脸图像的加权中心之间的距离值，基于所述距离值确定置信因子，所述置信因子反映所述鼻部区域中鼻尖相邻区域之间的对比度。
可选地，比较模块200还用于，计算各所述子光强加权中心与所述人脸图像的加权中心之间的子距离值，基于所述子距离值确定子置信因子；根据各所述子置信因子及所述子光线估计角度的权重，确定置信因子，所述置信因子反映所述鼻部区域中鼻尖相邻区域之间的对比度。
可选地，比较模块200还包括调整单元250，调整单元250用于，提取所述人脸图像中的眼窝区域，比较所述眼窝的光照强度；当所述眼窝的光照强度大于或等于预设阈值时，根据所述眼窝的光照强度确定所述人脸的头部倾斜角度；根据所述人脸倾斜角度所述光线估计角度进行调整。
本发明实施例提供的AR成像装置具体用于执行图1-图3所示实施例提供的所述方法，其实现原理、方法和功能用途和图1-图3所示实施例类似，在此不再赘述。
上述这些本发明实施例的AR成像装置可以作为其中一个软件或者硬件功能单元，独立设置在上述电子设备中，也可以作为整合在处理器中的其中一个功能模块，执行本发明实施例的AR成像方法。
根据图6所示，本发明实施例提供的电子设备包括：一个或多个处理器610以及存储器620，图6中以一个处理器610为例。执行所述的AR成像方法的设备还可以包括：输入装置630和输出装置630。其中，
处理器610、存储器620、输入装置630和输出装置640可以通过总线或者其他方式连接，图6中以通过总线连接为例。
存储器620作为一种非易失性计算机可读存储介质，可用于存储非易失性软件程序、非易失性计算机可执行程序以及模块，如本发明实施例中的所述AR成像方法对应的程序指令/模块。处理器610通过运行存储在存储器620中的非易失性软件程序、指令以及模块，从而执行服务器的各种功能应用以及数据处理，即实现所述AR成像方法。
存储器620可以包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需要的应用程序；存储数据区可存储根据本发明实施例提供的AR成像装置的使用所创建的数据等。此外，存储器620可以包括高速随机存取存储器620，还可以包括非易失性存储器620，例如至少一个磁盘存储器620件、闪存器件、或其他非易失性固态存储器620件。在一些实施例中，存储器620可选包括相对于处理器66远程设置的存储器620，这些远程存储器620可以通过网络连接至所述AR成像装置。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
输入装置630可接收输入的数字或字符信息，以及产生与AR成像装置的用户设置以及功能控制有关的键信号输入。输入装置630可包括按压模组等设备。
所述一个或者多个模块存储在所述存储器620中，当被所述一个或者多个处理器610执行时，执行所述AR成像方法。
本发明实施例的电子设备以多种形式存在，包括但不限于:
(1)移动通信设备:这类设备的特点是具备移动通信功能，并且以提供话音、数据通信为主要目标。这类终端包括:智能手机(例如iPhone)、多媒体手机、功能性手机，以及低端手机等。
(2)超移动个人计算机设备:这类设备属于个人计算机的范畴，有计算和处理功能，一般也具备移动上网特性。这类终端包括:PDA、MID和UMPC设备等，例如iPad。
(3)便携式娱乐设备:这类设备可以显示和播放多媒体内容。该类设备包括:音频、视频播放器(例如iPod)，掌上游戏机，电子书，以及智能玩具和便携式车载导航设备。
(4)服务器。
(6)其他具有数据交互功能的电子装置。
具体的，电子设备中包括图像获取模块，如图7所示，本实施例的所述图像获取模块包括镜头1000、自动聚焦音圈马达2000、机械防抖器3000以及图像传感器 4000，所述镜头1000固装在所述自动聚焦音圈马达2000上，所述镜头1000固装在所述自动聚焦音圈马达2000上，所述镜头1000用于获取图像，所述图像传感器4000将所述镜头1000获取的图像传输至所述识别模块，所述自动聚焦音圈马达2000安装在所述机械防抖器3000上，所述处理模块根据镜头1000内的陀螺仪检测到的镜头1000抖动的反馈驱动所述机械防抖器3000的动作，实现镜头1000的抖动补偿。
现有的防抖器大多由通电线圈在磁场中产生洛伦磁力驱动镜头1000移动，而要实现光学防抖，需要在至少两个方向上驱动镜头1000，这意味着需要布置多个线圈，会给整体结构的微型化带来一定挑战，而且容易受外界磁场干扰，进而影响防抖效果，因此公开号为CN106131435A的中国专利提供了一种微型光学防抖摄像头模组，其通过温度变化实现记忆合金丝的拉伸和缩短，以此拉动自动聚焦音圈马达2000移动，实现镜头1000的抖动补偿，微型记忆合金光学防抖致动器的控制芯片可以控制驱动信号的变化来改变记忆合金丝的温度，以此控制记忆合金丝的伸长和缩短，并且根据记忆合金丝的电阻来计算致动器的位置和移动距离。当微型记忆合金光学防抖致动器上移动到指定位置后反馈记忆合金丝此时的电阻，通过比较这个电阻值与目标值的偏差，可以校正微型记忆合金光学防抖致动器上的移动偏差。
但是申请人发现，由于抖动的随机性和不确定性，仅仅依靠上述技术方案的结构是无法实现在多次抖动发生的情况下能够对镜头1000进行精确的补偿，这是由于形状记忆合金的升温和降温均需要一定的时间，当抖动向第一方向发生时，上述技术方案可以实现镜头1000对第一方向抖动的补偿，但是当随之而来的第二方向的抖动发生时，由于记忆合金丝来不及在瞬间变形，因此容易造成补偿不及时，无法精准实现对多次抖动和不同方向的连续抖动的镜头1000抖动补偿，因此需要对其结构上进行改进。
结合附图8-11所示，本实施例对光学防抖器进行改进，将其设计为机械防抖器3000，其具体结构如下：
本实施例的所述机械防抖器3000包括活动板3100、活动框3200、弹性回复机构3300、基板3400以及补偿机构3500；所述活动板3100和基板3400的中部均设有供所述镜头穿过的通孔3700，所述自动聚焦音圈马达安装在所述活动板3100上，所述活动板3100安装在所述活动框3200内，由图中可以看出，本实施例的活动板3100的左右方向的宽度与活动框3200的内部宽度大致相同，使得所述活动板3100的相对两侧(左右两侧)与所述活动框3200相对两侧(左右两侧)的内壁滑动配合，以使所述活动板3100在活动框3200内可沿着第一方向往复滑动，本实施例的第一方向即 图中的上下方向。
具体的，本实施例的所述活动框3200的尺寸小于所述基板3400的尺寸，所述活动框3200的相对两侧分别通过两个弹性回复机构3300与所述基板3400相连接，本实施例的弹性回复机构3300为伸缩弹簧或者其他弹性件，而且需要说明的是，本实施例的弹性回复机构3300只允许其具有沿着图中左右方向(即下述的第二方向)伸缩和回弹的能力，无法沿着第一方向运动，设计弹性回复机构3300的目的也是在于当活动框3200发生补偿位移后便于活动框3200带动活动板3100复位，而具体的动作过程本实施例会在下述的工作过程进行具体描述。
本实施例的所述补偿机构3500在所述处理模块的驱动(可以是处理模块发送的动作指令)下带动所述活动板3100和活动板3100上的镜头动作，以实现镜头的抖动补偿。
具体的，本实施例的所述补偿机构3500包括驱动轴3510、齿轮3520、齿轮轨道3530以及限位轨道3540，所述驱动轴3510安装在所述基板3400上，具体是安装在基板3400的上表面，所述驱动轴3510与所述齿轮3520传动连接，驱动轴3510可以通过微型电机(图中未示出)等结构驱动，并且微型电机受控于上述的处理模块；所述齿轮轨道3530设置在所述活动板3100上，所述齿轮3520安装在所述齿轮轨道3530内并沿着所述齿轮轨道3530的预设方向运动，所述齿轮3520转动时通过所述齿轮轨道3530使得所述活动板3100能够产生向第一方向的位移和第二方向的位移，其中所述第一方向与所述第二方向垂直；所述限位轨道3540设置在所述活动板3100上或所述基板3400上，所述限位轨道3540用于防止所述齿轮3520脱离所述齿轮轨道3530。
具体的，本实施例的齿轮轨道3530和限位轨道3540具有以下两种结构形式：
如图7-9所示，本实施例的所述活动板3100下侧设有一腰型孔3550，所述腰型孔3550沿其周向(即腰型孔3550的环绕方向)设有多个与所述齿轮3520相啮合的齿3560，所述腰型孔3550和多个所述齿3560共同构成所述齿轮轨道3530，所述齿轮3520位于所述腰型孔3550内并与所述齿3560相啮合，使得齿轮3520在转动时能够带动齿轮轨道3530运动，进而直接带动活动板3100的运动；而为了使得齿轮3520在转动时时刻能够保证与齿轮轨道3530时刻保持啮合，本实施例将所述限位轨道3540设置在所述基板3400上，所述活动板3100底部设有安装在所述限位轨道3540内的限位件3570，所述限位轨道3540使所述限位件3570在其中的运动轨迹呈腰型，即限位件3570在现为轨道内的运动轨迹与活动板3100的运动轨迹相同，具体的，本 实施例的所述限位件3570为设置在所述活动板3100底面上的凸起。
再结合附图10和11所示，本实施例的所述齿轮轨道3530还可以由设置在所述活动板3100上的多个圆柱凸起3580组成，多个所述圆柱凸起3580沿所述第二方向均匀间隔布设，所述齿轮3520与多个所述凸起相啮合；而所述限位轨道3540为设置在所述活动板3100上的第一弧形限位件3590和第二弧形限位件3600，所述第一弧形限位件3590和第二弧形限位件3600分别设置在所述齿轮轨道3530沿第一方向上的相对两侧，防，在活动板3100运动至预设位置时，齿轮3520位于齿轮轨道3530的一侧，齿轮3520容易脱离圆柱凸起3580组成的齿轮轨道3530，因此通过第一弧形限位件3590或第二弧形限位件3600可以起到引导作用，使得活动板3100能够沿着齿轮轨道3530的预设方向运动，即所述第一弧形限位件3590和第二弧形限位件3600及多个所述凸起相配合使得所述活动板3100的运动轨迹呈腰型。
下面结合上述结构对本实施例的机械防抖器3000的工作过程进行详细的描述，以镜头1000两次抖动为例，两次抖动方向相反，且需要使得活动板3100向第一方向运动补偿一次，并随后向第二方向运动补偿一次。需要活动板3100向第一方向运动补偿时，陀螺仪事先将检测到的镜头1000抖动方向和距离反馈给所述处理模块，处理模块计算出需要活动板3100的运动距离，进而使得驱动轴3510带动齿轮3520转动，齿轮3520通过与齿轮轨道3530和限位轨道3540的配合，处理模块无线发送驱动信号，进而带动活动板3100向第一方向运动至补偿位置，补偿后再次通过驱动轴3510带动活动板3100复位，在复位过程中，弹性回复机构3300也为活动板3100的复位提供复位力，更加便于活动板3100恢复到初始位置。需要活动板3100向第二方向运动补偿时，处理方式与上述第一方向的补偿步骤相同，这里不过多赘述。
当然上述仅仅为简单的两次抖动，当发生多次抖动时，或者抖动的方向并非往复运动时，可以通过驱动多个补偿组件以补偿抖动，其基础工作过程与上述描述原理相同，这里不过多赘述，另外关于陀螺仪的检测反馈、处理模块向驱动轴3510发送控制指令等均为现有技术，这里也不过多描述。
结合上述说明可知，本实施例提供的机械补偿器不仅不会受到外界磁场干扰，防抖效果好，而且可以实现在多次抖动发生的情况下能够对镜头1000进行精确的补偿，补偿及时准确。另外，采用本实施例的机械防抖器不仅结构简单，各个部件所需的安装空间小，便于整个防抖器的集成化，而且具有较高的补偿精度。
具体的，本实施例的所述电子设备包括手机和用于安装所述手机的支架。电子设备包含支架的目的是由于图像获取环境的不确定性，因此需要使用支架对电子设 备进行支撑和固定。
另外，申请人发现，现有的手机支架仅仅具有支撑手机的功能，而不具有自拍杆的功能，因此申请人对支架做出第一步改进，将手机支架5000和支撑杆5200相结合，结合附图12所示，本实施例的所述支架5000包括手机安装座5100和可伸缩的支撑杆5200，支撑杆5200与手机安装座5100的中部(具体为下述基板3200的中部)通过阻尼铰链相连接，使得支撑杆5200在转动至图13的状态时，支架5000可形成自拍杆结构，而支撑杆5200在转动至图14的状态时，支架5000可形成手机支架5000结构。
而结合上述支架结构申请人又发现，手机安装座5100与支撑杆5200结合后占用空间较大，即使支撑杆5200可伸缩，但是手机安装座5100无法进行结构的变化，体积不会进一步缩小，无法将其放入衣兜或者小型的包内，造成支架5000携带不便的问题，因此本实施例对支架5000做出第二步改进，使得支架5000的整体收容性得到进一步的提高。
结合图12-15所示，本实施例的所述手机安装座5100包括可伸缩的连接板5110和安装于连接板5110相对两端的折叠板组5120，所述支撑杆5200与所述连接板5110中部通过阻尼铰链相连接；所述折叠板组5120包括第一板体5121、第二板体5122及第三板体5123，其中，所述第一板体5121的相对两端中的一端与所述连接板5110相铰接，所述第一板体5121的相对两端中的另一端与所述第二板体5122的相对两端中的一端相铰接；所述第二板体5122相对两端的另一端与所述第三板体5123相对两端中的一端相铰接；所述第二板体5122设有供手机边角插入的开口5130。
结合附图15所示，所述手机安装座5100用于安装手机时，所述第一板体5121、第二板体5122和第三板体5123折叠呈直角三角形状态，所述第二板体5122为直角三角形的斜边，所述第一板体5121和所述第三板体5123为直角三角形的直角边，其中，所述第三板体5123的一个侧面与所述连接板5110的一个侧面并排贴合，所述第三板体5123相对两端中的另一端与所述第一板体5121相对两端中的一端相抵，该结构可以使得三个折叠板处于自锁状态，并且将手机下部的两个边角插入到两侧的两个开口5130时，手机6000的下部两侧位于两个直角三角形内，通过手机、连接板5110和折叠板组5120件的共同作可以完成手机6000的固定，三角形状态在外力情况下无法打开，只有从开口5130抽出手机后才能解除折叠板组5120件的三角形状态。
而在手机安装座5100不处于工作状态时，将连接板5110缩小至最小长度，并且将折叠板组5120件与连接板5110相互折叠，用户可以将手机安装座5100折叠呈最 小体积，而由于支撑杆5200的可伸缩性，因此可以将整个支架5000收容呈体积最小的状态，提高了支架5000的收荣幸，用户甚至可以直接将支架5000放入衣兜或小的手包内，十分方便。
优选的，本实施例还在所述第三板体5123的一个侧面设有第一连接部，所述连接板5110与所述第三板体5123相贴合的侧面设有与所述第一连接部相配合的第一配合部，所述支架5000手机安装座5100用于安装手机时，所述第一连接部和所述第一配合部卡合连接。具体的，本实施例的第一连接部为一个凸条或凸起(图中未示出)，第一配合部为开设在连接板5110上的卡槽(图中未示出)。该结构不仅提高了折叠板组5120件处于三角形状态时的稳定性，而且在需要将手机安装座5100折叠至最小状态时也便于折叠板组5120件与连接板5110的连接。
优选的，本实施例还在所述第一板体5121相对两端中的一端设有第二连接部，所述第三板体5123相对两端中的另一端设有与所述第二连接部相配合的第二配合部，所述支架5000手机安装座5100用于安装手机时，所述第二连接部和所述第二配合部卡合连接。第二连接部可以是凸起(图中未示出)，第二配合部为与凸起相配合的开口5130或卡槽(图中未示出)。该结构提高了叠板组件处于三角形状态时的稳定性
另外，本实施例还可以在所述支撑杆5200的另一端可拆卸连接有底座(图中未示出)，在需要固定手机并且使手机6000具有一定高度时，可以将支撑杆5200拉伸呈一定长度，并通过底座将支架5000置于一个平面上，再将手机放置到手机安装座5100内，完成手机的固定；而支撑杆5200和底座的可拆卸连接可以使得两者可以单独携带，进一步提高了支架5000的收容性和携带的方便性。
以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的模块可以是或者也可以不是物理上分开的，作为模块显示的部件可以是或者也可以不是物理模块，即可以位于一个地方，或者也可以分布到多个网络模块上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。本领域普通技术人员在不付出创造性的劳动的情况下，即可以理解并实施。
本发明实施例提供了一种非暂态计算机可读存存储介质，所述计算机存储介质存储有计算机可执行指令，其中，当所述计算机可执行指令被电子设备执行时，使所述电子设备上执行上述任意方法实施例中的AR成像方法。
本发明实施例提供了一种计算机程序产品，其中，所述计算机程序产品包括存储在非暂态计算机可读存储介质上的计算机程序，所述计算机程序包括程序指令，其 中，当所述程序指令被电子设备执行时，使所述电子设备执行上述任意方法实施例中的AR成像方法。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到各实施方式可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件。基于这样的理解，上述技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品可以存储在计算机可读存储介质中，所述计算机可读记录介质包括用于以计算机(例如计算机)可读的形式存储或传送信息的任何机制。例如，机器可读介质包括只读存储器(ROM)、随机存取存储器(RAM)、磁盘存储介质、光存储介质、闪速存储介质、电、光、声或其他形式的传播信号(例如，载波、红外信号、数字信号等)等，该计算机软件产品包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行各个实施例或者实施例的某些部分所述的方法。
最后应说明的是：以上实施例仅用以说明本发明实施例的技术方案，而非对其限制；尽管参照前述实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的精神和范围。

Claims (10)

  1. 一种AR成像方法，其特征在于，包括：
    检测移动终端拍摄到的人脸图像，提取所述人脸图像中鼻部区域的子图像；
    基于光线确定所述子图像的光强加权中心，将所述光强加权中心与所述人脸图像的加权中心进行比较，得到光线估计角度；
    根据所述光线估计角度和所述移动终端的旋转角度，得到真实场景的光线角度，其中，所述真实场景为所述移动终端当前拍摄的真实世界；
    根据所述真实场景的光线角度和虚拟对象在所述真实场景中的预设位置，生成所述虚拟对象的阴影图像。
  2. 根据权利要求1所述的方法，其特征在于，所述基于光线确定所述子图像的光强加权中心，将所述光强加权中心与所述人脸图像的加权中心进行比较，得到光线估计角度，包括：
    将所述子图像分为若干个子区域，确定各所述子区域的子光强加权中心；
    将各所述子光强加权中心与所述人脸图像的加权中心进行比较，得到各所述子区域的子光线估计角度；
    计算各所述子区域的子光照强度，根据所述子区域的子光照强度确定所述子区域的子光线估计角度的权重；
    根据各所述子光线估计角度及所述子光线估计角度的权重，计算得到所述光线估计角度。
  3. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    计算所述光强加权中心与所述人脸图像的加权中心之间的距离值，基于所述距离值确定置信因子，所述置信因子反映所述鼻部区域中鼻尖相邻区域之间的对比度。
  4. 根据权利要求2所述的方法，其特征在于，所述方法还包括：
    计算各所述子光强加权中心与所述人脸图像的加权中心之间的子距离值，基于所述子距离值确定子置信因子；
    根据各所述子置信因子及所述子光线估计角度的权重，确定置信因子，所述置信因子反映所述鼻部区域中鼻尖相邻区域之间的对比度。
  5. 根据权利要求1-4任一项所述的方法，其特征在于，所述方法还包括：
    提取所述人脸图像中的眼窝区域，比较所述眼窝的光照强度；
    当所述眼窝的光照强度差值大于或等于预设阈值时，根据所述眼窝的光照强度确定所述人脸的头部倾斜角度；
    根据所述人脸倾斜角度所述光线估计角度进行调整。
  6. 一种AR成像装置，其特征在于，包括：
    提取模块，用于检测移动终端拍摄到的人脸图像，提取所述人脸图像中鼻部区域的子图像；
    比较模块，用于基于光线确定所述子图像的光强加权中心，将所述光强加权中心与所述人脸图像的加权中心进行比较，得到光线估计角度；
    获取模块，用于根据所述光线估计角度和所述移动终端的旋转角度，得到真实场景的光线角度，其中，所述真实场景为所述移动终端当前拍摄的真实世界；
    生成模块，用于根据所述真实场景的光线角度和虚拟对象在所述真实场景中的预设位置，生成所述虚拟对象的阴影图像。
  7. 根据权利要求6所述的装置，其特征在于，所述比较模块包括：
    分解单元，用于将所述子图像分为若干个子区域，确定各所述子区域的子光强加权中心；
    比较单元，用于将各所述子光强加权中心与所述人脸图像的加权中心进行比较，得到各所述子区域的子光线估计角度
    确定单元，用于计算各所述子区域的子光照强度，根据所述子区域的子光照强度确定所述子区域的子光线估计角度的权重；
    计算单元，用于根据各所述子光线估计角度及所述子光线估计角度的权重，计算得到所述光线估计角度。
  8. 根据权利要求6所述的装置，其特征在于，所述比较模块还用于，
    计算所述光强加权中心与所述人脸图像的加权中心之间的距离值，基于所述距离值确定置信因子，所述置信因子反映所述鼻部区域中鼻尖相邻区域之间的对比度。
  9. 根据权利要求7所述的装置，其特征在于，所述比较模块还用于，计算各所述子光强加权中心与所述人脸图像的加权中心之间的子距离值，基于所述子距离值确定子置信因子；根据各所述子置信因子及所述子光线估计角度的权重，确定置信因子，所述置信因子反映所述鼻部区域中鼻尖相邻区域之间的对比度。
  10. 一种电子设备，其特征在于，包括：至少一个处理器；以及，
    与所述至少一个处理器通信连接的存储器；其中，
    所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行权利要求1至5中任一项所述的AR成像方法。
PCT/CN2018/106784 2018-09-20 2018-09-20 一种ar成像方法、装置及电子设备 WO2020056689A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2018/106784 WO2020056689A1 (zh) 2018-09-20 2018-09-20 一种ar成像方法、装置及电子设备
CN201811110297.1A CN109214351B (zh) 2018-09-20 2018-09-21 一种ar成像方法、装置及电子设备

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/106784 WO2020056689A1 (zh) 2018-09-20 2018-09-20 一种ar成像方法、装置及电子设备

Publications (1)

Publication Number Publication Date
WO2020056689A1 true WO2020056689A1 (zh) 2020-03-26

Family

ID=64985149

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/106784 WO2020056689A1 (zh) 2018-09-20 2018-09-20 一种ar成像方法、装置及电子设备

Country Status (2)

Country Link
CN (1) CN109214351B (zh)
WO (1) WO2020056689A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112016505A (zh) * 2020-09-03 2020-12-01 平安科技（深圳）有限公司 基于人脸图像的活体检测方法、设备、存储介质及装置

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110033423B (zh) * 2019-04-16 2020-08-28 北京字节跳动网络技术有限公司 用于处理图像的方法和装置
CN115039137A (zh) * 2020-01-30 2022-09-09 Oppo广东移动通信有限公司 基于亮度估计渲染虚拟对象的方法、用于训练神经网络的方法以及相关产品
CN111340931A (zh) * 2020-02-17 2020-06-26 广州虎牙科技有限公司 场景处理方法、装置、用户端及存储介质
CN112213860B (zh) * 2020-11-17 2023-04-18 闪耀现实(无锡)科技有限公司 一种增强现实装置、穿戴式增强现实设备及控制增强现实装置的方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104240277A (zh) * 2013-06-24 2014-12-24 腾讯科技（深圳）有限公司 基于人脸检测的增强现实交互方法和系统
CN105825544A (zh) * 2015-11-25 2016-08-03 维沃移动通信有限公司 一种图像处理方法及移动终端
CN107134005A (zh) * 2017-05-04 2017-09-05 网易（杭州）网络有限公司 光照适配方法、装置、存储介质、处理器及终端
CN107656619A (zh) * 2017-09-26 2018-02-02 广景视睿科技（深圳）有限公司 一种智能投影方法、系统及智能终端
CN107749075A (zh) * 2017-10-26 2018-03-02 太平洋未来科技（深圳）有限公司 视频中虚拟对象光影效果的生成方法和装置

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102426695A (zh) * 2011-09-30 2012-04-25 北京航空航天大学 一种单幅图像场景的虚实光照融合方法
CN103400119B (zh) * 2013-07-31 2017-02-15 徐坚 基于人脸识别技术的混合显示眼镜交互展示方法
US9852550B2 (en) * 2015-08-05 2017-12-26 Civic Resource Group International Inc. System and method of markerless injection of ads in AR
CN105182662B (zh) * 2015-09-28 2017-06-06 神画科技（深圳）有限公司 具有增强现实效果的投影方法及系统
CN105741343A (zh) * 2016-01-28 2016-07-06 联想(北京)有限公司 一种信息处理方法及电子设备
US10373385B2 (en) * 2016-12-14 2019-08-06 Microsoft Technology Licensing, Llc Subtractive rendering for augmented and virtual reality systems
CN108427498A (zh) * 2017-02-14 2018-08-21 深圳梦境视觉智能科技有限公司 一种基于增强现实的交互方法和装置
CN106940897A (zh) * 2017-03-02 2017-07-11 苏州蜗牛数字科技股份有限公司 一种在ar场景中介入现实光影的方法
CN107025683A (zh) * 2017-03-30 2017-08-08 联想(北京)有限公司 一种信息处理方法及电子设备
CN108305317B (zh) * 2017-08-04 2020-03-17 腾讯科技（深圳）有限公司 一种图像处理方法、装置及存储介质
CN107564089B (zh) * 2017-08-10 2022-03-01 腾讯科技（深圳）有限公司 三维图像处理方法、装置、存储介质和计算机设备
CN107749076B (zh) * 2017-11-01 2021-04-20 太平洋未来科技（深圳）有限公司 增强现实场景中生成现实光照的方法和装置
CN108021241B (zh) * 2017-12-01 2020-08-25 西安维度视界科技有限公司 一种实现ar眼镜虚实融合的方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104240277A (zh) * 2013-06-24 2014-12-24 腾讯科技（深圳）有限公司 基于人脸检测的增强现实交互方法和系统
CN105825544A (zh) * 2015-11-25 2016-08-03 维沃移动通信有限公司 一种图像处理方法及移动终端
CN107134005A (zh) * 2017-05-04 2017-09-05 网易（杭州）网络有限公司 光照适配方法、装置、存储介质、处理器及终端
CN107656619A (zh) * 2017-09-26 2018-02-02 广景视睿科技（深圳）有限公司 一种智能投影方法、系统及智能终端
CN107749075A (zh) * 2017-10-26 2018-03-02 太平洋未来科技（深圳）有限公司 视频中虚拟对象光影效果的生成方法和装置

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112016505A (zh) * 2020-09-03 2020-12-01 平安科技（深圳）有限公司 基于人脸图像的活体检测方法、设备、存储介质及装置
CN112016505B (zh) * 2020-09-03 2024-05-28 平安科技（深圳）有限公司 基于人脸图像的活体检测方法、设备、存储介质及装置

Also Published As

Publication number Publication date
CN109214351A (zh) 2019-01-15
CN109214351B (zh) 2020-07-07

Similar Documents

Publication Publication Date Title
WO2020056689A1 (zh) 一种ar成像方法、装置及电子设备
CN109151340B (zh) 视频处理方法、装置及电子设备
CN108596827B (zh) 三维人脸模型生成方法、装置及电子设备
WO2020056690A1 (zh) 一种视频内容关联界面的呈现方法、装置及电子设备
KR102365721B1 (ko) 모바일 기기를 이용한 3차원 얼굴 모델 생성 장치 및 방법
WO2020056692A1 (zh) 一种信息交互方法、装置及电子设备
CN108614638B (zh) Ar成像方法和装置
CN108377398B (zh) 基于红外的ar成像方法、系统、及电子设备
CN109285216B (zh) 基于遮挡图像生成三维人脸图像方法、装置及电子设备
CN107462992B (zh) 一种头戴显示设备的调节方法、装置及头戴显示设备
CN108966017B (zh) 视频生成方法、装置及电子设备
CN109271911B (zh) 基于光线的三维人脸优化方法、装置及电子设备
WO2020056691A1 (zh) 一种交互对象的生成方法、装置及电子设备
US9160931B2 (en) Modifying captured image based on user viewpoint
WO2019200718A1 (zh) 图像处理方法、装置及电子设备
WO2019200720A1 (zh) 基于图像处理的环境光补偿方法、装置及电子设备
CN114339102B (zh) 一种录像方法及设备
CN115209057A (zh) 一种拍摄对焦方法及相关电子设备
CN108282650B (zh) 一种裸眼立体显示方法、装置、系统及存储介质
WO2020056693A1 (zh) 一种图片合成方法、装置及电子设备
CN113891005A (zh) 拍摄方法、装置及电子设备
CN208479691U (zh) 一种基于虚拟现实技术的投影手机
WO2021026782A1 (zh) 手持云台的控制方法、控制装置、手持云台及存储介质
CN105527784B (zh) 成像辅助装置
CN111736692A (zh) 显示方法、显示装置、存储介质与头戴式设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18933803

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 12/08/2021)

122 Ep: pct application non-entry in european phase

Ref document number: 18933803

Country of ref document: EP

Kind code of ref document: A1