WO2018059258A1

WO2018059258A1 - Implementation method and apparatus for providing palm decoration virtual image using augmented reality technology

Info

Publication number: WO2018059258A1
Application number: PCT/CN2017/102044
Authority: WO
Inventors: 谭建国; 张洁; 闫晓亮
Original assignee: 阿里巴巴集团控股有限公司
Priority date: 2016-09-30
Filing date: 2017-09-18
Publication date: 2018-04-05
Also published as: CN107886387A

Abstract

Disclosed are an implementation method for providing a palm decoration virtual image using augmented reality technology and an implementation apparatus for providing a palm decoration virtual image using augmented reality technology. The implementation method for providing a palm decoration virtual image using augmented reality technology comprises: obtaining a palm image; determining a palm feature position of a palm in the image according to the palm image; superimposing, according to the palm feature position, a virtual image of a predetermined palm decoration to a position, on the palm image, matching the palm decoration to obtain the palm decoration virtual image; and displaying the palm decoration virtual image by means of a display device. The method can solve the problem in the prior art that a real palm and a virtual palm decoration cannot be superimposed into a sample image in real time.

Description

Method and device for providing palm decoration virtual image by using augmented reality technology

The present application claims priority from the Chinese Patent Application No. 201610872093.6 filed on Sep. 30, 2016, entitled "Implementation Method and Apparatus for Providing Palm-Decorated Virtual Images Using Augmented Reality Technology", the entire contents of which are incorporated by reference. In this application.

Technical field

The present invention relates to the field of analog simulation technologies, and in particular, to an implementation method and apparatus for providing a palm decorative virtual image by using augmented reality technology.

Background technique

With the rapid development of the Internet, online shopping has penetrated into people's lives, and online shopping platforms such as Taobao and Tmall have become household names. Users who buy palms, rings, and other palm decorations through the Internet are also increasing, but users don't know the effect of wearing a palm ornament. Therefore, there is a need for a technique that combines a palm, a finger, a wrist in the real world with a palm ornament in the screen to exhibit a try-on effect.

AR (Augmented Reality) augmented reality technology, which is a new technology that integrates real world information and virtual world information "seamlessly". It is an entity information that is difficult to experience in a certain time and space of the real world. Visual information, sound, taste, touch, etc., through computer and other science and technology, simulation and then superimposition, the virtual information is applied to the real world, perceived by human senses, thus achieving a sensory experience beyond reality. The real environment and virtual objects are superimposed in real time on the same picture or space.

Using augmented reality technology to provide potential buyers with the effect of wearing palm ornaments on their real palms, it becomes a possible solution to try out the palm decorations in online shopping; however, in this specific scenario, how to combine the palms Features Real-time superimposition of palm and palm decorations on the screen, the prior art does not provide a viable solution.

Summary of the invention

The invention provides an implementation method and device for providing a palm-decorated virtual image by using augmented reality technology, so as to solve the problem that the real palm and the virtual palm decoration cannot be superimposed on the same screen in real time under the prior art.

The embodiment of the present application provides an implementation method for providing a palm decoration virtual image by using an augmented reality technology, and the method includes:

Get the palm image;

Determining a palm feature position of the palm in the image according to the palm image;

And superimposing a virtual image of the predetermined palm decoration on the palm image at a position matching the palm decoration according to the palm feature position to obtain the palm decoration virtual image;

The palm decorative virtual image is displayed by a display device.

Optionally, before the step of acquiring a palm image, the method includes:

Get an image;

Determining whether there is a palm in the image; if so, separating the palm image in the image from the image; if not, returning to the step of acquiring the image.

Optionally, the acquiring an image includes:

The acquisition is acquired by the camera of the mobile terminal.

Optionally, the determining whether there is a palm in the image is implemented by using the following method:

The classifier obtained by training using the machine learning method judges whether or not the image has a palm.

Optionally, the machine learning method includes at least one of the following methods:

Hog-svm learning, Haaris corner learning, Haar LBP cascade classifier learning, and neural network learning.

Optionally, the method for separating a palm image in the image from the image includes at least one of the following methods:

Maximum inter-class variance method, skin color detection method.

Optionally, the feature location of the palm includes:

The palm position of the palm, the wrist position and the finger position in the palm.

Optionally, determining a palm position of the palm includes:

A method using a geometric center of gravity or a method of using a corner point of a finger.

Optionally, the position of the finger includes:

The position of the five fingers in the palm.

Optionally, determining the location of the finger includes:

Determine the direction line and finger line of your finger.

Optionally, the position of the finger includes:

The position of the fingertip of the finger, the position of the root of the finger, and the center point of the finger.

Optionally, determining the position of the fingertip of the finger includes:

The point of skin color that is the farthest from the position of the palm of the palm.

Optionally, determining the position of the root of the finger includes:

Focusing on the center of the palm from small to large, the part of the skin color is less than five degrees is the root of the finger.

Optionally, determining the center point of the finger is determined as follows:

A concentric circle is drawn centering on the palm, and a concentric circle and an inscribed concentric circle are centrally located at the intersection of the skin color in the direction of the finger.

Optionally, determining the position of the wrist in the palm, including:

Drawing a concentric circle centering on the palm of the hand, when the angle of the concentric circle with the skin color of the palm is greater than 10 degrees and less than 40 degrees, the area where the concentric circle contacts the part of the palm is the wrist in the palm, the wrist The direction is from the center of the palm to the direction of the midpoint of the intersecting arc.

Optionally, the position of the wrist in the palm includes:

The center point of the wrist.

Optionally, determining the center point of the wrist is as follows:

The concentric circles are drawn around the palm, and the concentric circles and the inner concentric circles are in the center of the intersection of the skin color in the direction of the wrist.

Optionally, the position on the palm image matching the palm decoration includes:

The center point of the finger of the palm or the center point of the wrist or the center point of the nail of the finger.

Optionally, the virtual image of the predetermined palm decoration includes:

Obtained from the network.

Optionally, the step of obtaining the virtual image of the palm decoration is performed by superimposing a virtual image of a predetermined palm decoration on the palm image at a position matching the palm decoration according to the palm feature position. Previously, it also included:

Determining whether a size of the virtual image of the predetermined palm ornament matches a position on the palm image that matches the palm ornament; if so, entering a virtual palm predetermined object according to the palm feature position The image is superimposed on the palm image at a position matching the palm ornament, and the step of decorating the virtual image of the palm is obtained; if not, the virtual image of the palm ornament is regained.

Optionally, the step of obtaining the virtual image of the palm decoration is performed by superimposing a virtual image of a predetermined palm decoration on the palm image at a position matching the palm decoration according to the palm feature position. After that, it also includes:

The palm decorative virtual image is stored.

The embodiment of the present application further provides an implementation device for providing a palm decoration virtual image by using an augmented reality technology, and the device includes:

An acquisition unit for acquiring a palm image;

a determining unit, configured to determine, according to the palm image acquired by the acquiring unit, a palm feature position of the palm in the image;

a superimposing unit, configured to superimpose a virtual image of the predetermined palm decoration on a position of the palm image matching the palm decoration according to the palm feature position determined by the determining unit, to obtain the palm decoration virtual image;

And a display unit, configured to display, by the display device, the palm decoration virtual image obtained by the superimposing unit.

Compared with the prior art, the present invention has the following advantages:

At present, the prior art does not adopt the technology of augmented reality, and the virtual palm decoration is worn in a real environment.

The present invention provides an implementation method for providing a palm-decorated virtual image by using augmented reality technology, comprising: acquiring a palm image; determining a palm feature position of the palm in the image according to the palm image; and determining a predetermined palm according to the palm feature position A virtual image of the decoration is superimposed on the palm image at a position matching the palm decoration to obtain the palm decoration virtual image; the palm decoration virtual image is displayed by the display device.

The present invention provides an implementation method for providing a palm decorative virtual image by using an augmented reality technology, first determining a palm feature position of a palm in the palm image, and then superimposing the virtual image of the palm ornament according to the palm feature position. The position on the palm image that matches the palm ornament. In the technical solution, the virtual palm makeup is virtually worn by the augmented reality method in a position matching the palm of the user, thereby realizing the wearing of the virtual palm decoration in a real environment; By displaying the virtual image of the palm decoration, the user can intuitively see the effect of wearing the virtual palm decoration on the palm of the real hand, and achieving the effect of trying on the trial.

DRAWINGS

FIG. 1 is a flowchart of a method for implementing a palm-decorated virtual image by using augmented reality technology according to a first embodiment of the present application.

FIG. 2 is a schematic diagram of an apparatus for implementing a palm-decorated virtual image using an augmented reality technology according to a second embodiment of the present application.

detailed description

The above objects, features and advantages of the present application will be more clearly understood from the following description of the appended claims. It should be noted that the embodiments in the present application and the features in the embodiments may be combined with each other without conflict.

Numerous specific details are set forth in the description below in order to provide a thorough understanding of the application. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art can make similar promotion without departing from the scope of the present application. Therefore, the present application is not limited by the specific embodiments disclosed below.

The first embodiment of the present application provides an implementation method for providing a palm decoration virtual image by using augmented reality technology. Please refer to FIG. 1 , which illustrates a flow chart of an implementation method for providing a palm-decorated virtual image using augmented reality technology according to an embodiment of the present application. The details will be described below in conjunction with FIG. 1.

In step S101, a palm image is acquired.

The purpose of this step is to provide a precondition for determining the palm feature position of the palm in the image according to the palm image in the subsequent step.

The palm image is an image containing only the palm separated from the image actually containing the palm; the palm image is used in the palm decorative virtual image and is a real image in the augmented reality image.

As previously mentioned, the palm image is separated from the image containing the palm; the image contains any image that may be provided. For any image, the following processing is required.

First, it is necessary to distinguish whether or not the palm image is included in the image, which is a premise for performing the separation processing; secondly, the palm image needs to be separated from the original image. The following are explained separately.

The step of distinguishing whether the image of the palm is included in the image is performed before the step of acquiring the image of the palm, the specific step comprising: acquiring an image; determining whether there is a palm in the image; and if so, separating the palm image in the image from the image Come out; if not, return to the step of getting the image.

The purpose of this step is to obtain an image containing the palm of the hand before acquiring the palm image, and then separate the palm from the background image. Whether or not the palm is present can be determined only by judging whether or not the palm is included in the image. The subsequent steps can only be performed if the palm is included in the acquired image, and if the palm does not exist, the image needs to be retrieved again.

There are many ways to obtain an image. In combination with a specific usage scenario, that is, the buyer uses his own palm as the palm of the augmented reality method, the following method can be adopted to acquire an image, that is, the acquisition is acquired by the camera of the mobile terminal.

The mobile terminal includes: a mobile phone or a tablet.

Since the current mobile phone or tablet has a camera and can browse the shopping website online, the mobile phone Or a tablet as a mobile terminal is very suitable for capturing images containing the palm of your hand.

In this embodiment, the image including the palm may be a frame with a palm image collected by the camera of the mobile phone, and displayed on the image display device after the capture, so that the user can see the palm of the hand very intuitively. And you can see the effect of trying to wear the palm makeup.

The determining whether there is a palm in the image is implemented by using a classifier obtained by training using a machine learning method to determine whether the image has a palm.

The machine learning method includes at least one of the following methods: Hog-svm learning, Haaris corner learning, Haar LBP cascade classifier learning, and neural network learning.

A brief description of the process of obtaining a classifier using machine learning methods is as follows.

Collect a variety of skin colors, positive samples of various hand palms, and collect negative samples of various scenes that do not include the palms. These materials are used as training materials and are provided for Hog-svm learning, Haaris corner learning, Haar LBP level. The classifier initial model constructed by means of joint classifier learning and neural network learning is learned, and the result after learning forms a trained classifier, and the trained classifier can be used to distinguish whether there is a palm in a picture. The accuracy of the trained classifier can be continuously improved as the training material increases; the classifier that can be used must be a classifier that has verified the accuracy to the required threshold.

After the step of distinguishing whether the palm image is included in the image is completed, the step of separating the palm image from the original image is further performed on the selected image including the palm image. The method for separating the palm image in the image from the image includes at least one of the following methods: a maximum inter-class variance method, a skin color detection method.

The largest interclass variance method was proposed by Japanese scholar Nobuyuki Otsu in 1979. It is an adaptive threshold determination method, also known as Otsu, or OTSU. It divides the image into two parts, the background and the target, according to the grayscale characteristics of the image. The larger the variance between the background and the target, the greater the difference between the two parts that make up the image. When part of the target is divided into the background or part of the background is divided into the target, the difference between the two parts will be smaller. Therefore, the segmentation that maximizes the variance between classes means that the probability of misclassification is minimal. In this way, the palm as a complete image can be distinguished from the background in the image.

The skin color detection method means that the human skin is distributed in a range in the color space, and the skin color reflects the intensity of the color. Under certain lighting conditions, the skin color is normalized to satisfy the Gaussian distribution, so this feature can be utilized. The image is segmented into skin regions and non-skin regions. The skin-targeted object can be separated from the background by the skin color detection method.

The process of extracting the palm image from the background object using skin color detection: first the color space of the image Converting from RGB to HSV, H stands for hue, S stands for saturation, and V stands for brightness. The effect of light intensity on the results can be reduced in the HSV color space. Set the range RH of the skin color hue corresponding value, the range RS of the saturation S corresponding value, the pixel in the picture ROI area that satisfies the hue in the RH range and the saturation in the RS range, and consider it to be the skin; then use the morphological expansion and corrosion The image processing method removes imaginary points and noise. The part of the skin color, the gray level is set to white, the non-skin color part, and the gray level is set to black, and the palm of the hand can be extracted.

For example, when a user browses a ring on a Taobao webpage through a mobile phone, he sees a favorite one. If he wants to try on it, he needs to perform image acquisition in real time through the camera of the mobile phone, and the classifier obtained by the machine learning method is trained on the image. Whether there is a palm to judge, if there is a palm in the image collected by the user, the step of separating the palm image is performed; if there is no palm in the image collected by the user, the image needs to be re-acquired until the image containing the palm is acquired. .

Step S102, determining a palm feature position of the palm in the image according to the palm image.

The purpose of this step is to determine the palm feature position of the palm in the palm image. Since the palm makeup is to be virtually worn on a certain position of the palm, it is necessary to determine the palm feature position of the palm in the image based on the palm image. Only when the palm feature position of the palm is determined can the position of the palm image match the palm ornament.

The feature position of the palm can adopt various possible positions according to different requirements; the specific degree of the feature position can also be determined according to requirements. For example, it may include: a palm position of the palm, a wrist position and a finger position in the palm. Of course, the feature position may also be specific to a position such as a first joint point, a second joint point, or a finger root in the finger. In short, as long as these positions can easily determine the specific location of the palm decoration. Hereinafter, a method of specifically determining the feature position of the palm will be described with the palm position, the wrist position, and the finger position as the palm feature positions. The palm position of the palm is determined by a method of using a geometric center of gravity or a method of using a corner point of a finger.

The method using the geometric center of gravity is specifically: the 0th moment and the 1st moment of the image geometry are used to calculate the center of gravity of the palm, and the center of gravity of the palm is used as the palm of the palm.

Here, a brief description of the way in which the center of gravity of the palm is calculated using the 0th moment and the 1st moment is used.

The 0th moment m00 is the sum of the image gradations. In the binary image, the 0th moment represents the area of the object; the 1st moment m10 is the first moment of the horizontal direction, and the first moment of the m01 vertical direction is normalized by the 0th moment. With the 1st moment, the center coordinates (xG, yG) can be obtained.

M00=∑∑f(i,j);m10=∑∑i*f(i,j);m01=∑∑j*f(i,j)

xG=m10/m00; yG=m01/m00.

In the above formula, (i, j) is the coordinate at which the pixel is located, and f(i, j) is the pixel value of the coordinate point (i, j).

The method of calculating the feature points of the finger corner feature points: the harris corner point extraction algorithm is used to extract the three index points between the index finger and the middle finger, the middle finger and the ring finger, and the ring finger and the little finger. A circle can be determined using three feature points, and the center of the circle serves as the palm of the palm.

The position of the finger includes: a position of five fingers in the palm; determining the position of the finger includes: determining a direction line of the finger and a finger line.

The direction line of the finger is determined as follows. After determining the palm of the palm, draw the concentric circle centered on the palm, from big to small, the concentric circle with only one point contact of the same finger is the largest circumscribed circle of the finger; draw a contact point of the same finger along the palm with the circumscribed circle Line, this line is the direction line of the finger.

The finger line is determined in the following manner. Focusing on the center of the palm from small to large, the center of the finger is less than 5 degrees, and the midpoint of the intersection of the concentric circle and the root of the finger is the midpoint of the root of the finger, connecting the middle point of the finger root and the fingertip. Line, you can get the finger line.

Each finger has a finger's direction line and finger line. The position of the five fingers can be determined according to the relative position of the finger direction line, and the middle finger of the finger direction line is the middle finger. If the palm image is the back of the left hand, the left and right fingers are the ring finger and the little finger, and the right direction is the index finger and the thumb. If the image of the palm is the palm of the left hand, the right and the right are the ring finger and the little finger, and the left and right are the index finger and the thumb. If the image of the palm is the back of the right hand, the right and the right are the ring finger and the little finger. For the index finger and thumb; if the image of the palm is the palm of the right hand, the left and right sides are the ring finger and the little finger, and the left side is the index finger and the thumb in turn; then the specific position of each finger is determined according to the finger line of each finger.

By determining the position of the five fingers in the palm, the user can specify which finger the wrist and other accessories are worn on.

After determining the position of the finger, a more specific position inside each finger can be further determined; including: the position of the fingertip of the finger, the root position of the finger, and the center point of the finger.

In addition to determining the position of each finger, a more precise feature position is required, for example, by determining the position of the fingertip of the finger, the position of the base of the finger, and the center point of the finger, the position of the palm makeup such as the ring can be more accurately determined; By determining the position of the fingertip of the finger, it is possible to determine the position at which the palm makeup and the like are superimposed. Only by wearing the virtual image of the palm makeup in the most suitable position can the wearing effect be better displayed, so it is necessary to determine a more precise feature position.

Determining a position of the fingertip of the finger includes: a skin color point whose finger is farthest from the palm position.

Determining the position of the root of the finger, including: drawing a concentric circle from small to large centered on the palm, the skin color angle is less than 5 The portion of the degree is the root of the finger. The meaning of the skin color angle is as follows: in the binary grayscale picture, the skin color portion is grayscale is 255, the non-skin color portion grayscale is 0; the transition from zero to 255 is the boundary of the non-skin color and the skin color, and the angle corresponding to the boundary is A. ; Jumping from 255 to 0 is the boundary of skin color and non-skinning, the angle corresponding to this boundary is B; along the concentric circle, jumping from non-skin to skin tone, then jumping from skin to non-skinning angle (B- A), that is, the angle B minus the angle A, which is the skin color angle. In the method of judging the root portion of the finger having a skin color angle of less than 5 degrees, the value of 5 degrees is an empirical value, and most cases can be accurately determined. Of course, it can be adjusted according to actual conditions.

Determining the center point of the finger is determined by drawing a concentric circle centered on the palm center, and circumscribing a central position of the concentric circle and the inscribed concentric circle in the direction of the finger with the intersection of the skin color.

The external concentric circle refers to a circle centered on the palm of the palm and having only one intersection with the fingertip of the finger.

The inner concentric circle refers to an innermost circle centered on the palm of the palm and having a radius smaller than the radius of the outer concentric circle and having the intersection with the finger.

The determining the position of the wrist in the palm comprises: drawing a concentric circle centered on the palm, when four skin color angles appear on the concentric circle, wherein the color of the palm is greater than 10 degrees and less than 40 degrees. The area in which the concentric circles are in contact with the portion of the palm is the wrist in the palm, the direction of the wrist being the direction from the center of the palm to the midpoint of the intersecting arc. The four skin color angles described in the above method include the concentric circles and the ring finger, the middle finger, the index finger, and the skin color corner formed by the wrist. By intersecting the concentric circle with the ring finger, the middle finger, and the index finger, and not intersecting the thumb and the little finger, it can be determined that the concentric circle has a suitable diameter to ensure that the concentric circle has a line of intersection with the wrist. The above range of more than 10 degrees and less than 40 degrees is data obtained empirically, and the specific value may be appropriately adjusted depending on the situation.

The center point of the wrist can be further determined as needed.

The determining the center point of the wrist adopts a method of drawing a concentric circle centering on the palm, and connecting a concentric circle and an inner concentric circle at a center position of the intersection of the skin color in the wrist direction.

Obtaining the characteristic position of the center point of the wrist determines the position of the palm ornament such as the bracelet that needs to be worn on the wrist.

Step S103, according to the palm feature position, superimposing a virtual image of a predetermined palm ornament on a position on the palm image that matches the palm ornament, to obtain the palm decoration virtual image.

The purpose of this step is to determine a position on the palm image that matches the palm ornament according to the palm feature position, and superimpose the virtual image of the predetermined palm ornament on the palm image and the palm decoration. At the position where the objects match, the palm decorative virtual image is obtained. By obtaining the virtual image of the palm decoration, the effect of augmented reality is realized, and the real experience of the user is enhanced.

The position on the palm image that matches the palm ornament includes: a center point of the finger of the palm or a center point of the wrist or a center point of the nail of the finger.

Of course, other locations suitable for wearing palm decorations can also be included.

The midpoint of the nail of the finger is a direction opposite to the direction of the finger line, and is located 5 mm from the point where the external concentric circle of the finger is in contact with the fingertip.

It should be noted that the difference in the position of the palm image matching the palm decoration is different depending on the type of the jewelry to be worn. If you wear a palm ornament such as a ring, it is superimposed on the center of the finger; if you wear a palm ornament such as a bracelet or a bracelet, it is superimposed on the center of the wrist; if you put a palm ornament such as a nail sticker, Just superimposed on the midpoint of the nail.

Specifically, the virtual image of the predetermined palm decoration is superimposed on the palm image at a position matching the palm decoration, or may be automatically implemented by performing a virtual image of the palm decoration. After being translated and rotated, it is superimposed on the position matching the palm ornament. After the superposition, the effect of augmented reality is realized, and the user can see the effect of the try-on.

The predetermined virtual image of the palm decoration may be a two-dimensional image or a three-dimensional image, wherein the predetermined palm decoration includes a decoration that can be worn on the finger, a decoration worn on the wrist, and a decoration attached to the nail Things.

The virtual image of the predetermined palm ornament may be obtained from a network or from a memory of the mobile terminal.

The virtual image of the predetermined palm decoration is obtained from the network, and refers to downloading the virtual image of the palm decoration from the database of the network server to the mobile terminal through the application software.

The virtual image may also be an image stored in a memory of the mobile terminal, which means that the user has a virtual image of the predetermined palm decoration to be tried on in the storage space of the mobile terminal.

And superimposing a virtual image of the predetermined palm decoration onto the palm image at a position matching the palm decoration according to the palm feature position, before the step of obtaining the palm decoration virtual image, further including : determining whether a size of the virtual image of the predetermined palm ornament matches a position on the palm image that matches the palm ornament; if so, entering a predetermined palm ornament according to the palm feature position The virtual image is superimposed on the palm image at a position matching the palm ornament, and the step of decorating the virtual image of the palm is obtained; if not, the virtual image of the palm ornament is regained.

The purpose of this step is to determine whether the virtual image can be superimposed on the palm image to match the palm ornament by determining whether the size of the virtual image matches the size of the finger or wrist to which the virtual image is to be superimposed. Set it up. The size of the finger or the wrist can be obtained by placing a standard-sized object on the palm of the hand as a reference object when the image including the palm is captured by the camera; or directly inputting the palm and the wearing-related part by parameter input. Size is obtained. The size of the palm ornament can then be obtained from a database storing palm ornament images.

Specifically, if the virtual image is a palm ornament such as a ring worn on a finger, comparing the diameter of the virtual image with the diameter of the finger at the position of the superimposed virtual object, if the diameter of the virtual image is smaller than the superposition a diameter of the finger at a position on the palm image matching the palm ornament, determining that the size of the virtual image of the predetermined palm ornament does not match the position on the palm image that matches the palm ornament; If the diameter of the virtual image is greater than or equal to the diameter of the finger at the position on the palm image that matches the palm ornament, determining the size of the virtual image of the predetermined palm ornament and the palm image Matching the position matching the palm ornament; if the virtual image is a palm ornament such as a bracelet worn on the wrist, comparing the diameter of the virtual image with the width of the wrist at the position of the superimposed virtual object, if The diameter of the virtual image is less than or equal to the position of the palm image on the palm image matching the palm ornament Width, determining that the size of the virtual image of the predetermined palm ornament does not match the position on the palm image that matches the palm ornament; if the diameter of the virtual image is greater than the overlay of the palm image The width of the wrist at the position where the palm ornament matches, it is determined that the size of the virtual image of the predetermined palm ornament matches the position on the palm image that matches the palm ornament.

Step S104, displaying the palm decoration virtual image through the display device.

The purpose of this step is to display the palm decorative virtual image through the display device, and the user can visually see the wearing effect.

For example, when the user wears a virtual image of a ring on the ring finger, the user can visually observe the try-on effect of the ring by displaying the virtual image wearing the ring on the display screen through the display device.

And superimposing a virtual image of the predetermined palm decoration onto the palm image at a position matching the palm decoration according to the palm feature position, and after obtaining the palm decoration virtual image, further comprising : Store the palm decorative virtual image.

The purpose of this step is to store the virtual image of the palm decoration for the user to view the wearing effect of the virtual image of different palm decorations. The user can compare a plurality of the palm decorative virtual images to select the best palm decoration.

The storing the palm decorative virtual image means that the user stores the palm image virtual image that is satisfactory to the storage space of the mobile terminal.

For example, when a user browses a ring on a Taobao webpage through a mobile phone, he or she looks at five rings. I don't know which one is the most suitable. If you want to decide which one to buy by watching the trial effect, you can use the software program to get from the network server. The fancy ring is downloaded to the local, then the user selects the finger to try on, puts the ring on the selected finger in turn, and stores each virtual image wearing the ring in the storage space of the mobile phone, and then five The palm-decorated virtual image is simultaneously displayed on the display screen of the mobile phone, and the user can conveniently view the try-on effect, and the user can select the most satisfactory ring by watching the try-on effect.

Corresponding to the above implementation method for providing a palm decoration virtual image by using augmented reality technology, the present application also provides an implementation device for providing a palm decoration virtual image by using augmented reality technology. Since the embodiment of the device is substantially similar to the embodiment of the method, the description is relatively simple, and the relevant portions can be referred to the description of the method embodiment. The device embodiments described below are merely illustrative. The implementation device for providing a palm-decorated virtual image using augmented reality technology is as follows:

Please refer to FIG. 2 , which is a schematic diagram of an apparatus for implementing a palm-decorated virtual image using augmented reality technology according to a second embodiment of the present application.

The device for implementing a palm-decorated virtual image by using an augmented reality technology includes: a obtaining unit 201, a determining unit 202, a superimposing unit 203, and a display unit 204.

An obtaining unit 201, configured to acquire a palm image;

a determining unit 202, configured to determine, according to the palm image acquired by the acquiring unit, a palm feature position of the palm in the image;

a superimposing unit 203, configured to superimpose a virtual image of the predetermined palm decoration on a position of the palm image matching the palm decoration according to the palm feature position determined by the determining unit, to obtain the palm decoration Virtual image

The display unit 204 is configured to display the palm decoration virtual image obtained by the superimposing unit through the display device.

Optionally, the device further includes:

Obtaining an image unit for acquiring an image before the obtaining unit works;

a separating unit, configured to determine whether there is a palm in the image in the acquired image unit; if yes, separate the palm image in the image from the image; if not, return to the step of acquiring the image.

Optionally, the acquiring image unit includes:

The acquisition subunit is configured to acquire an image by using a camera of the mobile terminal.

Optionally, determining whether there is a palm in the image in the separating unit is implemented by using the following method:

Optionally, the method for separating the palm image in the image from the image in the separating unit includes at least one of the following methods:

Maximum inter-class variance method, skin color detection method.

Optionally, the determining a feature position of the palm in the unit includes:

Optionally, determining a palm position of the palm includes:

Optionally, the position of the finger includes:

The position of the five fingers in the palm.

Optionally, determining the location of the finger includes:

Determine the direction line and finger line of your finger.

Optionally, the position of the finger includes:

Optionally, determining the position of the fingertip of the finger includes:

Optionally, determining the position of the root of the finger includes:

Optionally, determining the position of the wrist in the palm, including:

Optionally, the position of the wrist in the palm includes:

The center point of the wrist.

Optionally, determining the center point of the wrist is as follows:

Optionally, a position on the palm image in the superimposing unit that matches the palm decoration includes:

Optionally, the virtual image of the predetermined palm decoration in the superimposing unit includes:

Obtained from the network.

Optionally, the device further includes:

Determining whether the subunit is matched, for determining whether the size of the virtual image of the predetermined palm decoration matches the position of the palm image matching the palm decoration before the superimposing unit works; if yes, entering the basis Positioning the palm feature, superimposing a virtual image of the predetermined palm ornament on the palm image at a position matching the palm ornament, obtaining the step of decorating the virtual image of the palm; if not, regaining the palm A virtual image of the ornament.

Optionally, the device further includes:

And a storage unit, configured to store the palm decoration virtual image after the superimposing unit works.

The present invention is disclosed in the above preferred embodiments, but it is not intended to limit the present invention, and any one skilled in the art can make possible variations and modifications without departing from the spirit and scope of the invention. The scope of protection should be determined by the scope defined by the claims of the present invention.

Claims

An implementation method for providing a palm-customized virtual image by using augmented reality technology, comprising:

Get the palm image;

Determining a palm feature position of the palm in the image according to the palm image;

And superimposing a virtual image of the predetermined palm decoration on the palm image at a position matching the palm decoration according to the palm feature position to obtain the palm decoration virtual image;

The palm decorative virtual image is displayed by a display device.
The method for providing a palm-decorated virtual image by using an augmented reality technology according to claim 1, wherein before the step of acquiring a palm image, the method comprises:

Get an image;

Determining whether there is a palm in the image; if so, separating the palm image in the image from the image; if not, returning to the step of acquiring the image.
The method of claim 2, wherein the acquiring an image comprises:

The acquisition is acquired by the camera of the mobile terminal.
The method for implementing a palm-decorated virtual image by using an augmented reality technology according to claim 2, wherein the determining whether there is a palm in the image is implemented by the following method:

The classifier obtained by training using the machine learning method judges whether or not the image has a palm.
The method for implementing a palm-decorated virtual image using an augmented reality technology according to claim 4, wherein the machine learning method comprises at least one of the following methods:

Hog-svm learning, Haaris corner learning, Haar LBP cascade classifier learning, and neural network learning.
The method for providing a palm-decorated virtual image by using an augmented reality technology according to claim 2, wherein the method for separating a palm image in the image from the image comprises at least one of the following methods:

Maximum inter-class variance method, skin color detection method.
The method for providing a palm-customized virtual image by using an augmented reality technology according to claim 1, wherein the feature position of the palm comprises:

The palm position of the palm, the wrist position and the finger position in the palm.
The method for providing a palm-decorated virtual image by using an augmented reality technology according to claim 7, wherein determining the palm position of the palm comprises:

A method using a geometric center of gravity or a method of using a corner point of a finger.
The method for providing a palm-decorated virtual image by using an augmented reality technology according to claim 7, wherein the position of the finger comprises:

The position of the five fingers in the palm.
The method for providing a palm-decorated virtual image by using an augmented reality technology according to claim 9, wherein determining the position of the finger comprises:

Determine the direction line and finger line of your finger.
The method for providing a palm-decorated virtual image by using an augmented reality technology according to claim 7, wherein the position of the finger comprises:

The position of the fingertip of the finger, the position of the root of the finger, and the center point of the finger.
The method for providing a palm-decorated virtual image by using an augmented reality technology according to claim 11, wherein determining the position of the fingertip of the finger comprises:

The point of skin color that is the farthest from the position of the palm of the palm.
The method for providing a palm-decorated virtual image by using an augmented reality technology according to claim 11, wherein determining the position of the finger root comprises:

Focusing on the center of the palm from small to large, the part of the skin color is less than five degrees is the root of the finger.
The method for implementing a palm-decorated virtual image by using an augmented reality technology according to claim 11, wherein determining a center point of the finger is determined as follows:

A concentric circle is drawn centering on the palm, and a concentric circle and an inscribed concentric circle are centrally located at the intersection of the skin color in the direction of the finger.
The method for providing a palm-decorated virtual image by using an augmented reality technology according to claim 7, wherein determining the position of the wrist in the palm comprises:

Drawing a concentric circle centered on the palm of the hand, when the angle of the concentric circle with the skin color of the palm is greater than 10 degrees and less than 40 degrees, the area where the concentric circle contacts the portion of the palm is the wrist in the palm, The direction of the wrist is the direction from the center of the palm to the midpoint of the intersecting arc.
The method for providing a palm-decorated virtual image by using an augmented reality technology according to claim 7, wherein the position of the wrist in the palm comprises:

The center point of the wrist.
The method for implementing a palm-decorated virtual image by using an augmented reality technology according to claim 16, wherein the center point of the wrist is determined by adopting the following method:

Draw a concentric circle centered on the palm, and connect the concentric circles and the inner concentric circles in the direction of the wrist to intersect with the skin color. Central location.
The method for providing a palm-customized virtual image by using an augmented reality technology according to claim 1, wherein the position of the palm image matching the palm decoration comprises:

The center point of the finger of the palm or the center point of the wrist or the center point of the nail of the finger.
The method for providing a palm-decorated virtual image by using an augmented reality technology according to claim 1, wherein the predetermined virtual image of the palm decoration comprises:

Obtained from the network.
The method for providing a palm-decorated virtual image by using augmented reality technology according to claim 1, wherein the virtual image of the predetermined palm decoration is superimposed on the palm image according to the palm feature position Before the step of obtaining the virtual image of the palm decoration in the position matching the palm ornament, the method further includes:

Determining whether a size of the virtual image of the predetermined palm ornament matches a position on the palm image that matches the palm ornament; if so, entering a virtual palm predetermined object according to the palm feature position The image is superimposed on the palm image at a position matching the palm ornament, and the step of decorating the virtual image of the palm is obtained; if not, the virtual image of the palm ornament is regained.
The method for providing a palm-decorated virtual image by using augmented reality technology according to claim 1, wherein the virtual image of the predetermined palm decoration is superimposed on the palm image according to the palm feature position After the step of obtaining the virtual image of the palm decoration in the position matching the palm decoration, the method further includes:

The palm decorative virtual image is stored.
An implementation device for providing a virtual image of a palm decoration by using an augmented reality technology, comprising:

An acquisition unit for acquiring a palm image;

a determining unit, configured to determine, according to the palm image acquired by the acquiring unit, a palm feature position of the palm in the image;

a superimposing unit, configured to superimpose a virtual image of the predetermined palm decoration on a position of the palm image matching the palm decoration according to the palm feature position determined by the determining unit, to obtain the palm decoration virtual image;

And a display unit, configured to display, by the display device, the palm decoration virtual image obtained by the superimposing unit.