CN117725251A

CN117725251A - Image processing method, device, electronic equipment and computer readable storage medium

Info

Publication number: CN117725251A
Application number: CN202311213842.0A
Authority: CN
Inventors: 杨静; 张道鑫; 宋德嘉
Original assignee: Xingyin Information Technology Shanghai Co ltd
Current assignee: Xingyin Information Technology Shanghai Co ltd
Priority date: 2023-09-19
Filing date: 2023-09-19
Publication date: 2024-03-19

Abstract

The embodiment of the application discloses an image processing method, an image processing device, electronic equipment and a computer readable storage medium, wherein the method comprises the following steps: receiving an input reference nail image; acquiring single nail feature information of each nail in the reference nail image; determining characteristic nail feature information according to the single nail feature information, wherein the characteristic nail feature information describes the integral nail feature of the reference nail image; searching a target nail image matched with the characteristic information of the characteristic nail from a preset search library; and displaying the target nail art image on a visual interface. By searching and matching based on the characteristic nail feature information, the searching process can consider all nails in the image, only consider nail features, but filter the features of non-nails in the image, so that the accuracy of image searching is improved, and compared with the one-by-one searching of single nails, the searching and matching efficiency of nail image searching and matching can be improved, and the user experience is improved.

Description

Image processing method, device, electronic equipment and computer readable storage medium

Technical Field

The embodiment of the application relates to the technical field of computers, in particular to an image processing method, an image processing device, electronic equipment and a computer readable storage medium.

Background

With popularization of sharing communities, more and more users participate in community sharing, image sharing is a main mode of current community sharing, and users can acquire information by looking at images shared by other users. However, as the number of sharing users and times increases, the number of images in the community increases, and the difficulty for users to acquire images increases.

Although the function of searching the images by the images appears at present, aiming at the image content of a specific type, the accuracy of the searched image is difficult to ensure, for example, the searching for the nail is difficult, because the effective area of the nail in the image is smaller, a great amount of noise exists in the searching process, the searched image is inaccurate, and the problems of lower efficiency and poor user experience exist when the searching for the single nail are carried out.

Disclosure of Invention

The embodiment of the application provides an image processing method, an image processing device, electronic equipment and a computer readable storage medium, which can improve the accuracy and efficiency of nail art image searching and improve the user experience.

The embodiment of the application provides an image processing method, which comprises the following steps:

receiving an input reference nail image;

acquiring single nail feature information of each nail in the reference nail image;

determining characteristic nail feature information according to the Shan Meijia feature information, wherein the characteristic nail feature information describes the nail feature of the whole reference nail image;

searching a target nail-beautifying image matched with the characteristic information of the characteristic nail-beautifying from a preset search library;

and displaying the target nail art image on a visual interface.

Correspondingly, the embodiment of the application also provides an image processing device, which comprises:

the receiving module is used for receiving the input reference nail art image;

the acquisition module is used for acquiring single nail feature information of each nail in the reference nail image;

the determining module is used for determining characteristic nail feature information according to the Shan Meijia characteristic information, wherein the characteristic nail feature information describes the nail feature of the whole reference nail image;

the searching module is used for searching a target nail-beautifying image matched with the characteristic feature information of the characteristic nail-beautifying from a preset search library;

and the display module is used for displaying the target nail art image on a visual interface.

Optionally, in some embodiments of the present application, the determining module includes:

the calculating unit is used for calculating the average value of each Shan Meijia characteristic information to obtain average value nail art characteristic information;

the first determining unit is used for taking the average nail feature information as characteristic nail feature information.

Optionally, in some embodiments of the present application, the apparatus further includes a conversion module, where the conversion module includes:

the acquisition unit is used for acquiring coordinate information of each nail in the reference nail image;

a second determining unit, configured to determine an envelope frame according to coordinate information of each nail, where each nail is in the envelope frame;

a third determining unit, configured to use the characteristic feature information of the feature nail as frame feature information of the envelope frame;

the search module comprises:

and the searching unit is used for searching the target nail art image matched with the frame characteristic information from a preset search library.

Wherein, in some embodiments of the present application, the apparatus further comprises a pre-configuration unit, the pre-configuration unit comprising:

the acquisition subunit is used for acquiring the nail art image to be matched;

the first determining subunit is used for determining an envelope frame to be matched of the nail art image to be matched and feature information of the frame to be matched corresponding to the envelope frame to be matched;

The construction subunit is used for constructing a preset search library according to the to-be-matched nail art image and the to-be-matched frame characteristic information corresponding to the to-be-matched nail art image;

the search unit includes:

a searching subunit, configured to search target frame feature information matched with the frame feature information from frame feature information to be matched in the preset search library;

and the second determining subunit is used for taking the to-be-matched nail art image corresponding to the target frame characteristic information as a target nail art image.

Wherein, in some embodiments of the present application, the search subunit is specifically configured to:

determining vector similarity between frame characteristic information to be matched in the preset search library and the frame characteristic information;

and taking the frame characteristic information to be matched, of which the vector similarity is larger than a first preset threshold value and smaller than a second preset threshold value, as target frame characteristic information, wherein the second preset threshold value is larger than the first preset threshold value.

Wherein, in some embodiments of the present application, the apparatus further comprises an interface display unit, the interface display unit comprises:

a first display subunit, configured to display the envelope frame for the reference nail image in a visual interface that displays the reference nail image;

The generation subunit is used for generating retrieval prompt information aiming at the envelope frame, wherein the retrieval prompt information is used for indicating to search the nail art image according to the region of the envelope frame;

and the second display subunit is used for displaying the retrieval prompt information in the visual interface, and the display position of the retrieval prompt information is opposite to the position of the envelope frame.

Wherein, in some embodiments of the present application, the acquisition module includes:

a fourth determining unit for determining nail-beautifying areas of the respective nails in the reference nail-beautifying image through a single nail detection model;

the extracting unit is used for extracting the characteristic information of the nail region corresponding to each nail through the nail feature model to obtain single nail feature information.

In a third aspect, an embodiment of the present application further provides an electronic device, where the electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program when executed by the processor implements the steps in the image processing method described above.

In a fourth aspect, embodiments of the present application further provide a computer readable storage medium, where a computer program is stored, where the computer program when executed by a processor implements the steps in the image processing method described above.

In a fifth aspect, embodiments of the present application also provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The computer instructions are read from the computer-readable storage medium by a processor of a computer device, and executed by the processor, cause the computer device to perform the methods provided in the various alternative implementations described in the embodiments of the present application.

According to the embodiment of the application, an input reference nail-beautifying image is received, single nail-beautifying feature information of each nail in the reference nail-beautifying image is obtained, characteristic nail-beautifying feature information is determined according to the Shan Meijia feature information, the characteristic nail-beautifying feature information describes the integral nail-beautifying feature of the reference nail-beautifying image, a target nail-beautifying image matched with the characteristic nail-beautifying feature information is searched from a preset search library, and the target nail-beautifying image is output and displayed on a visual interface. The method comprises the steps of determining characteristic nail feature information for describing integral nail feature of a reference nail image by referring to single nail feature information of each nail in the nail image, and searching and matching based on the characteristic nail feature information, so that the searching process can consider all nails in the image, only consider the nail feature, filter out non-nail feature in the image, improve the accuracy of image searching, search and matching are carried out by single characteristic nail feature information, and compared with one-by-one searching of single nail, the searching and matching efficiency of the nail image can be improved, and the user experience is improved.

Drawings

In order to more clearly illustrate the technical solutions of the present application, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic view of a scenario of an image processing method provided in an embodiment of the present application;

fig. 2 is a schematic flow chart of an image processing method according to an embodiment of the present application;

FIG. 3 is a first exemplary diagram of a search result in the prior art provided by an embodiment of the present application;

FIG. 4 is a second exemplary diagram of a search result in the prior art provided by an embodiment of the present application;

fig. 5 is a flowchart of a nail art image searching method provided in an embodiment of the present application;

fig. 6 is a search generation guide chart of a target nail image provided in the embodiment of the present application;

fig. 7 is a schematic structural diagram of an image processing apparatus provided in an embodiment of the present application;

fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to fall within the scope of the invention.

The embodiment of the application provides an image processing method, an image processing device, electronic equipment and a computer readable storage medium. Specifically, the embodiment of the application provides an image processing apparatus suitable for an electronic device, where the electronic device includes a terminal device and a server, the terminal device includes, but is not limited to, a device such as a mobile phone, a tablet computer, a computer, and the like, the server may be an independent physical server, may be a server cluster or a distributed system formed by a plurality of physical servers, may also be a cloud server that provides cloud services, a cloud database, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution networks (CDN, content Delivery Network), and basic cloud computing services such as big data and an artificial intelligent platform, and the server may be directly or indirectly connected through a wired or wireless communication manner.

In this embodiment of the present application, the image processing method may be executed by a terminal device alone, or the image processing method may be executed by a server alone, or the image processing method may be executed by a terminal device and a server together, referring to fig. 1, fig. 1 is a schematic view of a scenario provided in this embodiment of the present application in which the image processing method is executed by the terminal device and the server together, where the scenario includes a specific execution procedure of executing the image processing method by the terminal device and the server together, where the specific execution procedure of executing the image processing method by the terminal device and the server together is as follows:

The terminal device 10 obtains a reference nail image in response to the input, wherein the reference nail image is a reference image which is expected to be searched for a more similar nail image after the input, and then the terminal device 10 sends the reference nail image to the server 11, and the server 11 processes the reference nail image and retrieves a corresponding target nail image, specifically:

the server 11 receives the reference nail image, acquires single nail feature information of each nail in the reference nail image, determines characteristic nail feature information according to the Shan Meijia feature information, searches a target nail image matched with the characteristic nail feature information from a preset search library, and returns the target nail image to the terminal equipment 10;

the terminal device 10 receives the target nail image returned by the server 11, and outputs and displays the target nail image on the visual interface of the terminal device 10.

It can be understood that the characteristic nail feature information describes the nail feature of the whole reference nail image as a retrieval matching basis of the reference nail image.

It will be appreciated that the target nail image may comprise a plurality of images, and thus the plurality of images may be output and presented in the visual interface of the terminal device 10, respectively.

The image processing method provided by the embodiment of the application relates to machine learning in the field of artificial intelligence; according to the embodiment of the application, the accuracy and the efficiency of nail art image searching can be improved, and the user experience is improved.

Among these, artificial intelligence (AI, artificial Intelligence) is the theory, method, technique and application system that uses a digital computer or a digital computer-controlled machine to simulate, extend and expand human intelligence, sense the environment, acquire knowledge and use knowledge to obtain optimal results. In other words, artificial intelligence is an integrated technology of computer science that attempts to understand the essence of intelligence and to produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence, i.e. research on design principles and implementation methods of various intelligent machines, enables the machines to have functions of sensing, reasoning and decision. The artificial intelligence technology is a comprehensive subject, and relates to the technology with wide fields, namely the technology with a hardware level and the technology with a software level. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.

Among them, machine Learning (ML) is a multi-domain interdisciplinary, and involves multiple disciplines such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, and the like. It is specially studied how a computer simulates or implements learning behavior of a human to acquire new knowledge or skills, and reorganizes existing knowledge structures to continuously improve own performance. Machine learning is the core of artificial intelligence, a fundamental approach to letting computers have intelligence, which is applied throughout various areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, confidence networks, reinforcement learning, transfer learning, induction learning, teaching learning, and the like.

For example, in the embodiment of the application, the first-feature extraction is performed on the reference first-feature image through the trained machine learning model to obtain single first-feature information of each first-feature in the image, then the first-feature information representing the first-feature of the whole reference first-feature image is obtained through calculation, and the target first-feature image matched with the corresponding target first-feature image is retrieved from the preset retrieval library based on the first-feature information representing the first-feature of the whole reference first-feature image.

Corresponding characteristic nail feature information is correspondingly stored in nail art images in a preset search library, and the characteristic nail art feature information can be obtained through extraction of the machine learning model and calculation processing.

It can be understood that, the characteristic nail feature information for describing the integral nail feature of the reference nail image is determined by referring to the single nail feature information of each nail in the nail image, and searching and matching are carried out based on the characteristic nail feature information, so that the searching process can consider all nails in the image, only consider the nail feature, but filter the non-nail feature in the image, the accuracy of image searching is improved, and searching and matching are carried out by using the single characteristic nail feature information, compared with searching one by one of single nail, the searching and matching efficiency of the nail image can be improved, and the user experience is improved.

The following will describe in detail. It should be noted that the following description order of embodiments is not a limitation of the priority order of embodiments.

Referring to fig. 2, fig. 2 is a flowchart of an image processing method according to an embodiment of the present application. The specific flow of the image processing method is as follows:

101. an input reference nail image is received.

It can be understood that the reference nail image is an input image which needs to be approximately searched, the image mainly takes a finger as a main body, and nails are usually coated on the finger in the image, wherein the nails in the nail image can be directly coated on hands and obtained by shooting, or obtained by shooting an image without nails and by image stitching (such as P picture).

Correspondingly, the reference nail art image can be uploaded from a local album by a user, or can be shot in real time, or can be obtained by searching from a related website or community by the user. The communities refer to sharing communities, such as communities corresponding to video or image sharing applications.

Accordingly, the reference nail image may be received through a corresponding configuration interface, e.g., in response to an image lookup operation for use, an image input interface is provided in the terminal interface through which the user inputs a corresponding image (e.g., the user clicks an upload button, or "plus" control, in the interface) and the system receives the corresponding image through the interface.

102. And acquiring single nail feature information of each nail in the reference nail image.

It can be understood that the feature information of the single nail is feature information of the single nail, the feature information is in the form of a vector or code, and the feature information can be extracted by a feature extraction model.

It can be understood that by extracting feature information of the nail art, matching search of the nail art by using the feature information can be realized.

The feature information of the nail is extracted from the image, so that the extracted feature information is only aimed at the nail, and does not contain finger features, hand joint features, background image features and the like, and therefore the accuracy of image matching and searching can be ensured.

103. And determining characteristic nail feature information according to the Shan Meijia characteristic information, wherein the characteristic nail feature information describes the whole nail feature of the reference nail image.

Because there are usually more than one finger included in the image, performing nail-beautifying matching on each finger respectively results in lower retrieval efficiency and poorer user experience, and if feature extraction models are built for each finger respectively, cost increases.

Therefore, in the embodiment of the application, the characteristic nail feature information for describing the integral nail feature of the reference nail image is determined through the single nail feature information of each nail, so that the characteristic nail feature information is utilized to reflect or characterize the nail feature of the integral reference nail image.

It can be understood that, compared with the matching retrieval of each finger corresponding to one single nail feature information, the method and the device for acquiring the characteristic nail feature information are beneficial to realizing rapid retrieval matching of nail images and improving retrieval efficiency.

104. Searching a target nail-beautifying image matched with the characteristic information of the characteristic nail-beautifying from a preset search library.

It can be understood that the characteristic feature information of the representative nail art is single characteristic information of the integral nail art characteristic of the reference nail art image, and compared with the fact that the nail art of each finger is respectively searched and matched in the related art, the searching efficiency of the target nail art image can be improved.

It can be understood that in the embodiment of the present application, the preset search library is a search library that is pre-established and is used for storing the nail images to be matched, and the scheme also searches the corresponding target nail images from the search library.

For example, in the embodiment of the present application, the feature information of the feature of the nail to be matched may be stored in the preset search library, and then, the corresponding target nail image is determined by comparing and matching the feature information of the nail to be matched.

105. And displaying the target nail art image on a visual interface.

It can be understood that the feedback of the search matching of the user is completed by displaying the searched and matched target nail art image in the visual interface, and meanwhile, the user can also know the searched image content through the visual interface, so that the exploration experience is improved.

Optionally, in embodiments of the present application, the characteristic feature information of the characteristic nail may be obtained by means of a mean value of feature information, for example, by calculating a mean value of single nail feature information of each nail, so as to obtain characteristic feature information of the characteristic nail, that is, optionally, in some embodiments of the present application, the step of determining characteristic feature information of the characteristic nail according to the Shan Meijia feature information includes:

Calculating the average value of the Shan Meijia characteristic information to obtain average value nail art characteristic information;

and taking the average nail art characteristic information as characteristic nail art characteristic information.

It can be understood that the average value of the feature information of each nail can integrally reflect the nail feature condition of the whole image, so that the average value nail feature information obtained by the average value of each single nail feature information is used as the characteristic nail feature information, and the method has higher accuracy and representativeness.

Correspondingly, in the embodiment of the application, the characteristic difference of each nail and the nail without nail can be calculated, a part of nail is selected from a plurality of nails as a representative based on the magnitude of the characteristic difference, and the characteristic nail characteristic information of the reference nail image is obtained through the average value of the single nail characteristic information of the part of nail.

Accordingly, in the embodiment of the present application, the characteristic feature information of the representative nail may also be set to two according to the actual application situation, for example, different nail sets are classified into two types according to the difference situation of the feature information, where the feature information of multiple nail sets in each type is relatively close, and there is a relatively large difference in the feature information of multiple nail sets in different types, so that the extreme situation that the characteristic feature information of the representative nail set cannot effectively represent the nail set feature of the reference nail set image after the mean value processing due to the relatively large difference between the two types of feature information is avoided, and therefore, search matching is performed based on the characteristic feature information of the representative nail sets in two types, which is also helpful to promote accuracy of search matching.

Accordingly, in the related art, when extracting or searching and matching the nail features, two modes are generally adopted, firstly, a corresponding labeling frame is configured based on each nail, then, feature extraction is carried out based on the range of the labeling frame, and targeted searching and matching are carried out; secondly, a labeling frame is established by the region corresponding to the whole image, and then feature extraction and retrieval matching are carried out based on the labeling range of the whole region.

However, for the first case, as shown in fig. 3, fig. 3 is a first example diagram of a search effect in the prior art provided by the embodiment of the present application, where a labeling frame shown in the diagram is smaller, an influence of an influence factor is larger, stability and accuracy of feature information of a single nail are difficult to ensure, resulting in a non-intuitive feel of a recall effect, and in addition, when search matching is performed based on a single nail, a plurality of labeling frames are easy to appear in an interface, and the error detection interference is larger, resulting in poor user experience.

In view of the second situation, as shown in fig. 4, fig. 4 is a second exemplary diagram of a search effect in the prior art provided in this embodiment, in the search effect, a labeling frame for an entire image includes more invalid information except for nail art, so that it is difficult to ensure accuracy and validity of the extracted feature information, for example, the effect of the hand display gesture is greatly affected, the background interference is great, and the searched nail art is the same gesture, and the similarity in nail art design and structural features is low.

Thus, in the embodiment of the present application, an envelope frame with a moderate area (based on the whole image and between the single nail) may be redetermined based on the position coordinates of each nail in the image, and extraction of the nail feature and search matching of the nail image are performed based on the envelope frame, that is, optionally, in some embodiments of the present application, the step of "searching for a target nail image matched with the characteristic nail feature information from a preset search library" further includes:

acquiring coordinate information of each nail in the reference nail image;

determining an envelope frame according to the coordinate information of each nail, wherein each nail is in the envelope frame;

and taking the characteristic feature information of the characteristic nail as frame feature information of the envelope frame.

It will be appreciated that the envelope is an area determined based on the edge position formed by each nail, for example, based on the range delineated by the minimum abscissa, the maximum abscissa, the minimum ordinate, and the maximum ordinate in each nail coordinate information. The object is to obtain a minimum area containing all nail art, and to ensure that less invalid information is contained, for example, the area of influence of the background is reduced.

Correspondingly, the step of searching the target nail art image matched with the characteristic information of the characteristic nail art from the preset search library comprises the following steps:

searching a target nail art image matched with the frame characteristic information from a preset search library.

It can be understood that by taking the mean value of each single nail feature information as the characteristic nail feature information and taking the characteristic nail feature information as the frame feature information of the envelope frame, the purposes of representing the reference nail image by the envelope frame and carrying out retrieval matching by the characteristic nail feature information capable of representing the integral nail feature of the reference nail image are achieved, and the accuracy and efficiency of retrieving and matching target nail images are improved.

Accordingly, in the embodiment of the present application, the envelope frame may be further displayed in the visual interface, so that the user knows the image content according to which the matching is retrieved, that is, optionally, in some embodiments of the present application, after the step of determining the envelope frame according to the coordinate information of each nail, the method further includes:

displaying the envelope frame for the reference nail image in a visual interface displaying the reference nail image;

generating retrieval prompt information aiming at the envelope frame, wherein the retrieval prompt information is used for indicating to search nail art images according to the region of the envelope frame;

And displaying the retrieval prompt information in the visual interface, wherein the display position of the retrieval prompt information is opposite to the position of the envelope frame.

The envelope frame is displayed in the visual interface, so that search guidance is provided for the user, and the user is convenient to know the main content on which the search matching is based.

Further, for the envelope frame, the user can also manually adjust the size and the position of the envelope frame in the reference nail image, for example, when the user needs to exclude the nail of the individual finger, the envelope frame can be reduced by zooming the envelope frame, and the envelope frame can be dragged by dragging. Correspondingly, in the adjusted envelope frame, average value calculation of single nail feature information of each nail in the envelope frame can be carried out again, and characteristic nail feature information is updated, so that the approach degree of a search matching result to a user expected, namely the approach degree of the search matching result to a nail image in the envelope frame is improved.

Optionally, in this embodiment, before searching the target nail image corresponding to the matching from the preset search library, the preset search library is further constructed, where the preset search library may be constructed by collecting the nail image to be matched and then extracting the envelope frame of the nail image and the feature information of the frame to be matched, that is, optionally, in some embodiments of this application, before the step of searching the preset search library for the target nail image matching the frame feature information, the method further includes:

Acquiring a nail art image to be matched;

determining a to-be-matched envelope frame of the to-be-matched nail-painting image and feature information of the to-be-matched frame corresponding to the to-be-matched envelope frame;

and constructing a preset search library according to the to-be-matched nail art image and the feature information of the to-be-matched frame corresponding to the to-be-matched nail art image.

It can be understood that the nail art image to be matched can be obtained through sharing of users in the community, and after corresponding envelope frames to be matched and characteristic information of the frames to be matched are determined according to the images shared by the users, a preset search library is built based on the nail art image to be matched, the envelope frames to be matched and the characteristic information of the frames to be matched.

Correspondingly, the address link or the note identification corresponding to the nail art image to be matched can be stored, so that the corresponding nail art image can be conveniently found based on the address link or the note identification, and the visual display of the terminal can be facilitated.

Further, in the preset search library, the user identifier for sharing the to-be-matched nail art image, such as the data of the user name, the user head portrait and the like, can be stored, when being matched into the corresponding target nail art image, the user identifier can be displayed in an interface, and correspondingly, the user can enter the homepage of the user through the displayed user identifier, so that the user can view other more shares of the user, such as other shares of the same type and the like.

Accordingly, the matching of the target nail image may be determined by comparing the frame feature information of the reference nail image and the frame feature information of the nail image to be matched, that is, optionally, in some embodiments of the present application, the step of searching the target nail image matched with the frame feature information from the preset search library includes:

searching target frame characteristic information matched with the frame characteristic information from the frame characteristic information to be matched in the preset search library;

and taking the to-be-matched nail art image corresponding to the target frame characteristic information as a target nail art image.

It can be understood that, through matching the frame feature information to be matched and the frame feature information, the target frame feature information meeting the condition can be retrieved, and then the to-be-matched nail art image corresponding to the target frame feature information is used as the target nail art image.

Optionally, in this particular class of nail art, since nail art is a decorative design, and there is a specific and individual psychological need for some users, it is not desirable to have other nail art that is completely consistent with the design of their own nail art, so in this embodiment, the filtering may be further performed on the very close target nail art image according to the matching proximity degree, so as to meet the psychological need of the user, that is, optionally, in some embodiments of this application, the step of "searching the frame feature information to be matched with the frame feature information from the preset search library" includes:

It can be understood that the first preset threshold reflects the lower limit of the similarity, and the second preset threshold reflects the upper limit of the similarity, wherein the similarity is higher than the second preset threshold, so that the image is very close to the reference nail image.

The method comprises the steps of filtering images with similarity smaller than a first preset threshold value, avoiding partial images with larger similarity difference from being input, and ensuring that the output images and the input reference nail art images have certain similarity.

Accordingly, after the target nail images are retrieved and matched, the target nail images may be sequentially output and displayed according to the similarity, that is, optionally, in some embodiments of the present application, the step of displaying the target nail images on the visual interface includes:

And according to the vector similarity, sequencing and displaying the target nail art images corresponding to the target frame characteristic information on a visual interface.

The ordering display is performed according to the similarity, so that the user can be helped to progressively view the nail-beautifying images, the nail-beautifying images with higher similarity are displayed at the front, the browsing habit of the user is met, and the user can be helped to quickly acquire the more similar nail-beautifying images in time.

Optionally, in this embodiment of the present application, each nail may be detected by using a nail detection model, and then feature extraction is performed on each nail by using a feature extraction model to obtain feature information of a single nail, that is, optionally, in some embodiments of the present application, the step of "obtaining feature information of a single nail of each nail in the reference nail image" includes:

determining the nail beautifying areas of all nail beautifying areas in the reference nail beautifying image through a single nail detection model;

and extracting the characteristic information of the nail region corresponding to each nail through the cosmetic characteristic model aiming at each nail to obtain single nail characteristic information.

The single nail detection model is used for detecting single nails and displaying the area of each nail in the interface, and the corresponding area corresponds to the area of the nail on the nail.

The single nail detection model and the make-up feature model can be machine learning models obtained through sample data training, the training process is not limited, the single nail detection model nail contour detection device can detect nail contours, or the single nail detection model nail contour detection device can directly detect nail, and nail beautifying areas on all fingers can be identified.

It can be appreciated that the feature model of the nail is used for feature extraction of the nail to obtain single nail feature information.

The average value of the single nail feature information of each nail is used as the characteristic feature of the reference nail image, so that quick and accurate retrieval matching of the target nail image is realized.

Referring to fig. 5, fig. 5 is a schematic flow chart of a nail art image searching method according to an embodiment of the present application, where the flow chart of the nail art image searching method specifically includes:

201. receiving a reference nail image input by a user through a nail image retrieval interface;

the reference nail image may be an image uploaded by a user or an image downloaded by the user through a corresponding address link (URL), and accordingly, in this embodiment of the present application, the client may trigger the same type search by pressing the picture for a long time, for example, after the client presses the picture for a long time, pop up a control of "search the same type", and the user clicks the control to perform the search operation of the target nail image.

202. Determining the nail beautifying area of each nail in the reference nail beautifying image through a single nail detection model;

for example, for preprocessing on the query side, which refers to a nail image, when a user triggers a nail search function at a client, all nail frames contained in the image are acquired through a detection model (a deep learning model, such as a single nail detection model) to obtain N bounding boxes.

203. Extracting the nail beautifying features of each nail beautifying region through a cosmetic feature model to obtain single nail beautifying feature information;

for example, for N bounding boxes, all nail frame features are extracted by cosmetic feature model (empdding) to obtain feature float values of 125 dimensions (or 256dim, 512 dim).

204. Calculating the average value of each single nail feature information to obtain average value nail feature information;

for example, all features are normalized separately, and the mean feature (n×128- >1×128) of all features is calculated.

205. Determining position coordinate information of each nail according to each nail region;

for example, coordinate information of the nail frame (N) results in a list of N bounding boxes, i.e., nail art areas such as list [ bbox1 (x 1, y1, x2, y 2), …, bbox N (x 1N, y1N, x2N, y 2N) ].

206. Determining an envelope frame according to the minimum abscissa, the maximum abscissa, the minimum ordinate and the maximum ordinate in the position coordinate information;

for example, the outer envelope frames (min (x 1,..x 1N), min (y 1,., y 1N), max (x 2, …, x 2N), max (y 2, …, y 2N)) of the N nail frames are calculated. The envelope frame is displayed on the client side and used as a query for searching nail art by a user, so that a stack of single nail small frames is avoided, and the user can select and search the poor experience problem.

207. Taking the average nail art characteristic information as frame characteristic information of an envelope frame;

correspondingly, taking the mean value characteristic (N x 128- >1 x 128) as the envelope frame characteristic of the whole image to search the nail. The background information interference of the non-nail frame is removed, so that single nail information is guaranteed, and the high-grade semantic information of matching of N finger nail beautification is fused.

208. According to the similarity of the feature information, matching corresponding frame feature information to be matched from a preset search library;

and positioning the envelope frame to be matched in the database through similar processing with the preprocessing of the query side, extracting the characteristic information of the frame to be matched, and constructing a search library according to the envelope frame to be matched and the characteristic information of the frame to be matched. The search library stores 128dim vector representations which are searched and contain nail art pictures, wherein the built search library is a note picture containing nail art in a community (such as a small red book sharing community) and supports real-time warehousing of new notes. The search library can store the pictures containing the nail art as required as 128dim of vector features and address links or note identifications (url/note id) for acquiring note picture information for presentation.

In this embodiment of the present application, after a search library is constructed, a vector index library is also constructed by vector search (e.g. small red book self-grinding LVE) or ANN (Approximate Nearest Neighbor), topN output (url/noteid) closest to the query vector in the search library is obtained by LVE/ANN index, and the same pattern of picture or note is displayed by url or noteid.

209. Taking a to-be-matched nail-beautifying image corresponding to the feature information of the to-be-matched frame with the similarity larger than a first preset threshold value and smaller than a second preset threshold value as a target nail-beautifying image;

accordingly, the search for the cosmetic style image of nail beauty cannot be strictly the same, that is, all nails are matched consistently, so that the image with very close distance or similarity in a certain range is filtered out by post-processing, for example, the input d is between a first preset threshold th1 and a second preset threshold th2 (th 1< d < th 2), when the vector distance is closer, the two pictures are more similar, that is, the nail beauty of the two pictures is more similar, th1 is a low threshold, when the distance of the two vectors is very large, the correlation is very weak, th2 is a high threshold, and the two very similar identical images are not output.

210. And sequentially outputting and displaying the target nail images in a visual interface of the equipment according to the similarity corresponding to the target nail images.

For example, referring to fig. 5, fig. 5 is a schematic illustration of a target nail image provided in an embodiment of the present application, wherein the uppermost small image is a reference nail image, the lower large image is a target nail image that is searched for and matched, and simultaneously, for each target nail image, a user identifier of a sharer of the target nail image, a text description attached to the sharer when sharing the image, and the like are displayed simultaneously in a display process.

It can be understood that, in the embodiment of the application, the characteristic nail feature information for describing the integral nail feature of the reference nail image is determined by referring to the single nail feature information of each nail in the nail image, and searching and matching are performed based on the characteristic nail feature information, so that the searching process can consider all nails in the image, only consider the nail feature, but filter the non-nail feature in the image, the accuracy of image searching is improved, searching and matching are performed by using the single characteristic nail feature information, and compared with searching one by one of single nail, the searching and matching efficiency of the nail image can be improved, and the user experience is improved.

Referring to fig. 6, fig. 6 is a guide diagram generated by searching a target nail image provided in the embodiment of the present application, in which a reference nail image (query side) a is identified and detected by a single nail detection model B to obtain a single nail detection result C, the single nail detection result C includes nail regions of each nail, and is represented by nail frames, then an envelope frame D is generated by calculating region position information of each nail region, and is displayed in a visual interface, single nail feature information of each nail frame is obtained by extracting a cosmetic feature model E, and an average value of each single nail feature information is calculated to obtain average nail feature information F, the average nail feature information F is used as frame feature information F 'of the envelope frame D, and a corresponding target nail image H is searched and matched from a search library G according to the frame feature information F', and the target nail images H are ordered and output.

The nail art retrieval scheme based on the enveloping single nail mean value characteristic retrieval and the enveloping frame display is designed, so that the problem that the same gesture is recalled in the whole hand examination process and the frame experience problem of nail art retrieval are solved.

In order to facilitate better implementation of the image processing method, the application also provides an image processing device based on the image processing method. Where the meaning of the terms is the same as in the image processing method described above, specific implementation details may be referred to in the description of the method embodiments.

Referring to fig. 7, fig. 7 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application, where the image processing apparatus may specifically be as follows:

a receiving module 301, configured to receive an input reference nail image;

the acquiring module 302 is configured to acquire single nail feature information of each nail in the reference nail image;

the determining module 303 is configured to determine feature information of a feature of a nail according to the feature information Shan Meijia, where the feature information of a feature of a nail describes a nail feature of the whole reference nail image;

the searching module 304 is used for searching a target nail image matched with the characteristic feature information of the characteristic nail from a preset search library;

And the display module 305 is used for displaying the target nail image on a visual interface.

Optionally, in some embodiments of the present application, the determining module 303 includes:

the search module 304 includes:

the acquisition subunit is used for acquiring the nail art image to be matched;

the search unit includes:

Wherein, in some embodiments of the present application, the obtaining module 302 includes:

According to the embodiment of the application, firstly, an input reference nail image is received by a receiving module 301, then, single nail feature information of each nail in the reference nail image is obtained by an obtaining module 302, then, characteristic nail feature information describing the whole nail feature of the reference nail image is determined by a determining module 303 according to the Shan Meijia feature information, then, a target nail image matched with the characteristic nail feature information is searched for by a searching module 304 from a preset searching library, and finally, the target nail image is displayed on a visual interface by a displaying module 305.

In addition, the present application further provides an electronic device, as shown in fig. 8, which shows a schematic structural diagram of the electronic device according to the present application, specifically:

the electronic device may include one or more processing cores 'processors 401, one or more computer-readable storage media's memory 402, power supply 403, and input unit 404, among other components. It will be appreciated by those skilled in the art that the electronic device structure shown in fig. 8 is not limiting of the electronic device and may include more or fewer components than shown, or may combine certain components, or a different arrangement of components. Wherein:

The processor 401 is a control center of the electronic device, connects various parts of the entire electronic device using various interfaces and lines, and performs various functions of the electronic device and processes data by running or executing software programs and/or modules stored in the memory 402, and calling data stored in the memory 402, thereby performing overall monitoring of the electronic device. Optionally, processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, an application program, etc., and the modem processor mainly processes wireless communication. It will be appreciated that the modem processor described above may not be integrated into the processor 401.

The memory 402 may be used to store software programs and modules, and the processor 401 executes various functional applications and data processing by executing the software programs and modules stored in the memory 402. The memory 402 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like; the storage data area may store data created according to the use of the electronic device, etc. In addition, memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

The electronic device further comprises a power supply 403 for supplying power to the various components, preferably the power supply 403 may be logically connected to the processor 401 by a power management system, so that functions of managing charging, discharging, and power consumption are performed by the power management system. The power supply 403 may also include one or more of any of a direct current or alternating current power supply, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and the like.

The electronic device may further comprise an input unit 404, which input unit 404 may be used for receiving input digital or character information and generating keyboard, mouse, joystick, optical or trackball signal inputs in connection with user settings and function control.

Although not shown, the electronic device may further include a display unit or the like, which is not described herein. In particular, in this embodiment, the processor 401 in the electronic device loads executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 executes the application programs stored in the memory 402, so as to implement the steps in any of the image processing methods provided in the embodiments of the present application.

According to the embodiment of the application, an input reference nail-beautifying image is received, single nail-beautifying feature information of each nail in the reference nail-beautifying image is obtained, characteristic nail-beautifying feature information is determined according to the Shan Meijia feature information, the characteristic nail-beautifying feature information describes the integral nail-beautifying feature of the reference nail-beautifying image, a target nail-beautifying image matched with the characteristic nail-beautifying feature information is searched from a preset search library, and the target nail-beautifying image is output and displayed on a visual interface.

The method comprises the steps of determining characteristic nail feature information for describing integral nail feature of a reference nail image by referring to single nail feature information of each nail in the nail image, and searching and matching based on the characteristic nail feature information, so that the searching process can consider all nails in the image, only consider the nail feature, filter out non-nail feature in the image, improve the accuracy of image searching, search and matching are carried out by single characteristic nail feature information, and compared with one-by-one searching of single nail, the searching and matching efficiency of the nail image can be improved, and the user experience is improved.

The specific implementation of each operation above may be referred to the previous embodiments, and will not be described herein.

Those of ordinary skill in the art will appreciate that all or a portion of the steps of the various methods of the above embodiments may be performed by instructions, or by instructions controlling associated hardware, which may be stored in a computer-readable storage medium and loaded and executed by a processor.

To this end, the present application provides a computer readable storage medium having stored thereon a computer program that can be loaded by a processor to perform the steps of any of the image processing methods provided herein.

Wherein the computer-readable storage medium may comprise: read Only Memory (ROM), random access Memory (RAM, random Access Memory), magnetic or optical disk, and the like.

Because the instructions stored in the computer readable storage medium can execute the steps in any of the image processing methods provided in the present application, the beneficial effects that can be achieved by any of the image processing methods provided in the present application can be achieved, and detailed descriptions of the foregoing embodiments are omitted herein.

The foregoing has outlined some of the more detailed description of the image processing method, apparatus, electronic device, and computer readable storage medium that are provided herein, wherein specific embodiments are presented to facilitate the understanding of the method and core idea of the invention; meanwhile, as those skilled in the art will vary in the specific embodiments and application scope according to the ideas of the present invention, the present description should not be construed as limiting the present invention in summary.

It should be noted that, in the specific embodiment of the present application, the retrieval matching basis (reference nail image) input by the user, other related data such as nail image shared by the user, etc. when the above embodiments of the present application are applied to specific products or technologies, user permission or consent needs to be obtained, and the collection, use and processing of related data need to comply with related laws and regulations and standards of related countries and regions.

Claims

1. An image processing method, comprising:

receiving an input reference nail image;

and displaying the target nail art image on a visual interface.

2. The method of claim 1, wherein said determining characterizing nail art characteristic information from said Shan Meijia characteristic information comprises:

3. The method of claim 1, wherein prior to searching for a target nail image matching the characteristic nail feature information from a preset search library, the method further comprises:

acquiring coordinate information of each nail in the reference nail image;

taking the characteristic feature information of the characteristic nail as frame feature information of the envelope frame;

the searching the target nail art image matched with the characteristic information of the characteristic nail art from the preset search library comprises the following steps:

4. The method of claim 3, wherein before searching for the target nail image matching the frame feature information from a preset search library, the method further comprises:

acquiring a nail art image to be matched;

constructing a preset search library according to feature information of frames to be matched corresponding to the nail images to be matched;

the searching the target nail art image matched with the frame characteristic information from the preset search library comprises the following steps:

5. The method according to claim 4, wherein searching for target frame feature information matching the frame feature information from frame feature information to be matched in the preset search library comprises:

6. The method of claim 2, wherein after determining an envelope frame from the coordinate information of the respective nail, the method further comprises:

7. The method according to claim 1, wherein the acquiring single nail feature information of each nail in the reference nail image comprises:

8. An image processing apparatus, comprising:

the receiving module is used for receiving the input reference nail art image;

9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps in the image processing method according to any one of claims 1-7 when the computer program is executed.

10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, implements the steps in the image processing method according to any one of claims 1-7.