CN111104988B - Image recognition method and related device - Google Patents
Image recognition method and related device Download PDFInfo
- Publication number
- CN111104988B CN111104988B CN201911402913.5A CN201911402913A CN111104988B CN 111104988 B CN111104988 B CN 111104988B CN 201911402913 A CN201911402913 A CN 201911402913A CN 111104988 B CN111104988 B CN 111104988B
- Authority
- CN
- China
- Prior art keywords
- subclass
- major class
- target object
- class
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The embodiment of the application discloses an image identification method which is applied to electronic equipment and comprises the following steps: acquiring a first image containing image information of a target object; performing large-class identification on the target object on the local terminal equipment according to the pre-trained large-class classification model and the first image to obtain a large-class classification result of the target object; and carrying out subclass recognition on the target object according to a pre-trained subclass classification model, a major class classification result and the first image to obtain a target subclass classification result of the target object, wherein the subclass classification model is a classification model in a cloud Graphics Processor (GPU) server. The embodiment of the application is beneficial to improving the efficiency of object identification.
Description
Technical Field
The application relates to the technical field of image recognition, in particular to an image recognition method and a related device.
Background
Image recognition is an important area of artificial intelligence. It refers to a technique of performing object recognition on an image to recognize objects and objects of various modes.
The current general classification algorithm is to send the image to the algorithm model directly to perform local reasoning or send the image to a cloud server to perform reasoning so as to obtain the classification of the object in the image.
Disclosure of Invention
The embodiment of the application provides an image recognition method and a related device, aiming at improving the object recognition efficiency.
In a first aspect, an embodiment of the present application provides an image recognition method, applied to an electronic device, where the method includes:
acquiring a first image containing image information of a target object;
performing large-class identification on the target object on local equipment according to a pre-trained large-class classification model and the first image to obtain a large-class classification result of the target object;
and carrying out subclass recognition on the target object according to a pre-trained subclass classification model, the major class classification result and the first image to obtain a target subclass classification result of the target object, wherein the subclass classification model is a classification model in a cloud Graphics Processor (GPU) server.
In a second aspect, an embodiment of the present application provides an image recognition apparatus, applied to an electronic device, where the apparatus includes a processing unit and a communication unit, where,
the processing unit is used for acquiring a first image containing image information of a target object through the communication unit; performing large-class identification on the target object on local equipment according to a pre-trained large-class classification model and the first image to obtain a large-class classification result of the target object; and performing subclass recognition on the target object according to a pre-trained subclass classification model, the major class classification result and the first image to obtain a target subclass classification result of the target object, wherein the subclass classification model is a classification model in a cloud Graphics Processor (GPU) server.
In a third aspect, an embodiment of the present application provides an electronic device, including a processor, a memory, a communication interface, and one or more programs, where the one or more programs are stored in the memory and configured to be executed by the processor, the programs including instructions for performing steps in any of the methods of the first aspect of the embodiments of the present application.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program causes a computer to perform part or all of the steps as described in any of the methods of the first aspect of the embodiments of the present application.
In a fifth aspect, embodiments of the present application provide a computer program product, wherein the computer program product comprises a non-transitory computer readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps described in any of the methods of the first aspect of the embodiments of the present application. The computer program product may be a software installation package.
It can be seen that, in the embodiment of the present application, the electronic device first acquires a first image including image information of the target object; secondly, carrying out large-class identification on the target object on the local terminal equipment according to a pre-trained large-class classification model and a first image to obtain a large-class classification result of the target object; and finally, carrying out subclass recognition on the target object according to a pre-trained subclass classification model, a major class classification result and the first image to obtain a target subclass classification result of the target object, wherein the subclass classification model is a classification model in a cloud Graphics Processor (GPU) server. Therefore, the electronic equipment can execute the classification algorithm in a grading manner through the local model and the cloud model, so that the operation complexity of the electronic equipment can be reduced, the cloud processing time delay can be reduced, the local and cloud computing forces are integrated to accurately identify the object, and the image identification efficiency and accuracy are improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments or the description of the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a system architecture diagram of an image recognition system according to an embodiment of the present application;
fig. 2A is a schematic flow chart of an image recognition method according to an embodiment of the present application;
FIG. 2B is a flowchart illustrating another image recognition method according to an embodiment of the present application;
FIG. 2C is a flowchart illustrating another image recognition method according to an embodiment of the present application;
FIG. 3 is a flowchart of another image recognition method according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application;
fig. 5 is a block diagram showing functional units of an image recognition apparatus according to an embodiment of the present application.
Detailed Description
In order that those skilled in the art will better understand the present application, a technical solution in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The terms first, second and the like in the description and in the claims and in the above-described figures are used for distinguishing between different objects and not necessarily for describing a sequential or chronological order. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed steps or elements but may include other steps or elements not listed or inherent to such process, method, article, or apparatus.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.
The electronic device according to the embodiment of the present application may be an electronic device with communication capability, where the electronic device may include various handheld devices, vehicle-mounted devices, wearable devices, computing devices, or other processing devices connected to a wireless modem, and various types of User Equipment (UE), mobile Station (MS), terminal device (terminal device), and so on.
At present, the mobile terminal has limited computing power, if the whole process reasoning is carried out at the mobile terminal, the power consumption and the time consumption are very high, the image is directly sent to the cloud end, the time consumption for sending the picture data is high, the resource consumption of a server is high, and the cost is high.
In view of the foregoing, an embodiment of the present application provides an image recognition method, and the embodiment of the present application is described in detail below with reference to the accompanying drawings.
As shown in fig. 1, an embodiment of the present application provides an image recognition system 100, where the image recognition system includes an electronic device 110, a distribution server 120, and a cloud graphics processor GPU server 130 (two in the drawing), where the electronic device 110 is connected to the distribution server 120, the distribution server is connected to the GPU server 130, and the electronic device may also be directly connected to the GPU server (optionally, without constraint), where a pre-trained major class classification model, such as a dog, is provided in the electronic device, and a pre-trained specific major class classification model, such as a dog class classification model, is provided in the GPU server (a minor class may include a haxakey, a mizus, and so on) under the dog major class classification model.
Referring to fig. 2A, fig. 2A is a schematic flow chart of an image recognition method according to an embodiment of the present application, which can be implemented by the image recognition system shown in fig. 1, and the image recognition method includes the following operations as shown in the figure.
S201, the electronic device acquires a first image including image information of the target object.
The target object may be various targets such as a person, a cat, a dog, etc., which are not limited herein.
The image information of the target object should at least include image information only adapting to the specific shape feature of the target object, for example, a target object such as a person or an animal, may include head or torso information, for example, an object such as a vehicle, a table chair, or the like, and may include an image of a component specific to the object, for example, a wheel of a vehicle, a table top of a table, or the like.
S202, the electronic equipment performs large-class identification on the target object on the local equipment according to a pre-trained large-class classification model and the first image, and a large-class classification result of the target object is obtained.
The training of the large class classification model can be completed in the cloud, and the large class classification model is issued to the electronic equipment after being trained by the cloud, or is completed off-line by the local end of the electronic equipment, such as model training in an idle period. The general classification model may be, for example, a mobile device network mobilent of google, etc., and is not limited only herein.
The major class classification result is a prediction of the probability that the target object belongs to certain major classes, for example, the probability that one target object belongs to the major class is 0.9, the probability that one target object belongs to the major class is 0.08, and the probability that one target object belongs to the major class is 0.01.
S203, the electronic equipment performs subclass recognition on the target object according to a pre-trained subclass classification model, the major class classification result and the first image to obtain a target subclass classification result of the target object, wherein the subclass classification model is a classification model in a cloud graphics processor GPU server.
The subclass classification model can be trained in the cloud and issued to each GPU server, or the local end of each GPU server is trained. The subclass classification model may employ, for example, google's mobile device network mobilent et al, without limitation.
It can be seen that, in the embodiment of the present application, the electronic device first acquires a first image including image information of the target object; secondly, carrying out large-class identification on the target object on the local terminal equipment according to a pre-trained large-class classification model and a first image to obtain a large-class classification result of the target object; and finally, carrying out subclass recognition on the target object according to a pre-trained subclass classification model, a major class classification result and the first image to obtain a target subclass classification result of the target object, wherein the subclass classification model is a classification model in a cloud Graphics Processor (GPU) server. Therefore, the electronic equipment can execute the classification algorithm in a grading manner through the local model and the cloud model, so that the operation complexity of the electronic equipment can be reduced, the cloud processing time delay can be reduced, the local and cloud computing forces are integrated to accurately identify the object, and the image identification efficiency and accuracy are improved.
In one possible example, the broad class classification result includes a first confidence that the target object is predicted to be of each broad class; the step of performing subclass recognition on the target object according to a pre-trained subclass classification model, the major class classification result and the first image to obtain a target subclass classification result of the target object, including: determining at least one major class with a first confidence coefficient greater than a first preset confidence coefficient in the major class classification result; and determining a target subclass classification result of the target object according to the first confidence coefficient of the at least one major class, the first image and a pre-trained subclass classification model, wherein the second confidence coefficient of the target object predicted as the target subclass in the target subclass classification result is larger than the second preset confidence coefficient.
The first preset confidence may be, for example, a preset value of 90%, 95%, 60%, etc., and the second preset confidence may be a preset value of 95%, 90%, etc., which is not limited herein. The target subclass classification result may be a fine classification under the subclass, such as asians, europeans, etc. under the human subclass, as well as a genipin, halftoning, etc. under the dog subclass.
In a specific implementation, the implementation manner of determining, by the electronic device, the target subclass classification result of the target object according to the first confidence level of the at least one major class, the first image and the pre-trained subclass classification model may be various, which is not limited only herein.
In this example, after the electronic device obtains the major class classification result in the hierarchical image recognition process, the major class meeting the preliminary condition is screened out through the first preset confidence, then the first image is subjected to fine classification recognition according to the first confidence of the screened major class and the minor class classification model in the GPU server, and the minor class meeting the condition is screened out through the second preset confidence.
In one possible example, before the acquiring the first image including the image information of the target object, the method further includes: acquiring an original image containing image information of a target object; invoking a pre-trained detection model to detect the original image to obtain the position of the image information of the target object; and clipping the original image according to the position to obtain a first image only containing the image information of the target object.
The detection model may be a local or cloud training model, for example, a google mobile device network mobilent or the like may be used as the detection model, which is not limited herein. The detection model may also be a general target position detection algorithm, and may be flexibly set according to actual needs, which is not limited only herein.
Therefore, in the example, the detection model is used for primarily cutting the graph, so that the data size can be reduced, the transmission and processing speed can be improved, and the recognition accuracy can be improved.
Furthermore, the first image may also be the original image itself, i.e. the original image is directly used for large class recognition without preprocessing.
In one possible example, as shown in fig. 2B, the determining the target subclass classification result of the target object according to the first confidence level of the at least one major class, the first image, and the pre-trained subclass classification model includes: transmitting a first confidence level of the at least one major class, the first image, to a distribution server; receiving a target subclass classification result from the distribution server; the target subclass classification result is obtained by the distribution server executing the following operations: the following processing is performed for the at least one major class in the order of the first confidence level from high to low: detecting whether a reference subclass classification result corresponding to the currently processed subclass contains a subclass with a second confidence degree larger than a second preset confidence degree or not, wherein the reference subclass classification result is a classification result obtained by executing a preset subclass prediction mechanism; and if the detection result is yes, the reference subclass classification result corresponding to the currently processed major class is used as a target subclass classification result to be sent to the terminal.
The target subclass classification result is preferably a single classification result, and may also be multiple classification results, which depend on the imaging quality of the first picture, such as integrity, sharpness, and other indicators.
In this possible example, the preset subclass prediction mechanism includes the following processes: selecting GPU servers with major class categories matched with the currently processed major classes from a plurality of GPU servers; sending the currently processed major class and the first image to the selected GPU server, wherein the GPU server is used for processing the first image to obtain a reference minor class classification result of the target object in the currently processed major class; and transmitting the reference subclass classification result to the distribution server. Therefore, the GPU server corresponds to the subclass classification model of each major class one by one, namely, the special class is special, the focusing degree is high, the algorithm accuracy and efficiency are high, and the accurate and efficient recognition of the subclass is facilitated.
In this possible example, the distribution server is further configured to perform the following operations: if the detection result is negative, judging whether the currently processed major class is the last unprocessed major class in the at least one major class; if the judgment result is yes, a notification that the subclass classification cannot be identified is sent to the terminal; if the judgment result is negative, updating the currently processed major class to be the next major class to be processed.
In a specific implementation, after receiving the notification that the subclass classification cannot be identified, the electronic device may output the notification through a screen or the like.
In this example, the electronic device interacts with the distribution server, and the distribution server is instructed to interact with the GPU server to predict the subclass classification result step by step according to the order from high to low according to the first confidence level until the target subclass classification result meeting the requirements is found, which is favorable for quick acquisition of the recognition result, and improves the speed and efficiency of acquiring the final recognition result at the electronic device side.
In one possible example, as shown in fig. 2C, the determining the target subclass classification result of the target object according to the first confidence level of the at least one major class, the first image, and the pre-trained subclass classification model includes: the following processing is performed for the at least one major class in the order of the first confidence level from high to low: detecting whether a reference subclass classification result corresponding to the currently processed subclass contains a subclass with a second confidence degree larger than a second preset confidence degree or not, wherein the reference subclass classification result is a classification result obtained by executing a preset subclass prediction mechanism; and if the detection result is yes, taking the reference subclass classification result corresponding to the currently processed major class as a target subclass classification result.
In this possible example, the preset subclass prediction mechanism includes the following processes: selecting GPU servers with major class categories matched with the currently processed major classes from a plurality of GPU servers; sending the currently processed major class and the first image to the selected GPU server, wherein the GPU server is used for processing the first image to obtain a reference minor class classification result of the target object in the currently processed major class; and transmitting the reference subclass classification result to the electronic device. Therefore, the GPU server corresponds to the subclass classification model of each major class one by one, namely, the special class is special, the focusing degree is high, the algorithm accuracy and efficiency are high, and the accurate and efficient recognition of the subclass is facilitated. Therefore, the GPU server corresponds to the subclass classification model of each major class one by one, namely, the special class is special, the focusing degree is high, the algorithm accuracy and efficiency are high, and the accurate and efficient recognition of the subclass is facilitated.
In this possible example, the method further includes: if the detection result is negative, judging whether the currently processed major class is the last unprocessed major class in the at least one major class; if the judgment result is yes, generating and outputting a notification that the subclass classification cannot be identified; if the judgment result is negative, updating the currently processed major class to be the next major class to be processed.
The target subclass classification result is preferably a single classification result, and may also be multiple classification results, which depend on the imaging quality of the first picture, such as integrity, sharpness, and other indicators.
In this example, the electronic device directly interacts with the GPU server to predict the subclass classification result step by step according to the order of the first confidence from high to low until the target subclass classification result meeting the requirement is found, which is favorable for quick acquisition of the recognition result and improves the speed and efficiency of acquiring the final recognition result at the electronic device side.
Referring to fig. 3, fig. 3 is a schematic flow chart of an image recognition method according to an embodiment of the present application, as shown in the figure, the image recognition method includes the following steps.
S301, the electronic equipment acquires an original image containing image information of a target object;
s302, the electronic equipment invokes a pre-trained detection model to detect the original image, so as to obtain the position of the image information of the target object;
and S303, the electronic equipment cuts the original image according to the position to obtain a first image only containing the image information of the target object.
S304, the electronic equipment acquires a first image containing image information of the target object.
S305, the electronic equipment carries out large-class identification on the target object on the local equipment according to a pre-trained large-class classification model and the first image to obtain a large-class classification result of the target object; the major class classification result includes a first confidence that the target object is predicted to be of each major class.
S306, the electronic equipment determines at least one major class with the first confidence coefficient larger than the first preset confidence coefficient in the major class classification result, and executes the following processing procedure for the at least one major class according to the sequence of the first confidence coefficient from high to low:
s307, selecting GPU servers with major class categories matched with the major classes currently processed from a plurality of GPU servers;
s308, sending the currently processed major class and the first image to the selected GPU server, wherein the GPU server is used for processing the first image to obtain a reference minor class classification result of the target object in the currently processed major class; and transmitting the reference subclass classification result to the electronic device.
S309, receiving the reference subclass classification result from the GPU server corresponding to the currently processed subclass;
S310, detecting whether a reference subclass classification result corresponding to the currently processed major class contains a subclass with a second confidence degree larger than a second preset confidence degree, wherein the reference subclass classification result is a classification result obtained by executing a preset subclass prediction mechanism; if yes, go to step S311; if not, go to step S312.
S311, taking the reference subclass classification result corresponding to the currently processed major class as a target subclass classification result.
S312, judging whether the currently processed major class is the last unprocessed major class in the at least one major class; if yes, go to step S313, otherwise go to step S314.
S313, generating and outputting a notification that the subclass classification cannot be identified.
And S314, updating the currently processed major class to be the next major class to be processed, namely returning to the step S307 to continue execution.
It can be seen that, in the embodiment of the present application, the electronic device first acquires a first image including image information of the target object; secondly, carrying out large-class identification on the target object on the local terminal equipment according to a pre-trained large-class classification model and a first image to obtain a large-class classification result of the target object; and finally, carrying out subclass recognition on the target object according to a pre-trained subclass classification model, a major class classification result and the first image to obtain a target subclass classification result of the target object, wherein the subclass classification model is a classification model in a cloud Graphics Processor (GPU) server. Therefore, the electronic equipment can execute the classification algorithm in a grading manner through the local model and the cloud model, so that the operation complexity of the electronic equipment can be reduced, the cloud processing time delay can be reduced, the local and cloud computing forces are integrated to accurately identify the object, and the image identification efficiency and accuracy are improved.
Referring to fig. 4, in accordance with the embodiment shown in fig. 2A and fig. 3, fig. 4 is a schematic structural diagram of an electronic device 400 according to an embodiment of the present application, as shown in the fig. 4, the electronic device 400 includes an application processor 410, a memory 420, a communication interface 430, and one or more programs 421, where the one or more programs 421 are stored in the memory 420 and configured to be executed by the application processor 410, and the one or more programs 421 include instructions for executing any of the steps in the method embodiments. The following is a detailed description.
Acquiring a first image containing image information of a target object; performing large-class identification on the target object on local equipment according to a pre-trained large-class classification model and the first image to obtain a large-class classification result of the target object; and performing subclass recognition on the target object according to a pre-trained subclass classification model, the major class classification result and the first image to obtain a target subclass classification result of the target object, wherein the subclass classification model is a classification model in a cloud Graphics Processor (GPU) server.
In one possible example, the broad class classification result includes a first confidence that the target object is predicted to be of each broad class; in terms of performing subclass recognition on the target object according to the pre-trained subclass classification model, the major class classification result and the first image to obtain a target subclass classification result of the target object, the instructions in the program 421 are specifically configured to perform the following operations: determining at least one major class with a first confidence coefficient greater than a first preset confidence coefficient in the major class classification result; and determining a target subclass classification result of the target object according to the first confidence coefficient of the at least one major class, the first image and a pre-trained subclass classification model, wherein the second confidence coefficient of the target object predicted as the target subclass in the target subclass classification result is larger than the second preset confidence coefficient.
In one possible example, the program 421 further includes instructions for: before acquiring a first image containing image information of a target object, acquiring an original image containing the image information of the target object; and calling a pre-trained detection model to detect the original image, so as to obtain the position of the image information of the target object; and clipping the original image according to the position to obtain a first image only containing the image information of the target object.
In one possible example, in determining the target subclass classification result of the target object according to the first confidence level of the at least one major class, the first image, and a pre-trained subclass classification model, the instructions in the program 421 are specifically configured to: transmitting a first confidence level of the at least one major class, the first image, to a distribution server; and receiving a target subclass classification result from the distribution server; the target subclass classification result is obtained by the distribution server executing the following operations: the following processing is performed for the at least one major class in the order of the first confidence level from high to low: detecting whether a reference subclass classification result corresponding to the currently processed subclass contains a subclass with a second confidence degree larger than a second preset confidence degree or not, wherein the reference subclass classification result is a classification result obtained by executing a preset subclass prediction mechanism; and if the detection result is yes, the reference subclass classification result corresponding to the currently processed major class is used as a target subclass classification result to be sent to the terminal.
In one possible example, the distribution server is further configured to: if the detection result is negative, judging whether the currently processed major class is the last unprocessed major class in the at least one major class; if the judgment result is yes, a notification that the subclass classification cannot be identified is sent to the terminal; if the judgment result is negative, updating the currently processed major class to be the next major class to be processed.
In one possible example, in determining the target subclass classification result of the target object according to the first confidence level of the at least one major class, the first image, and a pre-trained subclass classification model, the instructions in the program 421 are specifically configured to: the following processing is performed for the at least one major class in the order of the first confidence level from high to low: detecting whether a reference subclass classification result corresponding to the currently processed subclass contains a subclass with a second confidence degree larger than a second preset confidence degree or not, wherein the reference subclass classification result is a classification result obtained by executing a preset subclass prediction mechanism; and if the detection result is yes, the reference subclass classification result corresponding to the currently processed major class is used as a target subclass classification result to be sent to the terminal.
In one possible example, the program 421 further includes instructions for: if the detection result is negative, judging whether the currently processed major class is the last unprocessed major class in the at least one major class; if the judgment result is yes, generating and outputting a notification that the subclass classification cannot be identified; if the judgment result is negative, updating the currently processed major class to be the next major class to be processed.
In one possible example, the preset subclass prediction mechanism includes the following processes: selecting GPU servers with major class categories matched with the currently processed major classes from a plurality of GPU servers; and sending the currently processed major class and the first image to the selected GPU server, wherein the GPU server is used for processing the first image to obtain a reference minor class classification result of the target object in the currently processed major class.
The foregoing description of the embodiments of the present application has been presented primarily in terms of a method-side implementation. It will be appreciated that the electronic device, in order to achieve the above-described functions, includes corresponding hardware structures and/or software modules that perform the respective functions. Those of skill in the art will readily appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as hardware or combinations of hardware and computer software. Whether a function is implemented as hardware or computer software driven hardware depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The embodiment of the application can divide the functional units of the electronic device according to the method example, for example, each functional unit can be divided corresponding to each function, and two or more functions can be integrated in one processing unit. The integrated units may be implemented in hardware or in software functional units. It should be noted that, in the embodiment of the present application, the division of the units is schematic, which is merely a logic function division, and other division manners may be implemented in actual practice.
Fig. 5 is a block diagram showing functional units of an image recognition apparatus 500 according to an embodiment of the present application. The image recognition apparatus 500 is applied to an electronic device, and the image recognition apparatus includes a processing unit 501 and a communication unit 502, where the processing unit 501 is configured to perform any step in the above method embodiments, and when performing data transmission such as sending, the communication unit 502 is selectively invoked to complete a corresponding operation. The following is a detailed description.
The processing unit 501 is configured to acquire, through the communication unit 502, a first image including image information of a target object; performing large-class identification on the target object on local equipment according to a pre-trained large-class classification model and the first image to obtain a large-class classification result of the target object; and performing subclass recognition on the target object according to a pre-trained subclass classification model, the major class classification result and the first image to obtain a target subclass classification result of the target object, wherein the subclass classification model is a classification model in a cloud Graphics Processor (GPU) server.
In one possible example, the broad class classification result includes a first confidence that the target object is predicted to be of each broad class; in terms of performing subclass recognition on the target object according to the pre-trained subclass classification model, the major class classification result and the first image to obtain a target subclass classification result of the target object, the processing unit 501 is specifically configured to: determining at least one major class with a first confidence coefficient greater than a first preset confidence coefficient in the major class classification result; and determining a target subclass classification result of the target object according to the first confidence coefficient of the at least one major class, the first image and a pre-trained subclass classification model, wherein the second confidence coefficient of the target object predicted as the target subclass in the target subclass classification result is larger than the second preset confidence coefficient.
In a possible example, before the processing unit 501 obtains the first image including the image information of the target object through the communication unit 502, the processing unit is further configured to: acquiring an original image containing image information of a target object through the communication unit 502; and calling a pre-trained detection model to detect the original image, so as to obtain the position of the image information of the target object; and clipping the original image according to the position to obtain a first image only containing the image information of the target object.
In one possible example, in determining the target subclass classification result of the target object according to the first confidence level of the at least one major class, the first image and the pre-trained subclass classification model, the processing unit 501 is specifically configured to: transmitting a first confidence level of the at least one major class, the first image, to a distribution server through the communication unit 502; and receiving, by the communication unit 502, a target subclass classification result from the distribution server; the target subclass classification result is obtained by the distribution server executing the following operations: the following processing is performed for the at least one major class in the order of the first confidence level from high to low: detecting whether a reference subclass classification result corresponding to the currently processed subclass contains a subclass with a second confidence degree larger than a second preset confidence degree or not, wherein the reference subclass classification result is a classification result obtained by executing a preset subclass prediction mechanism; and if the detection result is yes, the reference subclass classification result corresponding to the currently processed major class is used as a target subclass classification result to be sent to the terminal.
In one possible example, the distribution server is further configured to: if the detection result is negative, judging whether the currently processed major class is the last unprocessed major class in the at least one major class; if the judgment result is yes, a notification that the subclass classification cannot be identified is sent to the terminal; if the judgment result is negative, updating the currently processed major class to be the next major class to be processed.
In one possible example, the processing unit 501 is specifically configured to, in the determining the target subclass classification result of the target object according to the first confidence level of the at least one major class, the first image and the pre-trained subclass classification model: the following processing is performed for the at least one major class in the order of the first confidence level from high to low: detecting whether a reference subclass classification result corresponding to the currently processed subclass contains a subclass with a second confidence degree larger than a second preset confidence degree or not, wherein the reference subclass classification result is a classification result obtained by executing a preset subclass prediction mechanism; and if the detection result is yes, the reference subclass classification result corresponding to the currently processed major class is used as a target subclass classification result to be sent to the terminal.
In one possible example, the processing unit 501 is further configured to: if the detection result is negative, judging whether the currently processed major class is the last unprocessed major class in the at least one major class; if the judgment result is yes, generating and outputting a notification that the subclass classification cannot be identified; if the judgment result is negative, updating the currently processed major class to be the next major class to be processed.
In one possible example, the preset subclass prediction mechanism includes the following processes: selecting GPU servers with major class categories matched with the currently processed major classes from a plurality of GPU servers; and sending the currently processed major class and the first image to the selected GPU server, wherein the GPU server is used for processing the first image to obtain a reference minor class classification result of the target object in the currently processed major class.
The image recognition device 500 may further comprise a storage unit 503 for storing program code and data of the electronic apparatus. The processing unit 501 may be a processor, the communication unit 502 may be a touch display screen or a transceiver, and the storage unit 503 may be a memory.
It can be understood that, since the method embodiment and the apparatus embodiment are different presentation forms of the same technical concept, the content of the method embodiment portion in the present application should be synchronously adapted to the apparatus embodiment portion, which is not described herein.
The embodiment of the application also provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program makes a computer execute part or all of the steps of any one of the above method embodiments, and the computer includes an electronic device.
Embodiments of the present application also provide a computer program product comprising a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform part or all of the steps of any one of the methods described in the method embodiments above. The computer program product may be a software installation package, said computer comprising an electronic device.
It should be noted that, for simplicity of description, the foregoing method embodiments are all described as a series of acts, but it should be understood by those skilled in the art that the present application is not limited by the order of acts described, as some steps may be performed in other orders or concurrently in accordance with the present application. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily required for the present application.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to related descriptions of other embodiments.
In the several embodiments provided by the present application, it should be understood that the disclosed apparatus may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, such as the above-described division of units, merely a division of logic functions, and there may be additional manners of dividing in actual implementation, such as multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, or may be in electrical or other forms.
The units described above as separate components may or may not be physically separate, and components shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units described above, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable memory. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a memory, comprising several instructions for causing a computer device (which may be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the above-mentioned method of the various embodiments of the present application. And the aforementioned memory includes: a U-disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Those of ordinary skill in the art will appreciate that all or a portion of the steps in the various methods of the above embodiments may be implemented by a program that instructs associated hardware, and the program may be stored in a computer readable memory, which may include: flash disk, read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic disk or optical disk.
The foregoing has outlined rather broadly the more detailed description of embodiments of the application, wherein the principles and embodiments of the application are explained in detail using specific examples, the above examples being provided solely to facilitate the understanding of the method and core concepts of the application; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present application, the present description should not be construed as limiting the present application in view of the above.
Claims (8)
1. An image recognition method, applied to an electronic device, comprising:
acquiring a first image containing image information of a target object;
performing large-class recognition on the target object on local equipment according to a pre-trained large-class classification model and the first image to obtain a large-class classification result of the target object, wherein the large-class classification result comprises a first confidence degree that the target object is predicted to be of each large class;
Performing subclass recognition on the target object according to a pre-trained subclass classification model, the major class classification result and the first image to obtain a target subclass classification result of the target object, wherein the method comprises the following steps:
determining at least one major class with a first confidence coefficient greater than a first preset confidence coefficient in the major class classification result;
the following processing is performed for the at least one major class in the order of the first confidence level from high to low:
selecting a GPU server with a major class matched with the currently processed major class from a plurality of cloud graphic processor GPU servers;
sending the currently processed major class and the first image to the selected GPU server, wherein the GPU server is used for processing the first image to obtain a reference minor class classification result of the target object in the currently processed major class;
detecting whether a reference subclass classification result corresponding to the currently processed major class contains a subclass with a second confidence coefficient larger than a second preset confidence coefficient or not;
and if the detection result is yes, taking the reference subclass classification result corresponding to the currently processed major class as the target subclass classification result of the target object.
2. The method of claim 1, wherein prior to the acquiring the first image comprising the image information of the target object, the method further comprises:
Acquiring an original image containing image information of a target object;
invoking a pre-trained detection model to detect the original image to obtain the position of the image information of the target object;
and clipping the original image according to the position to obtain a first image only containing the image information of the target object.
3. The method according to claim 1 or 2, wherein after the performing, on the local device, the large class recognition on the target object according to the pre-trained large class classification model and the first image, to obtain a large class classification result of the target object, the method further comprises:
transmitting a first confidence level of the at least one major class, the first image, to a distribution server;
receiving a target subclass classification result from the distribution server;
the target subclass classification result is obtained by the distribution server executing the following operations:
the following processing is performed for the at least one major class in the order of the first confidence level from high to low:
detecting whether a reference subclass classification result corresponding to the currently processed subclass contains a subclass with a second confidence degree larger than a second preset confidence degree or not, wherein the reference subclass classification result is a classification result obtained by executing a preset subclass prediction mechanism;
And if the detection result is yes, the reference subclass classification result corresponding to the currently processed major class is used as a target subclass classification result to be sent to the terminal.
4. A method according to claim 3, wherein the distribution server is further adapted to:
if the detection result is negative, judging whether the currently processed major class is the last unprocessed major class in the at least one major class;
if the judgment result is yes, a notification that the subclass classification cannot be identified is sent to the terminal;
if the judgment result is negative, updating the currently processed major class to be the next major class to be processed.
5. The method according to claim 1, wherein the method further comprises:
if the detection result is negative, judging whether the currently processed major class is the last unprocessed major class in the at least one major class;
if the judgment result is yes, generating and outputting a notification that the subclass classification cannot be identified;
if the judgment result is negative, updating the currently processed major class to be the next major class to be processed.
6. An image recognition apparatus, characterized by being applied to an electronic device, comprises a processing unit and a communication unit, wherein,
The processing unit is used for acquiring a first image containing image information of a target object through the communication unit; carrying out large-class recognition on the target object on the local terminal equipment according to a pre-trained large-class classification model and the first image to obtain a large-class classification result of the target object, wherein the large-class classification result comprises a first confidence degree that the target object is predicted to be of each large class; and performing subclass recognition on the target object according to a pre-trained subclass classification model, the major class classification result and the first image to obtain a target subclass classification result of the target object, wherein the method comprises the following steps of: determining at least one major class with a first confidence coefficient greater than a first preset confidence coefficient in the major class classification result; the following processing is performed for the at least one major class in the order of the first confidence level from high to low: selecting a GPU server with a major class matched with the currently processed major class from a plurality of cloud graphic processor GPU servers; sending the currently processed major class and the first image to the selected GPU server, wherein the GPU server is used for processing the first image to obtain a reference minor class classification result of the target object in the currently processed major class; detecting whether a reference subclass classification result corresponding to the currently processed major class contains a subclass with a second confidence coefficient larger than a second preset confidence coefficient or not; and if the detection result is yes, taking the reference subclass classification result corresponding to the currently processed major class as the target subclass classification result of the target object.
7. An electronic device comprising a processor, a memory, a communication interface, and one or more programs stored in the memory and configured to be executed by the processor, the programs comprising instructions for performing the steps in the method of any of claims 1-5.
8. A computer-readable storage medium, characterized in that it stores a computer program for electronic data exchange, wherein the computer program causes a computer to perform the method according to any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911402913.5A CN111104988B (en) | 2019-12-28 | 2019-12-28 | Image recognition method and related device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911402913.5A CN111104988B (en) | 2019-12-28 | 2019-12-28 | Image recognition method and related device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111104988A CN111104988A (en) | 2020-05-05 |
| CN111104988B true CN111104988B (en) | 2023-09-29 |
Family
ID=70424450
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911402913.5A Active CN111104988B (en) | 2019-12-28 | 2019-12-28 | Image recognition method and related device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111104988B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111797894A (en) * | 2020-05-27 | 2020-10-20 | 北京齐尔布莱特科技有限公司 | Image classification method and computing device |
| CN113516486A (en) * | 2021-04-07 | 2021-10-19 | 阿里巴巴新加坡控股有限公司 | Image recognition method, device, equipment and storage medium |
| CN113505700A (en) * | 2021-07-12 | 2021-10-15 | 北京字跳网络技术有限公司 | Image processing method, device, equipment and storage medium |
| CN113516088B (en) * | 2021-07-22 | 2024-02-27 | 中移（杭州）信息技术有限公司 | Object recognition method, device and computer readable storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018054283A1 (en) * | 2016-09-23 | 2018-03-29 | 北京眼神科技有限公司 | Face model training method and device, and face authentication method and device |
| WO2018119599A1 (en) * | 2016-12-26 | 2018-07-05 | 深圳前海达闼云端智能科技有限公司 | Method and device for searching for person and communication system |
| CN207993164U (en) * | 2017-07-24 | 2018-10-19 | 图灵通诺（北京）科技有限公司 | The checkout apparatus of image recognition technology based on convolutional neural networks |
| CN108776819A (en) * | 2018-06-05 | 2018-11-09 | Oppo广东移动通信有限公司 | A kind of target identification method, mobile terminal and computer readable storage medium |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7925070B2 (en) * | 2004-03-30 | 2011-04-12 | Sysmex Corporation | Method for displaying virtual slide and terminal device for displaying virtual slide |
| EP3136289A1 (en) * | 2015-08-28 | 2017-03-01 | Thomson Licensing | Method and device for classifying an object of an image and corresponding computer program product and computer-readable medium |
| CN107690659B (en) * | 2016-12-27 | 2021-08-10 | 达闼机器人有限公司 | Image recognition system and image recognition method |
-
2019
- 2019-12-28 CN CN201911402913.5A patent/CN111104988B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018054283A1 (en) * | 2016-09-23 | 2018-03-29 | 北京眼神科技有限公司 | Face model training method and device, and face authentication method and device |
| WO2018119599A1 (en) * | 2016-12-26 | 2018-07-05 | 深圳前海达闼云端智能科技有限公司 | Method and device for searching for person and communication system |
| CN207993164U (en) * | 2017-07-24 | 2018-10-19 | 图灵通诺（北京）科技有限公司 | The checkout apparatus of image recognition technology based on convolutional neural networks |
| CN108776819A (en) * | 2018-06-05 | 2018-11-09 | Oppo广东移动通信有限公司 | A kind of target identification method, mobile terminal and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111104988A (en) | 2020-05-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111104988B (en) | Image recognition method and related device | |
| CN110689500B (en) | Face image processing method and device, electronic equipment and storage medium | |
| CN108764487A (en) | For generating the method and apparatus of model, the method and apparatus of information for identification | |
| CN109740668B (en) | Deep model training method and device, electronic equipment and storage medium | |
| CN109002759A (en) | text recognition method, device, mobile terminal and storage medium | |
| US20220051058A1 (en) | Unmanned driving behavior decision-making and model training | |
| CN110517225B (en) | AI image recognition method, apparatus, device and storage medium | |
| CN104867065A (en) | Method and equipment for processing orders | |
| CN112132073B (en) | Garbage classification method and device, storage medium and electronic equipment | |
| CN111160157B (en) | Text extraction method based on DWG drawing and related products | |
| CN112840305B (en) | Font switching method and related product | |
| CN112950443A (en) | Adaptive privacy protection method, system, device and medium based on image sticker | |
| CN110223694B (en) | Voice processing method, system and device | |
| CN112862073B (en) | Compressed data analysis method and device, storage medium and terminal | |
| CN115101069A (en) | Voice control method, device, equipment, storage medium and program product | |
| CN113989559A (en) | Method, device and equipment for determining probability threshold of classification model and storage medium | |
| CN110909797B (en) | Image detection method and device, equipment and storage medium | |
| CN113688975A (en) | Neural network training method and device, electronic equipment and storage medium | |
| CN113505844A (en) | Label generation method, device, equipment, storage medium and program product | |
| CN113919852A (en) | Product infringement judgment method and device, terminal equipment and storage medium | |
| CN112862021A (en) | Content labeling method and related device | |
| CN115226152B (en) | Communication method, system, electronic equipment and storage medium of satellite network | |
| CN117292120B (en) | Light-weight visible light insulator target detection method and system | |
| CN113672332B (en) | Window display method, device, apparatus, storage medium, and program | |
| CN113516183B (en) | Fault response method, system, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |