WO2019153286A1

WO2019153286A1 - Image classification method and device

Info

Publication number: WO2019153286A1
Application number: PCT/CN2018/076081
Authority: WO
Inventors: 孙伟; 谭利文; 杜明亮
Original assignee: 华为技术有限公司
Priority date: 2018-02-09
Filing date: 2018-02-09
Publication date: 2019-08-15
Also published as: CN111566639A

Abstract

The present invention relates to the field of image processing. Provided are an image classification method and device capable of reducing a computation load in an image classification process and improving image classification efficiency. The method comprises: a first device capturing image data, acquiring image generation information upon capturing the image data, and generating a first image file comprising the image data and the image generation information (S501); the first device performing, on the basis of the image generation information, an image classification operation on the first image file (S502); and in response to an operation to view a gallery, the device displaying the first image file in a classification directory of the gallery (S503).

Description

Image classification method and device

Technical field

The embodiments of the present application relate to the field of image processing, and in particular, to an image classification method and device.

Background technique

Image classification is a method in which a device automatically divides a plurality of images into at least two types of images according to feature information of each image in a plurality of images for classification management. In the process of image classification, the amount of calculation required for the device to analyze multiple images to obtain their feature information is large.

Moreover, after one device performs image classification on the image a, the image a is transmitted to another device, and when the other device performs image classification on the image a, it is necessary to perform image analysis on the image a again to acquire the feature of the image a. information. Among them, repeatedly acquiring the feature information of the same image will generate a large amount of redundant calculation.

Summary of the invention

The embodiment of the present application provides an image classification method and device, which can reduce the calculation amount in the image classification process and improve the image classification efficiency.

In a first aspect, an embodiment of the present application provides an image classification method, where the image classification method includes: capturing image data by a device, and acquiring image generation information when capturing image data, and generating a first image file including image data and image generation information. And then performing an image classification operation on the first image file using the image generation information; the device displays the first image file in the catalogue of the gallery in response to the operation for viewing the gallery.

In the image classification method provided by the embodiment of the present application, when the device captures image data, the image generation information when the image data is captured may be acquired, and then the first image file including the image data and the image generation information is generated. In this way, when the device performs an image classification operation on the first image file, the image classification operation can be directly performed using the image generation information. That is, the device can skip the recognition image data to obtain image generation information. In this way, the amount of calculation in the image classification process can be reduced, and the image classification efficiency can be improved.

In a possible design manner of the first aspect, the image generation information may include: information of a shooting parameter, information of a shooting mode, information of a shooting scene, and information of a camera type.

In another possible design manner of the first aspect, the shooting parameter may include a parameter such as an exposure value, the panoramic mode, a normal mode, and the like, and the shooting scene may include a person shooting scene, a building shooting scene, and a natural Scenery shooting scenes, indoor shooting scenes, and outdoor shooting scenes. The classification feature information is feature information obtained by performing an image recognition operation on the image data. The camera type is used to indicate that the image data was captured using a front camera or a rear camera.

In another possible design manner of the first aspect, when the shooting scene is a person shooting scene, the image generation information further includes character feature information. Exemplarily, the character feature information includes information such as the number of faces, face indication information, face location information, indication information of other objects (such as animals), and the number of other objects. The face indication information is used to indicate that the image data of the first image file includes a face or does not include a face; the indication information of the other object is used to indicate that the image data includes other objects or does not include other objects; Used to indicate the position of the face in the image data.

In another possible design manner of the first aspect, when performing the image classification operation, the device may perform an image recognition operation on the image data to analyze the image data to obtain image generation information. That is, the device can skip the image recognition operation on the image data to obtain the image generation information. That is to say, the above image generation information is not obtained by performing an image recognition operation. In this way, the amount of calculation in the image classification process can be reduced, and the image classification efficiency can be improved.

In another possible design manner of the first aspect, the feature information required by the device to perform the image classification operation includes not only the image generation information but also the classification feature information. Specifically, the foregoing apparatus performs image classification operation on the first image file by using image generation information, including: performing, by the device, an image recognition operation on the image data to analyze the image data to obtain classification feature information; and the device uses the image generation information and the classification feature information, An image classification operation is performed on the first image file.

In another possible design manner of the first aspect, after analyzing the image data to obtain the classification feature information, the device may further save the classification feature information in the first image file to obtain the updated first image file. In this way, when the device or other device performs the image classification operation again on the first image file, the feature information (image generation information and classification feature information) stored in the first image file can be directly used without re-recognizing the first The image data of the image file obtains the feature information.

In another possible design manner of the first aspect, the image classification algorithm used by the device to analyze the image data to obtain different feature information is different. The image classification algorithm used by the device to analyze image data to obtain image generation information includes a first algorithm. In this way, the device does not perform an image recognition operation on the image data, and the method for analyzing the image data to obtain the image generation information includes: the device performs an image recognition operation on the image data without using the first algorithm, and analyzes the image data to obtain the image corresponding to the first algorithm. Image generation information.

It should be noted that the first algorithm in the embodiment of the present application may include one or more image classification algorithms. The image generation information in the embodiment of the present application may include feature information of one or more attributes, and the feature information of each attribute corresponds to an image classification algorithm.

In another possible design manner of the first aspect, the device may perform an image recognition operation on the image data by using a second algorithm to analyze the image data to obtain classification feature information corresponding to the second algorithm; and then use the image generation information and The feature information is classified, and an image classification operation is performed on the first image file. The first algorithm is different from the second algorithm.

In another possible design manner of the first aspect, when the device performs an image classification operation on the first image file again, the device may read the feature information (image generation information and classification feature information) saved in the first image file. When the device determines that the first feature information (image generation information and/or classification feature information) is stored in the first image file, the image recognition operation may be performed on the image data without using the third algorithm to analyze the image data. First feature information corresponding to the third algorithm; directly performing image classification operation on the first image file by using the first feature information. That is, the device can skip the recognition of the image data by using the third algorithm to analyze the image data to obtain the first feature information, and directly perform the image classification operation on the first image file by using the first feature information, thereby reducing the calculation amount of performing the image classification operation.

In another possible design manner of the first aspect, the first feature information may not be included in the first image file. When the device determines that the first feature information is not included in the first image file, the device may identify the image data by using a third algorithm to acquire the first feature information, and perform an image classification operation by using the first feature information. Moreover, the device may save the first feature information in the first image file to obtain the updated first image file, so that when the device or other device performs the image classification operation on the first image file again, the device may directly save and save the image file. The first feature information in the first image file does not need to re-use the first algorithm to identify the image data of the first image file.

In another possible design manner of the first aspect, the algorithm version used by the device to perform the image classification operation is continuously updated over time, and the same algorithm of different versions is used to identify the image data of the first image file. Characteristic information is different. Based on this, the classification feature information further includes a version of the image classification algorithm. In this case, even if the first feature information is stored in the first image file, the algorithm version identifying the first feature information and the version of the third algorithm are not necessarily the same. In view of the above, performing the image classification operation by using the first feature information may include: determining, by the device, that the algorithm version identifying the first feature information is the same as the version of the third algorithm; and the device directly performing the image classification operation by using the first feature information, skipping The first image is identified using a third algorithm to obtain first feature information. In this way, the amount of calculation in the image classification process can be reduced, and the image classification efficiency can be improved.

In another possible design manner of the first aspect, the method of the embodiment of the present application further includes: determining, by the device, that the algorithm version that identifies the first feature information is different from the version of the first algorithm; the device uses the third algorithm to identify the image data. And acquiring the first feature information, and performing the image classification operation by using the first feature information; and updating the first feature information saved in the first image file by using the first feature information that is identified, to obtain the updated first image file. In this way, when the device or other device performs the image classification operation on the first image file again, the new first feature information saved in the first image file can be directly utilized without re-using the first algorithm to identify the first Image data of an image file.

In another possible design manner of the first aspect, the format of the first image file is an Exchangeable image file format (EXIF).

In another possible design of the first aspect, the image generation information is saved in a Maker Note field of the first image file. Of course, the above classification feature information is also saved in the Maker Note field of the first image file.

In another possible design of the first aspect, the image generation information is saved in a Maker Note field of the first image file in a Tagged Image File Format (TIFF) format. Of course, the above classification feature information is also saved in the Maker Note field of the first image file using TIFF.

In a second aspect, an embodiment of the present application provides an image classification method, where the image classification method includes: the device captures image data by using a camera; and the device acquires image generation information when capturing image data, and generates a first image including image data and image generation information. An image file; wherein the format of the first image file is EXIF, and the image generation information is saved in a Maker Note field of the first image file; the image generation information includes information of a shooting parameter, information of a shooting mode, information of a shooting scene, and a camera type At least one of the information; the device directly performs an image classification operation on the first image file using the image generation information; instead of performing an image recognition operation on the image data to analyze the image data to obtain image generation information, and then using the image generation information obtained by the analysis An image classification operation is performed on the first image file; finally, in response to the operation for viewing the gallery, the first image file is displayed in the catalog of the gallery.

In the image classification method provided by the embodiment of the present application, when the camera captures image data, the device may acquire image generation information when capturing image data, and then generate a first image file including image data and image generation information. Thus, when the device performs an image classification operation on the first image file, the image classification operation can be directly performed using the image generation information. That is, the device can skip the recognition image data to obtain image generation information. In this way, the amount of calculation in the image classification process can be reduced, and the image classification efficiency can be improved.

In a possible design manner of the second aspect, the foregoing apparatus performs image classification operation on the first image file by using image generation information, including: performing, by the device, an image recognition operation on the image data, to analyze the image data to obtain classification feature information; The device performs an image classification operation on the first image file using the image generation information and the classification feature information.

In another possible design manner of the second aspect, after analyzing the obtained classification feature information, the device may save the classification feature information in the first image file to obtain the updated first image file.

In another possible design manner of the second aspect, the image generation information is saved in a Maker Note field by using TIFF.

In a third aspect, an embodiment of the present application provides an apparatus, where the apparatus includes: an acquiring unit, a classifying unit, and a display unit. An acquisition unit, configured to capture image data, and acquire image generation information when the image data is captured, generate a first image file including the image data and the image generation information; and a classification unit configured to utilize the acquisition unit Obtaining the image generation information, performing an image classification operation on the first image file; and displaying means for displaying the first image file in a classification directory of the gallery in response to the operation for viewing the gallery.

In a possible design manner of the third aspect, the classification unit is further configured to perform an image recognition operation on the image data; wherein the image generation information is not obtained by the classification unit identifying the image data.

In another possible design manner of the third aspect, the foregoing classification unit is specifically configured to perform an image recognition operation on the image data to analyze the image data to obtain classification feature information, and use the image generation information and the classification feature information to The image file performs an image classification operation.

In another possible design manner of the third aspect, the foregoing apparatus further includes: an update unit. The updating unit is configured to perform an image recognition operation on the image data by the classification unit to analyze the image data to obtain the classification feature information, and save the classification feature information in the first image file to obtain the updated first image file.

In a fourth aspect, an embodiment of the present application provides an apparatus, including: a device, a processor, a memory, a camera, and a display; the memory and the display are coupled to the processor, the display is configured to display an image file, and the memory includes non-volatile a storage medium for storing computer program code, the computer program code comprising computer instructions for capturing image data when the processor executes the computer instruction, the processor for acquiring the camera capture Image generation information at the time of image data, generating a first image file including the image data and the image generation information; and performing image classification operation on the first image file using the image generation information. A display for displaying a first image file in a catalogue of the gallery in response to an operation for viewing the gallery.

In a possible design manner of the fourth aspect, the image generation information is not obtained by the processor performing an image recognition operation on the image data.

In another possible design manner of the fourth aspect, the processor is specifically configured to perform an image recognition operation on the image data to analyze the image data to obtain classification feature information, and use the image generation information and the classification feature information to The image file performs an image classification operation.

In another possible design manner of the fourth aspect, the processor is further configured to perform an image recognition operation on the image data to analyze the image data to obtain the classification feature information, and then save the classification feature information in the first image file. Get the updated first image file.

It should be noted that the image generation information, the format of the first image file, the image generation information, and the classification feature described in the second aspect, the third aspect, and the fourth aspect of the embodiments of the present application and any possible design manner thereof For the location of the information in the first image file, the format of the image generation information and the classification feature information in the vendor annotation field, reference may be made to the related description in the first aspect of the design, which is not described herein.

In a fifth aspect, an embodiment of the present application provides a control device, where the control device includes a processor and a memory, where the memory is used to store computer program code, where the computer program code includes computer instructions, and when the processor executes the computer instruction, the control The apparatus performs the method as described in the first and second aspects of the embodiments of the present application and any of its possible design approaches.

In a sixth aspect, the embodiment of the present application provides a computer storage medium, where the computer storage medium includes computer instructions, when the computer instruction is run on a device, causing the device to perform the first aspect of the embodiment of the present application Any of the methods described in the possible design approach.

In a seventh aspect, the embodiment of the present application provides a computer program product, when the computer program product is run on a computer, causing the computer to perform the first aspect and the second aspect, and any one of the embodiments of the present application. The method of design described.

In addition, the third aspect, the fourth aspect, and any one of the design manners, and the technical effects brought by the second aspect, the fourth aspect to the seventh aspect, may refer to the technology brought by different design modes in the foregoing first aspect. The effect will not be described here.

DRAWINGS

1 is a schematic structural diagram of hardware of a mobile phone according to an embodiment of the present application;

2 is a schematic diagram of a data structure of a Joint Photographic Experts Group (JPEG) image file according to an embodiment of the present application;

FIG. 3 is a schematic diagram of a data structure of an EXIF image file according to an embodiment of the present disclosure;

4 is a schematic diagram 1 of a system principle framework of an image classification method according to an embodiment of the present application;

FIG. 5 is a flowchart 1 of an image classification method according to an embodiment of the present application;

6 is a schematic diagram 2 of a system principle framework of an image classification method according to an embodiment of the present application;

FIG. 7A is a schematic diagram of an example of a mobile phone interface according to an embodiment of the present application;

FIG. 7B is a second flowchart of an image classification method according to an embodiment of the present disclosure;

8 is a schematic diagram showing the data structure of a Maker Note field of the EXIF image shown in FIG. 3;

9 is a schematic diagram 1 of a data structure of a TIFF field in the Maker Note field shown in FIG. 8;

10 is a second schematic diagram of a data structure of a TIFF field in the Maker Note field shown in FIG. 8;

FIG. 11 is a schematic diagram 3 of a system principle framework of an image classification method according to an embodiment of the present disclosure;

FIG. 12 is a flowchart 3 of an image classification method according to an embodiment of the present disclosure;

FIG. 13 is a schematic structural diagram 1 of a device according to an embodiment of the present disclosure;

FIG. 14 is a schematic structural diagram 2 of a device according to an embodiment of the present disclosure.

Detailed ways

In the following, the terms "first" and "second" are used for descriptive purposes only, and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, features defining "first" and "second" may include one or more of the features either explicitly or implicitly. For example, the first feature information and the second feature information refer to different feature information in the first image file, and the non-first image file has two feature information.

The embodiment of the present application provides an image classification method, which can be applied to an image classification of a first image file by a device. The image file (such as the first image file) in the embodiment of the present application refers to an image file obtained by encoding and compressing an image captured by the camera, such as a JPEG image file. The image described in the embodiment of the present application can be understood as an electronic picture (hereinafter referred to as a picture).

The image classification in the present application refers to the device according to the feature information of the image data in each image file of the plurality of image files, such as shooting mode (such as panoramic mode) information, shooting scene (such as character scene) information, and face. Information such as the number (such as 3 faces) is divided into at least two types of image files (ie, clustering a plurality of image files).

The devices (such as the first device and the second device) in the embodiment of the present application may be a mobile phone (such as the mobile phone 100 shown in FIG. 1), a tablet computer, a personal computer (PC), and a personal digital assistant. , PDA), netbooks, wearable electronic devices, augmented reality (AR), virtual reality (VR) devices, car computers and other terminal devices.

For example, the device may manage the image saved in the device, and perform the method provided in the embodiment of the present application to perform image classification on the image saved in the device. Alternatively, a client (or an application) for managing a picture may be installed in the device, and the client may manage a picture saved in the cloud server after logging in to a picture management account; and the client may also be used for The method provided in the embodiment of the present application performs image classification on a picture in the cloud server.

Alternatively, the device in the embodiment of the present application may be a cloud server for storing and managing a picture, and the cloud server may receive the picture uploaded by the terminal, and then perform the method provided by the embodiment of the present application to perform image classification on the picture uploaded by the terminal. . The specific form of the above device is not specifically limited in the embodiment of the present application.

As shown in FIG. 1 , the mobile phone 100 is used as the device. The mobile phone 100 may specifically include: a processor 101, a radio frequency (RF) circuit 102, a memory 103, a touch screen 104, a Bluetooth device 105, and one or more sensors. 106, Wireless Fidelity (WiFi) device 107, positioning device 108, audio circuit 109, peripheral interface 110, and power supply device 111 and the like. These components can communicate over one or more communication buses or signal lines (not shown in Figure 1). It will be understood by those skilled in the art that the hardware structure shown in FIG. 1 does not constitute a limitation to a mobile phone, and the mobile phone 100 may include more or less components than those illustrated, or some components may be combined, or different component arrangements.

The various components of the mobile phone 100 will be specifically described below with reference to FIG. 1 :

The processor 101 is a control center of the mobile phone 100, and connects various parts of the mobile phone 100 by using various interfaces and lines, and executes the mobile phone 100 by running or executing an application stored in the memory 103 and calling data stored in the memory 103. Various functions and processing data. In some embodiments, processor 101 can include one or more processing units. In some embodiments of the present application, the processor 101 may further include a fingerprint verification chip for verifying the collected fingerprint.

The radio frequency circuit 102 can be used to receive and transmit wireless signals during transmission or reception of information or calls. In particular, the radio frequency circuit 102 can process the downlink data of the base station and then process it to the processor 101; in addition, transmit the data related to the uplink to the base station. Generally, radio frequency circuits include, but are not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. In addition, the radio frequency circuit 102 can also communicate with other devices through wireless communication. The wireless communication can use any communication standard or protocol including, but not limited to, global mobile communication systems, general packet radio services, code division multiple access, wideband code division multiple access, long term evolution, email, short message service, and the like.

The memory 103 is used to store applications and data, and the processor 101 executes various functions and data processing of the mobile phone 100 by running applications and data stored in the memory 103. The memory 103 mainly includes a storage program area and a storage data area, wherein the storage program area can store an operating system, an application required for at least one function (such as a sound playing function, an image playing function, etc.); the storage data area can be stored according to the use of the mobile phone. Data created at 100 o'clock (such as audio data, phone book, etc.). Further, the memory 103 may include a high speed random access memory (RAM), and may also include a nonvolatile memory such as a magnetic disk storage device, a flash memory device, or other volatile solid state storage device. The memory 103 can store various operating systems, for example,

operating system,

Operating system, etc. The above memory 103 may be independent and connected to the processor 101 via the above communication bus; the memory 103 may also be integrated with the processor 101.

The touch screen 104 may specifically include a touch panel 104-1 and a display 104-2.

The touch panel 104-1 can collect touch events on or near the user of the mobile phone 100 (for example, the user uses any suitable object such as a finger, a stylus, or the like on the touch panel 104-1 or on the touchpad 104. The operation near -1), and the collected touch information is sent to other devices (for example, processor 101). The touch event of the user in the vicinity of the touch panel 104-1 may be referred to as a hovering touch; the hovering touch may mean that the user does not need to directly touch the touchpad in order to select, move or drag a target (eg, an icon, etc.) , and only the user is located near the device to perform the desired function. In addition, the touch panel 104-1 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves.

A display (also referred to as display) 104-2 can be used to display information entered by the user or information provided to the user as well as various menus of the mobile phone 100. The display 104-2 can be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The touchpad 104-1 can be overlaid on the display 104-2, and when the touchpad 104-1 detects a touch event on or near it, it is transmitted to the processor 101 to determine the type of touch event, and then the processor 101 may provide a corresponding visual output on display 104-2 depending on the type of touch event. Although in FIG. 1, the touchpad 104-1 and the display 104-2 are implemented as two separate components to implement the input and output functions of the handset 100, in some embodiments, the touchpad 104- 1 is integrated with the display screen 104-2 to implement the input and output functions of the mobile phone 100. It is to be understood that the touch screen 104 is formed by stacking a plurality of layers of materials. In the embodiment of the present application, only the touch panel (layer) and the display screen (layer) are shown, and other layers are not described in the embodiment of the present application. . In addition, the touch panel 104-1 may be disposed on the front surface of the mobile phone 100 in the form of a full-board, and the display screen 104-2 may also be disposed on the front surface of the mobile phone 100 in the form of a full-board, so that the front of the mobile phone can be borderless. Structure.

In addition, the mobile phone 100 can also have a fingerprint recognition function. For example, the fingerprint reader 112 can be configured on the back of the handset 100 (eg, below the rear camera) or on the front side of the handset 100 (eg, below the touch screen 104). For another example, the fingerprint collection device 112 can be configured in the touch screen 104 to implement the fingerprint recognition function, that is, the fingerprint collection device 112 can be integrated with the touch screen 104 to implement the fingerprint recognition function of the mobile phone 100. In this case, the fingerprint capture device 112 is disposed in the touch screen 104 and may be part of the touch screen 104 or may be otherwise disposed in the touch screen 104. The main component of the fingerprint collection device 112 in the embodiment of the present application is a fingerprint sensor, which can employ any type of sensing technology, including but not limited to optical, capacitive, piezoelectric or ultrasonic sensing technologies.

The mobile phone 100 may also include a Bluetooth device 105 for enabling data exchange between the handset 100 and other short-range devices (eg, mobile phones, smart watches, etc.). The Bluetooth device in the embodiment of the present application may be an integrated circuit or a Bluetooth chip or the like.

The handset 100 can also include at least one type of sensor 106, such as a light sensor, motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display of the touch screen 104 according to the brightness of the ambient light, and the proximity sensor may turn off the power of the display when the mobile phone 100 moves to the ear. . As a kind of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (usually three axes). When it is stationary, it can detect the magnitude and direction of gravity. It can be used to identify the gesture of the mobile phone (such as horizontal and vertical screen switching, related Game, magnetometer attitude calibration), vibration recognition related functions (such as pedometer, tapping), etc. As for the mobile phone 100 can also be configured with gyroscopes, barometers, hygrometers, thermometers, infrared sensors and other sensors, here Let me repeat.

The WiFi device 107 is configured to provide the mobile phone 100 with network access complying with the WiFi-related standard protocol, and the mobile phone 100 can access the WiFi access point through the WiFi device 107, thereby helping the user to send and receive emails, browse web pages, and access streaming media. It provides users with wireless broadband Internet access. In some other embodiments, the WiFi device 107 can also function as a WiFi wireless access point, and can provide WiFi network access for other devices.

The positioning device 108 is configured to provide a geographic location for the mobile phone 100. It can be understood that the positioning device 108 can be specifically a receiver of a positioning system such as a Global Positioning System (GPS) or a Beidou satellite navigation system, or a Russian GLONASS. After receiving the geographical location transmitted by the positioning system, the positioning device 108 sends the information to the processor 101 for processing, or sends it to the memory 103 for storage. In some other embodiments, the positioning device 108 can also be a receiver of an Assisted Global Positioning System (AGPS), which assists the positioning device 108 in performing ranging and positioning services by acting as an auxiliary server. In this case, the secondary location server provides location assistance over a wireless communication network in communication with a location device 108 (i.e., a GPS receiver) of the device, such as handset 100. In still other embodiments, the positioning device 108 can also be a WiFi access point based positioning technology. Since each WiFi access point has a globally unique Media Access Control (MAC) address, the device can scan and collect broadcast signals of surrounding WiFi access points when WiFi is turned on, so that it can be obtained. The MAC address broadcasted to the WiFi access point; the device sends the data (such as the MAC address) capable of indicating the WiFi access point to the location server through the wireless communication network, and the location server retrieves the geographical location of each WiFi access point. And in combination with the strength of the WiFi broadcast signal, the geographic location of the device is calculated and sent to the location device 108 of the device.

The audio circuit 109, the speaker 113, and the microphone 114 can provide an audio interface between the user and the handset 100. The audio circuit 109 can transmit the converted electrical data of the received audio data to the speaker 113 for conversion to the sound signal output by the speaker 113; on the other hand, the microphone 114 converts the collected sound signal into an electrical signal by the audio circuit 109. After receiving, it is converted into audio data, and then the audio data is output to the RF circuit 102 for transmission to, for example, another mobile phone, or the audio data is output to the memory 103 for further processing.

The peripheral interface 110 is used to provide various interfaces for external input/output devices (such as a keyboard, a mouse, an external display, an external memory, a subscriber identity module card, etc.). For example, it is connected to the mouse through a Universal Serial Bus (USB) interface, and is connected to a Subscriber Identification Module (SIM) card provided by the service provider through a metal contact on the card slot of the subscriber identity module. . Peripheral interface 110 can be used to couple the external input/output peripherals described above to processor 101 and memory 103.

In the embodiment of the present invention, the mobile phone 100 can communicate with other devices in the device group through the peripheral interface 110. For example, the peripheral interface 110 can receive display data sent by other devices for display, etc. No restrictions are imposed.

The mobile phone 100 may further include a power supply device 111 (such as a battery and a power management chip) that supplies power to the various components. The battery may be logically connected to the processor 101 through the power management chip to manage charging, discharging, and power management through the power supply device 111. And other functions.

Although not shown in FIG. 1, the mobile phone 100 may further include a camera (front camera and/or rear camera), a flash, a micro projection device, a near field communication (NFC) device, and the like, and details are not described herein.

The execution body of the image classification method provided by the embodiment of the present application may be an image processing device, which is a device that can be used for managing images (such as the mobile phone 100 shown in FIG. 1), or a central processing unit of the device. (Central Processing Unit, CPU), or a control module for image processing in the device, or a client for managing images in the device. In the embodiment of the present application, the image classification method performed by the above device is taken as an example, and the image classification method provided by the present application is described.

The following refers to the terms involved in this application:

(1) JPEG is an international image compression standard.

(2) JPEG File Interchange Format (JFIF) is an image file format standard. JFIF is a JPEG-encoded file format conforming to the JPEG exchange format standard. The image data in the JFIF file is compressed using JPEG; therefore, JFIF is also called "JPEG/JFIF".

Among them, JPEG/JFIF is the most commonly used format for storing and transmitting pictures on the World Wide Web.

As shown in FIG. 2, the JPEG image file (that is, the image file in the JPEG format) starts with the character string "0xFFD8" and ends with the character string "0xFFD9". Among them, the JPEG image file has a series of "0xFF**" format characters in the file header, called "JPEG mark" or "JPEG segment", which is used to mark the information segment of the JPEG image file. Among them, “0xFFD8” is used to mark the beginning of image information, “0xFFD9” is used to mark the end of image information, and there is no information behind these two JPEG identifiers, and other JPEG identifiers will follow some information characters.

(3) The EXIF image file (that is, the image file in the EXIF format) is one of the above JPEG image files, and conforms to the JPEG standard.

The EXIF image adds a shooting parameter (referred to as a first shooting parameter) of the camera-captured image in the JPEG image file. For example, the first shooting parameter may include: shooting date, shooting equipment parameters (such as camera brand and model, lens parameters, flash parameters, etc.), shooting parameters (such as shutter speed, aperture F value, ISO speed, focal length, metering) Mode, etc.), image processing parameters (such as sharpening, contrast, saturation, white balance, etc.) and GPS positioning data of captured images.

Wherein, as shown in FIG. 3, the EXIF image file may include EXIF information, where the EXIF information includes an EXIF field 301 (a field in the EXIF image file for saving the first shooting parameter) and a Maker Note field 302. The parameters are saved in the EXIF field 301. The Maker Note field 302 is a field reserved for the vendor to hold vendor-specific annotation data. The EXIF information starts with the character string "0xFFE0" in the EXIF image and ends with the character string "0xFFEF", which is 64 KB (kilobytes).

The format of the first image file in the embodiment of the present application may be EXIF (ie, the first image file may be an EXIF image file), and the feature information in the first image file may be saved in the Maker Note field of the EXIF image file. The image data field 303 of the EXIF image file is used to save image data. In this way, when the other device or the device performs the image classification operation on the first image file, the feature information in the first image file can be read, and the image is directly executed on the first image file by using the feature information in the first image file. The classification operation; instead of performing an image recognition operation on the image data in the first image file, the feature information for performing the image classification operation is obtained by a large number of calculations. Through this scheme, the amount of calculation in the image classification process can be reduced, and the image classification efficiency can be improved.

Of course, the format of the first image file in the embodiment of the present application includes, but is not limited to, the EXIF. The first image file in the embodiment of the present application may also be another format file including a reserved field, and the reserved field may be used for saving. Feature information for performing image classification operations. Moreover, the foregoing reserved field includes, but is not limited to, the Maker Note field, and the reserved field may be used in the first image file, and may be used to save any field of the feature information for performing the image classification operation. This is not a limitation.

Please refer to FIG. 4 , which is a schematic diagram of a system principle framework of an image classification method provided by an embodiment of the present application. In the embodiment of the present application, the first device may save the feature information of the image data in the first image file during the process of capturing the image and performing the image classification operation, and directly use the image file stored in the image file when performing the image classification operation. Feature information.

Specifically, as shown in FIG. 4, the device may acquire feature information (image generation information in the embodiment of the present application) when the image data is collected by the camera, and save the feature information in the first image file (ie, perform 401). Generating a first image file 403; subsequently, when the image classification engine 402 of the device performs an image classification operation, the feature information (ie, image generation information) in the first image file 403 may be directly read, and the image data of the first image file is identified. Obtaining new feature information (such as classification feature information in the embodiment of the present application); then performing image classification operation on the first image file by using the read feature information and the new feature information; finally, the identified new Feature information, updating feature information in the first image file (eg, adding new feature information).

An image classification method provided by an embodiment of the present application is described in detail below by using specific embodiments.

The embodiment of the present application provides an image classification method. Here, the format of the first image file is EXIF (ie, the first image file is an EXIF image file), and the feature information is stored in the Maker Note field of the EXIF image file as an example, and is implemented in the present application. The image classification method provided by the example is explained. As shown in FIG. 5, the method in this embodiment of the present application includes S501-S503:

S501. The first device captures image data, and acquires image generation information when capturing image data, and generates a first image file including image data and image generation information.

The preset field (such as the Maker Note field) of the first image file (such as the EXIF image file) of the embodiment of the present application may be used to save the feature information of the image data of the first image file. The feature information may include image generation information. The image generation information is feature information acquired by the first device when the camera of the first device captures the image data. For a detailed example of the image generation information, reference may be made to the subsequent description of the embodiments of the present application, and details are not described herein.

In general, a first image file including image data may be generated after the first device captures image data. In the embodiment of the present application, the method for capturing an image file by the first device is different from the method for capturing an image by the device in the conventional solution. Specifically, the first device not only acquires image data captured by the camera, but also acquires image generation information when the image data is captured, and then generates an image file including the image data and the image generation information.

Exemplarily, the image generation information may include a shooting parameter (referred to as a second shooting parameter) when the camera captures image data. For example, the second shooting parameters described above may include information of a shooting mode (such as a panoramic mode and a normal mode), a shooting scene (such as a human shooting scene, a building shooting scene, a natural scenery shooting scene, an indoor shooting scene, and an outdoor shooting scene, etc.) Information and camera type (camera type indicates that the image data is captured by the front camera or the rear camera). The first device may determine information such as the shooting mode information, the shooting scene information, and the camera type, in response to the selection of the shooting mode, the shooting scene, and the front camera or the rear camera when the camera captures the image data. The normal mode in the embodiment of the present application refers to a mode in which a photo is taken using a rear camera.

Wherein, when the shooting scene is a person shooting scene, the image generation information further includes character feature information, where the character feature information includes a number of faces, face indication information, face position information, and indications of other objects (such as animals). Information such as the number of information and other objects. The face indication information is used to indicate that the image data of the first image file includes a face or does not include a face; the indication information of the other object is used to indicate that the image data includes other objects or does not include other objects; Used to indicate the position of the face in the image data.

It should be noted that the second shooting parameter in the embodiment of the present application is different from the first shooting parameter saved in the EXIF field 301 of the EXIF image shown in FIG. 3. In general, when the camera captures an image, the first shooting parameter can only be saved in the image without recording the second shooting parameter in the embodiment of the present application; and the second shooting parameter can be used to perform image classification on the image. operating. Thus, when performing an image classification operation on an image, it is necessary to identify the image to obtain a second shooting parameter, which causes a problem that the amount of calculation of the image classification operation is large. In the embodiment of the present application, the first device may generate a first image file including the image data and the second shooting parameter when the image data of the first image file is captured. In this way, when the image classification operation is performed on the image, the image classification operation can be directly read from the first image file and performed using the second imaging parameter, and the calculation amount when the image classification operation is performed on the image can be reduced, thereby improving the image. Classification efficiency. Of course, the first image parameter (such as an EXIF image file) in the embodiment of the present application may also include the foregoing first parameter. The first shooting parameter is saved in the EXIF field of the EXIF image file.

Please refer to FIG. 6 , which is a schematic diagram showing the principle of generating an image file including feature information of image data when an image is captured according to an embodiment of the present application. As shown in FIG. 6, the camera engine can call an algorithm such as a scene algorithm and a face algorithm when the camera captures an image (ie, 61), and recognizes a user operation when the camera captures image data (ie, 62) to collect image generation of the image. Information (ie 63); then the collected image generation information and the image captured by the camera are passed to the JPEG creator 64; the image generation information from 63 is packed into a byte array by the MakerNote Maker in the JPEG Maker 64 (referred to as The feature byte array), the image from 61 is packed into a byte array by the EXIF maker in the JPEG maker 64; then, the image file including the image data and the above-described feature byte array is generated by the JPEG maker 64 (ie, An image file).

Optionally, the first device may periodically perform the following S502 to perform image classification on the plurality of image files including the first image file.

Exemplarily, it is assumed that the first device is the mobile phone 100 shown in (a) of FIG. 7A. As shown in (a) of FIG. 7A, the photo album of the mobile phone 100 includes photos 1 to 8 and the like, and the first image file is any photo in the album of the mobile phone 100, such as the first image file is FIG. 7A. Photo 1 shown in (a). The mobile phone 100 can periodically perform image classification on photos in the album of the mobile phone 100. Thus, the mobile phone 100 can display the album interface shown in (b) of FIG. 7A, in which the mobile phone 100 displays the result of image classification of the photos in the album. For example, the mobile phone 100 divides the photos 1 - 8 into "person" album b, "animal" album a, and "landscape" album c. Wherein, the "person" album b includes the photo 1, the photo 3, the photo 5, and the photo 8 shown in (a) of FIG. 7A, and the "animal" photo album a includes the one shown in (a) of FIG. 7A. Photo 2 and Photo 7, "Landscape" album c includes Photo 4 and Photo 6 shown in (a) of Fig. 7A. For example, taking the "person" album b as an example, when the user clicks on the "person" album b shown in (b) of FIG. 7A, the mobile phone 100 can display the "person" shown in (c) of FIG. 7A. In the album interface, the "People" album interface includes Photo 1, Photo 3, Photo 5, and Photo 8.

Alternatively, the first device may perform image classification on the pictures in the album of the first device by performing S502 in response to the user operation. Based on the example shown in FIG. 7A, the mobile phone 100 can also perform image classification on the photos 1 - 8 in response to the user's click operation on the "alliance" button shown in (a) of FIG. 7A, and then display the image in FIG. 7A. The album interface shown in (b).

S502. The first device performs image classification operation on the first image file by using image generation information.

In the embodiment of the present application, the image generation information is not obtained by the first device performing an image recognition operation on the image data, that is, the first device does not perform an image recognition operation on the image data in order to obtain the image generation information. That is, the first device may skip the following steps: performing an image recognition operation on the image data, analyzing the image data to obtain image generation information, and directly using the saved image generation information in the first image file to perform execution on the first image file. Image classification operation.

It can be understood that the image classification algorithm adopted by the first device to analyze the image data to obtain different feature information is different. The image classification algorithm used by the first device to analyze the image data to obtain the image generation information includes the first algorithm. In this way, the first device does not perform an image recognition operation on the image data by using the first algorithm, and analyzes the image data to obtain image generation information corresponding to the first algorithm.

S503. The device displays the first image file in a category directory of the gallery in response to the operation for viewing the gallery.

The image classification directory in the embodiment of the present application may display the first image file according to the classification result obtained by executing S502.

Exemplarily, based on the example shown in FIG. 7A, the mobile phone 100 may also display the photo 1 - photo 8 in a responsive manner to the user's click operation on the "alliance" button shown in (a) of FIG. 7A. The album interface shown in (b) of 7A (ie, the catalogue of the gallery).

Optionally, the foregoing operation for querying the gallery may be that the user inputs a keyword in a search box of the gallery, and the device may input the keyword in the search box of the gallery in response to the user, and display the first image file in the category directory of the gallery. Multiple image files that match the keywords entered by the user.

Exemplarily, taking the photo 1 - photo 8 shown in (a) of FIG. 7A in the mobile phone 100 as an example, the mobile phone 100 performs the image classification operation on the image file (such as photo 1 - photo 8) by executing the above S501-S502. . When the user enters the keyword "person" in the search box of the gallery, the mobile phone 100 can display the character image files of the photo 1, the photo 3, the photo 5, and the photo 8 in the catalogue of the gallery. When the user inputs the keyword "two people" in the search box of the gallery, the mobile phone 100 can display the character image files of the photo 3 and the photo 5 in the catalogue of the gallery.

In the image classification method provided by the embodiment of the present application, the image file captured by the first device includes not only image data, but also image generation information. In this way, the first device can directly perform image classification operation on the first image file by using the image generation information; without performing image recognition operation on the image data to analyze the image data to obtain image generation information, the image classification process can be reduced. The amount of calculation can further improve the efficiency of image classification.

Further, the feature information required for the first device to perform the image classification operation includes not only the image generation information but also the classification feature information (the feature information obtained by performing the image recognition operation on the image data, and the classification feature information is different from the image generation information) Therefore, before the first device performs an image classification operation on the first image file, the first device may further perform an image recognition operation on the image data to analyze the image data to obtain classification feature information; and then use the image generation information and the classification feature information. And performing an image classification operation on the first image file. Specifically, as shown in FIG. 7B, the foregoing S502 may include S701-S702:

S701. The first device performs an image recognition operation on the image data to analyze the image data to obtain classification feature information.

The first device may perform image recognition operations on the image data by using different image classification algorithms (such as the second algorithm) different from the foregoing first algorithm to analyze the image data to obtain classification feature information corresponding to the second algorithm. The second algorithm is different from the first algorithm described above, and the classification feature information is different from the image generation information. The method for performing the image recognition operation on the image data by the first device to obtain the classification feature information by using the second algorithm may refer to the method for performing the image recognition operation on the image data to obtain the classification feature information in the conventional technology, which is not described herein. .

S702. The first device performs image classification operation on the first image file by using image generation information and classification feature information.

The method for performing an image classification operation on the first image file by using the image generation information and the classification feature information by the first device may refer to a method for performing an image classification operation on the image file according to the feature information of the image file in the conventional technology. The examples are not described here.

Further, as shown in FIG. 7B, after S701, the method in the embodiment of the present application further includes S703:

S703. The first device saves the classification feature information in the first image file to obtain the updated first image file.

It can be understood that, after S703, the image generation information and the classification feature information are included in the first image file. In the embodiment of the present application, the image generation information and the classification feature information are collectively referred to as feature information of the first image file. The feature information (image generation information and classification feature information) in the embodiment of the present application is saved in a preset field of the first image file. In the embodiment of the present application, the Maker Note field 302 shown in FIG. 3 is used as an example to describe the format of the preset field and the manner in which the feature information is saved in the preset field:

Exemplarily, the feature information in the embodiment of the present application may be saved in the Maker Note field by using TIFF. The data format for storing the feature information in the Maker Note field 302 includes, but is not limited to, TIFF. Other data formats for storing the feature information in the Maker Note field 302 are not described herein.

As shown in FIG. 3, the Maker Note field 302 includes an information header 302a, a check field 302b, a Tagged Image File Format (TIFF) header 302c, and a TIFF field 302d.

Among them, as shown in FIG. 8, the information header 302a is used to store vendor information. For example, "huawei" may be saved in the information header 302a; the verification field 302b is used to store verification information, which identifies the integrity and accuracy of the information held in the Maker Note field 302, for example, the verification information may It is a Cyclic Redundancy Check (CRC). The TIFF header 302c is configured to save TIFF indication information for indicating that the format of the feature information stored in the TIFF field 302d is an Image File Directory (IFD) format. The TIFF field 302d is used to hold feature information such as image generation information and classification feature information.

In the embodiment of the present application, the device performs an image classification operation on the image, and after obtaining new feature information (ie, classification feature information), the feature information saved in the Maker Note field 302 can be updated (ie, the Maker Note field 302 is modified). In this case, in order to avoid that the verification information held by the check field 302b indicates that the Maker Note field 302 has been tampered with due to the device updating the feature information held in the Maker Note field 302; the device may be saved in the update Maker Note field 302. At the time of the feature information, new check information is generated for the check field 302b.

Please refer to FIG. 8 , which is a schematic diagram showing an example of a data structure of a Maker Note field provided by an embodiment of the present application. The format in which the feature information is saved in the TIFF field 302d in FIG. 8 is the IFD format.

Specifically, one or more IFDs may be saved in the TIFF field 302d. For example, as shown in FIG. 8, IFD0 and IFD1 are included in the TIFF field 302d. Taking IFD0 as an example, the IFD information in the TIFF field 302d is described:

IFD0 includes a directory field and a data field, and the directory field of IFD0 is used to store a directory of sub-IFDs (such as sub-IFD1 and sub-IFD2) in IFD0 and a connection end tag of the IFD0. The data field of IFD0 is used to store sub-IFDs (such as sub-IFD1 and sub-IFD2, etc.). The connection end tag of IFD0 is used to indicate the location where IFD0 ends. The feature information in the embodiment of the present application may be saved in a directory or may be saved in a data domain.

Optionally, each sub-IFD may also include a directory domain and a data domain. For example, the sub-IFD1 in IFD0 includes a directory domain and a data domain. For the function of the directory domain and the data domain of the sub-IFD1, reference may be made to the description of the directory domain and the data domain of the IFD in the embodiment of the present application.

It is assumed that three IFDs (such as IFD1-IFD3) are included in the TIFF field 302d, and IFD-0IFD2 does not include the sub-IFD. Please refer to FIG. 9, which shows a schematic structural diagram of a directory of the IFD shown in FIG. As shown in FIG. 9, the directory of the IFD includes a plurality of tag items, and each tag item includes a tag identifier (Identity, ID), a tag type, and a tag value.

The tag value in the embodiment of the present application may be feature information; or, the feature information is stored in a data domain of the IFD, and the tag value is an address offset of the feature information in the data domain. For example, when a feature information is less than or equal to 4 bytes, the tag value is the feature information; when a feature information is greater than 4 bytes, the feature information needs to be saved in the data domain, and the tag value is the feature information in the data. The address offset of the domain.

For example, taking four tag entries in IFD0 as an example, the tag IDs of the three tag entries are 0x001, 0x002, 0x003, and 0x004, respectively. The tag type corresponding to the tag ID 0x001 is Unsigned long, and the tag value is used to indicate the shooting mode information of the image (for example, when the tag value is 0, the image is taken in the self-timer mode; when the tag value is 1, Indicates that the image was taken in panorama mode). The tag type corresponding to the tag ID0x002 is Unsigned byte, and its tag value is used to indicate the camera type of the image (for example, when the tag value is 0, it indicates that the image was taken using the rear camera; when the tag value is 1, the image is Shot with the front camera). The label type corresponding to the label ID0x003 is Undefined, and the label value is used to indicate the face indication information (for example, when the label value is 0, it means that there is no face in the image; when the label value is 1, it indicates a human face in the image). The tag type corresponding to tag ID 0x004 is Unsigned byte, and its tag value is address offset 1, which is the address offset of the face location information in the data field of IFD0.

The feature information that the first device needs to use when performing the image classification operation on the first image file may include the feature information of the preset multiple attributes. In general, the first device may use different algorithms to identify image data in the first image file to obtain feature information of the corresponding attribute. The "preset multiple attributes" in the embodiment of the present application is determined by an image classification client (referred to as a client) in the first device. Specifically, the “preset multiple attributes” is determined by the attribute of the feature information that needs to be identified when the image classification client in the first device performs an image classification operation on the image.

For example, assume that the attributes of the feature information that the client of the first device needs to recognize when performing an image classification operation on the image file include: a face attribute, a scene attribute, and a mode attribute. The face attribute corresponds to the number of faces and the face indication information, and the scene attribute corresponds to the shooting scene information, and the mode attribute corresponds to the shooting mode information. Then, the preset plurality of attributes include a face attribute, a scene attribute, and a mode attribute. The feature information of each attribute corresponds to an image classification algorithm. For example, the feature information of the face attribute corresponds to the face algorithm.

When the feature information of an attribute is more complicated or more, in order to facilitate the extraction and storage of the feature information of the attribute, the feature information of the attribute may be saved in the sub-IFD of an IFD. For example, the feature information of the face attribute may include: a version of the face algorithm, a number of faces, and face position information. Assume that IFD0 includes three sub-IFDs (such as sub-IFD1-sub-IFD3).

Please refer to FIG. 10, which shows the structure of the directory of the sub-IFD in the IFD0 shown in FIG. As shown in FIG. 10, the directory of the sub-IFD in the IFD includes a plurality of tag items, and each tag item includes a tag ID, a tag type, and a tag value. Taking the sub-IFD1 of IFD0 as an example, it is assumed that the sub-IFD1 of IFD0 includes three tag items, and the tag IDs of the three tag items are 0x001, 0x002, and 0x003, respectively. The tag type corresponding to the tag ID 0x001 is Unsigned long, and the tag value is used to indicate the version of the face algorithm; the tag type corresponding to the tag ID 0x002 is Unsigned long, and the tag value is used to indicate the number of faces. The tag type corresponding to the tag ID 0x003 is Unsigned byte, and the tag value is the address offset 2, which is the address offset of the face location information in the data field of the sub-IFD1 of the IFD0.

It should be noted that the tag IDs in the tag entries in different IFDs in this embodiment may be the same. Specifically, since the identifiers (such as IDs) of different IFDs are different, even if the label items in the two IFDs use the same label ID, the device can distinguish the label items in different IFDs. Also, the tag IDs in the tag entries in different sub-IFDs may be the same. Specifically, since the identifiers (such as IDs) of different sub-IFDs are different, even if the label items in the two sub-IFDs use the same label ID, the device can distinguish the label items in different sub-IFDs. Moreover, the tag types in the embodiments of the present application include, but are not limited to, the Unsigned long, the Unsigned byte, and the Undefined.

It can be understood that when the first device adds new feature information in the TIFF field 302d, a tag item, an IFD or a sub-IFD may be added in the TIFF field 302d for saving the feature information of the new attribute. For example, IFD2 is added to the TIFF field 302d shown in FIG.

Please refer to FIG. 11 , which is a schematic diagram of a principle for performing an image classification operation according to an embodiment of the present application. As shown in FIG. 11, when the device performs an image classification operation on an image file, the image classification engine may first read the feature information saved in the Maker Note field of the image file, and after reading the feature information, the image is parsed by Maker Note. And the EXIF parser parsing the read feature information (ie, 1101); for the feature information of the partial attribute (ie, the feature information not saved in the Maker Note field), the image classification algorithm (ie, 1102) may be used to identify the image data to obtain the Feature information of the attribute (ie, new feature information 1104); then the image classification operation (ie, 1103) may be performed on the image using the feature information read by the execution 1101 and the new feature information (ie, 1104); and the image classification engine further The new feature information 1104 can be used to update the feature information (ie, 1105) stored in the Maker Note field of the image file to obtain an updated image file.

It can be understood that the first device may acquire the image file before performing the image classification operation on the image file. Specifically, as shown in FIG. 12, the method in this embodiment of the present application includes S1200:

S1200. The first device acquires the first image file.

The manner in which the first device acquires the first image file includes, but is not limited to, the manner shown in the above S501, and the first device may further receive the image file sent by the second device. That is, S1200 may include S1200a: the first device receives the first image file sent by the second device. Wherein, in different implementation manners (implementation manners a-d), the first image file received by the first device from the second device is different:

Implementation mode a: The second device captures the first image file by capturing an image file as shown in S501. The first device receives image generation information from the first image file of the second device.

Implementation b: the second device captures the first image file by using the captured image file shown in S501; and the second device performs the image classification operation on the first image file by performing the image classification method provided by the embodiment of the present application. . After the second device performs the image classification operation on the first image file, the classification feature information of the image data in the first image file is saved in the first image file. That is, the first device receives the image generation information and the classification feature information from the first image file of the second device.

Implementation c: The second device does not have the function of capturing an image file in the manner shown in S501, and the first device does not include the image generation information in the first image file received from the second device, and does not include the classification feature information.

Implementation d: The second device does not have the function of capturing the image file in the manner shown in S501; however, the second device performs the image classification operation on the first image file by performing the image classification method provided by the embodiment of the present application. After the second device performs an image classification operation on the first image file, the second image file includes classification feature information, and does not include image generation information.

After S502, when the first device performs an image classification operation on the first image file again, or when the first device performs an image classification operation on the first image file received from the second device, the first device may first read Feature information saved in the first image file; when it is determined that the first feature information (image generation information and/or classification feature information) is stored in the first image file, the image recognition operation may be performed on the image data without using the third algorithm And analyzing the image data to obtain first feature information corresponding to the third algorithm; and directly performing image classification operation on the first image file by using the first feature information. That is, the device can skip the recognition of the image data by using the third algorithm to analyze the image data to obtain the first feature information, and directly perform the image classification operation on the first image file by using the first feature information, thereby reducing the calculation amount of performing the image classification operation. Specifically, after S502, or after S1200a, the method in the embodiment of the present application further includes S1201-S1205. For example, as shown in FIG. 12, after S1200, the method of the embodiment of the present application further includes S1201-S1205:

S1201: The first device acquires feature information in the first image file.

The feature information (such as image generation information and classification feature information) in the embodiment of the present application is saved in a preset field of the first image file (such as a Maker Note field). For a specific storage manner of the feature information in the image file, reference may be made to the detailed description of the embodiment of the present application in FIG. 8 to FIG. 10, and details are not described herein again.

S1202: The first device determines whether the first feature information is included in the first image file.

Specifically, when the first device determines that the first feature information is included in the first image file, that is, the first feature information is saved in a preset field (such as a Maker Note field) of the first image file, the first device may not adopt the first feature information. The three algorithms recognize the image data to obtain the first feature information (ie, skip S1203), and directly perform the image classification operation using the first feature information (ie, perform S1205).

When the first device determines that the first feature information is not included in the first image file, that is, the first feature information is not saved in the preset field (such as the Maker Note field) of the first image file, the first device may adopt the third algorithm. The image data in the first image file is identified to obtain the first feature information (ie, execution S1203), and then the image classification operation is performed using the first feature information obtained in S1203 (ie, execution S1205).

The first feature information may be feature information of the first attribute, and the third algorithm is an image classification algorithm for identifying feature information of the first attribute. Exemplarily, it is assumed that the feature information of an image can be divided into the feature information of the attribute a and the feature information of the attribute b according to the attribute of the feature information; then the plurality of preset attributes may include: the attribute a and the attribute b. For example, the feature information of one image may be classified into feature information of a shooting attribute and feature information of a face attribute according to attributes of the feature information.

When the feature information of the attribute a can be divided into the feature information of the attribute a-sub-attribute 1 and the feature information of the attribute a-sub-attribute 2, the preset attribute may include: attribute a-sub Attribute 1, attribute a-sub attribute 2 and attribute b. For example, the feature information of the shooting attribute may include shooting mode information and shooting scene information, etc.; the feature information of the face attribute may be divided into: face indication information, a version of the face algorithm, and a number of faces.

The Maker Note field of the first image file may save the feature information of the image data according to different attributes. For example, as shown in FIG. 8, the feature information of one attribute is stored in one IFD, and the attribute information of the feature information stored in different IFDs is different.

For example, the IFD0 shown in FIG. 8 can be used to save feature information of shooting attributes such as the above-described shooting mode information and shooting scene information, and the shooting mode information is saved in the sub-IFD1 in the IFD0, and the shooting scene information is saved in the sub-IFD2 in the IFD0; The IFD1 shown in 8 can be used to save the feature information of the face attribute (the version of the face algorithm, the number of faces and the face position information, etc.), and the version of the face algorithm is saved in the child IFD1 in the IFD1, in the IFD1. The number of faces is saved in the child IFD2, and the face position information is saved in the child IFD3 in the IFD1.

The attributes of the feature information saved by each IFD can be pre-agreed. For example, each IFD saves the feature information of the tag corresponding to an attribute. Among them, multiple tags can be saved in each IFD, and each tag includes a tag ID, a tag type, and a tag value.

When the feature information of the first attribute is not saved in the Maker Note field (the TIFF field in the Maker Note field), the tag corresponding to the first attribute may not be included in the IFD of the TIFF field. Of course, in the embodiment of the present application, the feature information of the first attribute may not be saved by setting the label value of the feature information of the first attribute to be empty (such as Null).

S1203. The first device uses a third algorithm to identify image data in the first image file to obtain first feature information.

The method for identifying the image data of the first image file by using the image classification algorithm (such as the third algorithm) to obtain the first feature information may refer to the image classification operation when the device performs the image classification operation on the image in the conventional technology. The method for identifying the feature information of the image data by the algorithm is not described herein again in the embodiment of the present application.

S1204. The first device saves the first feature information in the first image file to obtain the updated first image file.

The attribute of the feature information saved by each IFD may be pre-agreed. For example, the tag ID of each IFD corresponds to the feature information of one attribute; therefore, the first device may save the feature information of the first attribute in the Maker Note. In the TIFF field in the field, the tag value corresponding to the tag ID of the first attribute.

S1205. The first device performs an image classification operation on the first image file by using the first feature information.

The method for performing an image classification operation on the first image file by using the first feature information by the first device may refer to a method for performing an image classification operation on the image file according to the feature information of the image file in the conventional technology. Let me repeat. For the principle of performing the image classification operation provided by the embodiment of the present application, reference may be made to the schematic diagram shown in FIG. 11 , which is not repeatedly described herein.

In the image classification method provided by the embodiment of the present application, when performing the image classification operation on the first image file, the first device may first acquire the feature information in the first image file, and determine whether the first feature information is included in the first image file; When the first feature information is included in the first image file, the image data of the first image file may be skipped by using the third algorithm to obtain the first feature information. In this way, the amount of calculation in the image classification process can be reduced, and the image classification efficiency can be improved.

It can be understood that the version of the algorithm used by the device to perform the image classification operation is continuously updated with time, and the feature information obtained by using the same algorithm of different versions to identify the image data of the first image file is different. Based on this, the classification feature information further includes a version of the image classification algorithm. In this case, even if the first feature information is stored in the preset field, the algorithm version identifying the first feature information is not necessarily the same as the version of the third algorithm. In view of the situation, after the S1202, when the first image information is included in the first image file, the method of the embodiment of the present application further includes S1301, and the S1204 may be replaced by S1204a:

S1301: The first device determines whether the algorithm version identifying the first feature information is the same as the version of the third algorithm.

Specifically, when the algorithm version identifying the first feature information is the same as the version of the third algorithm, the first device may skip using the third algorithm to identify the image data in the first image file to obtain the first feature information (ie, skipping S1203), performing an image classification operation on the first image file directly by using the first feature information (ie, executing S1205).

When the algorithm version identifying the first feature information is different from the version of the third algorithm, the first device may identify the image data by using the third algorithm to obtain the first feature information (ie, execute S1203), and then use the method obtained by executing S1203. A feature information performs an image classification operation on the first image file (ie, execution S1205).

S1204a: The first device uses the first feature information to update the saved first feature information in the first image file to obtain the updated first image file.

It can be understood that, when the first feature information is not included in the first image file, the method for saving the feature information in the first image file by the first device is: the first device adds the first feature to the preset field of the first image file. information.

When the first feature information is included in the first image file, but the algorithm version identifying the first feature information is different from the version of the third algorithm, the method for the first device to update the feature information saved in the first image file is: the first device The saved first feature information in the preset field of the first image file is replaced with the identified first feature information.

Further, the algorithm version for identifying the first feature information is different from the version of the third algorithm, and may be divided into two cases: (1) the algorithm version identifying the first feature information is lower than the version of the third algorithm; and (2) identifying the first The algorithm version of a feature information is higher than the version of the third algorithm. Specifically, after the S1301, when the algorithm version of the first feature information is different from the version of the third algorithm, the method of the embodiment of the present application further includes S1401:

S1401: The first device determines whether an algorithm version that identifies the first feature information is lower than a version of the third algorithm.

Specifically, when the algorithm version identifying the first feature information is lower than the version of the third algorithm, the first device may use the third algorithm with a higher version to identify the image data in the first image file to obtain the feature of the first attribute. The information (ie, S1203 is performed), and then the image classification operation is performed on the first image file using the first feature information obtained in S1203 (ie, S1205 is performed).

When the algorithm version of the first feature information is higher than the version of the third algorithm, the first device does not need to use the third version of the lower algorithm to identify the image data to obtain the feature information of the first attribute, and may skip the third algorithm. The image data is identified to obtain the feature information of the first attribute (ie, skipping S1203), and the image classification operation is performed on the first image file directly using the first feature information (ie, execution S1205).

In the image classification method provided by the embodiment of the present application, after the algorithm version used in the image classification operation is updated, the first device may update the feature information saved in the preset field by using the feature information identified by the version updated algorithm. In this way, when the image classification operation is performed on the image again, the updated feature information saved in the preset field can be directly used, the calculation amount in the image classification process can be reduced, and the image classification efficiency can be improved.

It can be understood that, in order to implement the above functions, the first device and the second device and the like described above include hardware structures and/or software modules corresponding to each function. Those skilled in the art will readily appreciate that the embodiments of the present application can be implemented in a combination of hardware or hardware and computer software in combination with the elements and algorithm steps of the various examples described in the embodiments disclosed herein. Whether a function is implemented in hardware or computer software to drive hardware depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods to implement the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the embodiments of the present application.

The embodiment of the present application may perform the division of the function module on the first device according to the foregoing method example. For example, each function module may be divided according to each function, or two or more functions may be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of the module in the embodiment of the present application is schematic, and is only a logical function division, and the actual implementation may have another division manner.

As shown in FIG. 13 , the device 1300 is the first device in the foregoing method embodiment. The device 1300 includes an acquisition unit 1301, a classification unit 1302, and a display unit 1303.

The obtaining unit 1301 is configured to support the device 1300 to perform S501, S1200, S1201 in the foregoing method embodiments, and/or other processes for the techniques described herein. The foregoing classification unit 1302 is configured to support the device 1300 to perform S502, S701, S702, S1203, S1205 in the foregoing method embodiments, and/or other processes for the techniques described herein. The above display unit 1303 is used to support the device 1300 to perform S503 in the above method embodiments, and/or other processes for the techniques described herein.

Further, the above device 1300 further includes an update unit. The update unit is configured to support the device 1300 to perform S703, S1204, S1204a in the above method embodiments, and/or other processes for the techniques described herein.

Further, the above device 1300 further includes a determining unit. The determining unit is configured to support the device 1300 to perform S1202, S1301, S1401 in the above method embodiments, and/or other processes for the techniques described herein.

Of course, the above device 1300 may also include other unit modules. For example, the above device 1300 further includes: a storage unit. The storage unit is for saving the first image file. Alternatively, the first image file may be saved in a cloud server, and the device 1300 may perform an image classification operation on the image file in the cloud server. The above device 1300 may further include: a transceiver unit. The device 1300 can interact with other devices through a transceiver unit. For example, the device 1300 can transmit an image file to other devices through the transceiver unit, or receive an image file sent by other devices.

In the case of the integrated unit, the obtaining unit 1301 and the classifying unit 1302 and the like may be integrated into one processing module. The transceiver unit may be an RF circuit, a WiFi module or a Bluetooth module of the device 1300, and the storage unit may be the device 1300. The display unit 1303 may be a display module such as a display (touch screen).

FIG. 14 is a schematic diagram showing a possible structure of a terminal involved in the above embodiment. The device 1400 includes a processing module 1401, a storage module 1402, and a display module 1403.

The processing module 1401 is configured to perform control management on the device 1400. The display module 1403 is configured to display classification results of image files and image files. The storage module 1402 is configured to save program code and data of the device 1400. The device 1400 described above may also include a communication module 1404 for communicating with other devices. For example, the communication module 1404 is used to receive or send a message or image file to other devices.

The processing module 1401 may be a processor or a controller, and may be, for example, a CPU, a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), and field programmable. Field Programmable Gate Array (FPGA) or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It is possible to implement or carry out the various illustrative logical blocks, modules and circuits described in connection with the present disclosure. The processor may also be a combination of computing functions, for example, including one or more microprocessor combinations, a combination of a DSP and a microprocessor, and the like. The communication module 1404 can be a transceiver, a transceiver circuit, a communication interface, or the like. The storage module 1402 can be a memory.

When the processing module 1401 is a processor (such as the processor 101 shown in FIG. 1), the communication module 1404 is a radio frequency circuit (such as the radio frequency circuit 102 shown in FIG. 1), and the storage module 1402 is a memory (such as the memory shown in FIG. 1). 103), when the display module 1403 is a touch screen (including the touch panel 104-1 and the display panel 104-2 shown in FIG. 1 ), the device provided by the present application may be the mobile phone 100 shown in FIG. 1 . The 1404 may include not only a radio frequency circuit but also a WiFi module and a Bluetooth module. The communication modules such as the radio frequency circuit, the WiFi module, and the Bluetooth module may be collectively referred to as a communication interface, wherein the processor, the communication interface, the touch screen, and the memory may be coupled through a bus. together.

The embodiment of the present application further provides a control device, including a processor and a memory, where the memory is used to store computer program code, where the computer program code includes computer instructions, when the processor executes the computer instruction, The image classification method described in the above method embodiment.

The embodiment of the present application further provides a computer storage medium, where the computer program code is stored, and when the processor executes the computer program code, the device performs the related method steps in FIG. 5 or FIG. 12 to implement the foregoing embodiment. The method in .

The embodiment of the present application further provides a computer program product, when the computer program product is run on a computer, causing the computer to perform the related method steps in FIG. 5 or FIG. 12 to implement the method in the foregoing embodiment.

The device 1300 and the device 1400, the computer storage medium or the computer program product provided by the present application are all used to perform the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can be referred to the corresponding ones provided above. The beneficial effects in the method are not described here.

Through the description of the above embodiments, those skilled in the art can clearly understand that for the convenience and brevity of the description, only the division of the above functional modules is illustrated. In practical applications, the above functions can be allocated according to needs. It is completed by different functional modules, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. For the specific working process of the system, the device and the unit described above, reference may be made to the corresponding process in the foregoing method embodiments, and details are not described herein again.

In the several embodiments provided by the present application, it should be understood that the disclosed system, apparatus, and method may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be used. Combinations can be integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application, in essence or the contribution to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium. A number of instructions are included to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to perform all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: a flash memory, a mobile hard disk, a read only memory, a random access memory, a magnetic disk, or an optical disk, and the like, which can store program codes.

The foregoing is only a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto, and any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application. It should be covered by the scope of protection of this application. Therefore, the scope of protection of the present application should be determined by the scope of the claims.

Claims

An image classification method, comprising:

The device captures image data, and acquires image generation information when the image data is captured, and generates a first image file including the image data and the image generation information;

The device performs an image classification operation on the first image file by using the image generation information;

The device displays the first image file in a catalogue of the gallery in response to an operation for viewing a gallery.
The method according to claim 1, wherein the image generation information includes at least one of information of a shooting parameter, information of a shooting mode, information of a shooting scene, and information of a camera type.
The method according to claim 2, wherein the photographing parameter comprises an exposure value, the photographing mode comprises a panorama mode and a normal mode, the photographing scene including a person photographing scene, a building photographing scene, and a natural scenery photographing scene The camera type is used to indicate that the image data is captured by a front camera or a rear camera;

Wherein, when the shooting scene is a scene shot by the character, the image generation information further includes character feature information.
The method according to any one of claims 1 to 3, wherein the image generation information is not obtained by performing an image recognition operation.
The method according to any one of claims 1 to 4, wherein the device performs an image classification operation on the first image file by using the image generation information, including:

Performing an image recognition operation on the image data to analyze the image data to obtain classification feature information;

The device performs an image classification operation on the first image file using the image generation information and the classification feature information.
The method according to claim 5, wherein after the device performs an image recognition operation on the image data to analyze the image data to obtain classification feature information, the method further includes:

The device saves the classification feature information in the first image file to obtain an updated first image file.
The method according to any one of claims 1 to 6, wherein the format of the first image file is an exchangeable image file format EXIF.
The method of any of claims 1-7, wherein the image generation information is stored in a vendor comment field of the first image file.
The method of claim 8 wherein said image generation information is stored in said vendor annotation field in a tag image file format TIFF.
An apparatus, comprising:

An acquiring unit, configured to capture image data, and acquire image generation information when the image data is captured, and generate a first image file including the image data and the image generation information;

a classifying unit, configured to perform an image classification operation on the first image file by using the image generation information acquired by the acquiring unit;

And a display unit configured to display the first image file according to the classification result of the classification unit in response to an operation for viewing the gallery.
The device according to claim 10, wherein the image generation information acquired by the acquisition unit includes at least one of information of a shooting parameter, information of a shooting mode, information of a shooting scene, and information of a camera type.
The apparatus according to claim 11, wherein said photographing parameter includes an exposure value, said photographing mode includes a panorama mode and a normal mode, said photographing scene including a person photographing scene, a building photographing scene, and a natural scenery photographing scene The camera type is used to indicate that the image data is captured by a front camera or a rear camera;

Wherein, when the shooting scene is a scene shot by the character, the image generation information further includes character feature information.
The apparatus according to any one of claims 10 to 12, wherein the classification unit is further configured to perform an image recognition operation on the image data;

The image generation information is not obtained by the classification unit identifying the image data.
The device according to any one of claims 10 to 13, wherein the classifying unit is specifically configured to perform an image recognition operation on the image data to analyze the image data to obtain classification feature information; The image generation information and the classification feature information are performed, and an image classification operation is performed on the first image file.
The device according to claim 14, wherein the device further comprises:

And an updating unit, configured to perform an image recognition operation on the image data by the classification unit to analyze the image data to obtain classification feature information, save the classification feature information in the first image file, and obtain an updated The first image file.
The device according to any one of claims 10-15, wherein the format of the first image file is an exchangeable image file format EXIF.
The apparatus of any of claims 10-16, wherein the image generation information is stored in a vendor annotation field of the first image file.
The apparatus according to claim 17, wherein said image generation information is stored in said vendor comment field in a tag image file format TIFF.
An apparatus, comprising: the apparatus comprising: a processor, a memory, a camera, and a display; the memory and the display being coupled to the processor, the display for displaying an image file, the memory A non-volatile storage medium for storing computer program code, the computer program code comprising computer instructions, when the processor executes the computer instructions

The camera is configured to capture image data;

The processor is configured to acquire image generation information when the camera captures the image data, generate a first image file including the image data and the image generation information, and generate information by using the image generation information The first image file performs an image classification operation;

The display is configured to display the first image file in a category in response to an operation for viewing a gallery.
The device according to claim 19, wherein the image generation information acquired by the processor includes at least one of information of a shooting parameter, information of a shooting mode, information of a shooting scene, and information of a camera type.
The apparatus according to claim 20, wherein said photographing parameter includes an exposure value, said photographing mode includes a panorama mode and a normal mode, said photographing scene including a person photographing scene, a building photographing scene, and a natural scenery photographing scene The camera type is used to indicate that the image data is captured by a front camera or a rear camera;

Wherein, when the shooting scene is a scene shot by the character, the image generation information further includes character feature information.
The apparatus according to any one of claims 19 to 21, wherein the image generation information is not obtained by the processor performing an image recognition operation on the image data.
The device according to any one of claims 19 to 22, wherein the processor is specifically configured to perform an image recognition operation on the image data to analyze the image data to obtain classification feature information; The image generation information and the classification feature information are performed, and an image classification operation is performed on the first image file.
The device according to claim 23, wherein the processor is further configured to perform an image recognition operation on the image data to analyze the image data to obtain classification feature information, in the first image The classification feature information is saved in the file to obtain an updated first image file.
The device according to any one of claims 19-24, wherein the format of the first image file is an exchangeable image file format EXIF.
Apparatus according to any of claims 19-25, wherein said image generation information is stored in a vendor comment field of said first image file.
The apparatus according to claim 26, wherein said image generation information is stored in said vendor comment field in a tag image file format TIFF.
A control device, comprising: a processor and a memory, the memory for storing computer program code, the computer program code comprising computer instructions, when the processor executes the computer instruction The control device performs the method of any of claims 1-9.
A computer storage medium, characterized in that the computer storage medium comprises computer instructions that, when executed on a device, cause the device to perform the method of any one of claims 1-9.
A computer program product, wherein the computer program product, when run on a computer, causes the computer to perform the method of any of claims 1-9.