CN115689925A - Image processing method, intelligent terminal and storage medium - Google Patents

Abstract

The application discloses an image processing method, an intelligent terminal and a storage medium, wherein the image processing method comprises the following steps: responding to preset operation, and determining a target area needing highlight removal in the image to be processed; and carrying out highlight removing treatment on the target area based on a pre-trained highlight removing model to obtain a target image. According to the image processing method, the highlight removing area in the image is determined according to the operation input by the user, so that the user can selectively remove the highlight from the image, the image quality is improved, and the user experience is improved.

Description

Image processing method, intelligent terminal and storage medium

Technical Field

The present application relates to the field of terminal technologies, and in particular, to an image processing method, an intelligent terminal, and a storage medium.

Background

In the process of conception and realization of the application, the inventor discovers that: under the condition of uneven illumination, local highlight areas often appear in the shot images, so that the shot images are unnatural, and the attractiveness and the naturalness of the images are affected. In some implementations, the highlight removal processing mode for the image can only remove the whole highlight of the image, and the image effect is unnatural.

The foregoing description is provided for general background information and is not admitted to be prior art.

Disclosure of Invention

The application aims to provide an image processing method, an intelligent terminal and a storage medium.

The application provides an image processing method, which can be applied to an intelligent terminal (such as a mobile phone), and comprises the following steps:

s1, responding to a preset operation, and determining a target area needing highlight removal in an image to be processed;

and S2, carrying out highlight removing treatment on the target area based on a pre-trained highlight removing model to obtain a target image.

Optionally, before the step S1, the method further includes:

and inputting the image to be processed into the highlight removing model to perform highlight area identification, and acquiring and displaying at least one highlight area in the image to be processed.

Optionally, the step S1 includes:

receiving a selection operation for a highlight area and/or an object displayed on the image to be processed, and determining the highlight area and/or the highlight area on the selected object selected by the selection operation as the target area.

Optionally, the step S1 includes:

and responding to the triggering operation of an automatic selection key, and selecting a highlight area meeting preset conditions from the image to be processed to be determined as the target area.

Optionally, the meeting of the preset condition includes at least one of: the position of the highlight area is a preset position, the area of the highlight area is larger than or equal to a preset area threshold, and the object where the highlight area is located is a preset type object.

Optionally, the step S1 includes:

and receiving touch operation on the image to be processed, and determining a closed area defined by a touch track corresponding to the touch operation as the target area.

Optionally, before the step S1, the method further includes:

acquiring a training sample image;

and training the initial highlight removing model according to the training sample image to obtain the trained highlight removing model.

Optionally, the training sample image comprises a no highlights image and/or a highlights image.

Optionally, the method further comprises:

and responding to the shooting operation, and saving and displaying the target image.

The application also provides an intelligent terminal, intelligent terminal includes: a memory, a processor, wherein the memory has stored thereon a computer program which, when executed by the processor, implements the steps of the image processing method as set forth in any one of the above.

The present application further provides a storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the image processing method as set forth in any one of the above.

The embodiment of the application provides an image processing method, an intelligent terminal and a storage medium, wherein the image processing method comprises the following steps: responding to preset operation, and determining a target area needing highlight removal in the image to be processed; and carrying out highlight removing treatment on the target area based on a pre-trained highlight removing model to obtain a target image. Therefore, the highlight removing area in the image is determined according to the operation input by the user, so that the user can selectively remove the highlight from the image, the image quality is improved, and the user experience is improved.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application. In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the description of the embodiments will be briefly described below, and it is obvious for those skilled in the art to obtain other drawings without inventive step.

Fig. 1 is a schematic diagram of a hardware structure of a mobile terminal according to an embodiment of the present disclosure;

fig. 2 is a communication network system architecture diagram according to an embodiment of the present application;

fig. 3 is a schematic flowchart of an image processing method according to an embodiment of the present application;

fig. 4 is a schematic flowchart of an image processing method according to an embodiment of the present disclosure;

FIG. 5 is a schematic diagram of an original image in an embodiment of the present application;

fig. 6 is a schematic diagram of an original image of a highlight region where a face is detected in an embodiment of the present application;

fig. 7 is a schematic diagram of an original image with highlight areas removed in the embodiment of the present application.

The implementation, functional features and advantages of the objectives of the present application will be further explained with reference to the accompanying drawings. Specific embodiments of the present application have been shown by way of example in the drawings and will be described in more detail below. The drawings and written description are not intended to limit the scope of the inventive concepts in any manner, but rather to illustrate the concepts of the application by those skilled in the art with reference to specific embodiments.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, a reference to an element identified by the phrase "comprising one of 82308230- \8230;" 8230; "does not exclude the presence of additional identical elements in any process, method, article, or apparatus that comprises that element, and optionally, components, features, elements having the same designation in different embodiments of the application may have the same meaning or may have different meanings as determined by their interpretation in that particular embodiment or by their further context in that particular embodiment.

It should be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope herein. The word "if," as used herein, may be interpreted as "at \8230; \8230when" or "when 8230; \823030when" or "in response to a determination," depending on the context. Also, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context indicates otherwise. It will be further understood that the terms "comprises," "comprising," "includes" and/or "including," when used in this specification, specify the presence of stated features, steps, operations, elements, components, items, species, and/or groups, but do not preclude the presence, or addition of one or more other features, steps, operations, elements, components, species, and/or groups thereof. The terms "or" and/or "as used herein are to be construed as inclusive or meaning any one or any combination. Thus, "a, B or C" or "a, B and/or C" means "any of the following: a; b; c; a and B; a and C; b and C; A. b and C ". An exception to this definition will occur only when a combination of elements, functions, steps or operations are inherently mutually exclusive in some way.

It should be understood that, although the steps in the flowcharts in the embodiments of the present application are shown in sequence as indicated by the arrows, the steps are not necessarily performed in sequence as indicated by the arrows. The steps are not performed in the exact order shown and may be performed in other orders unless explicitly stated herein. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, in different orders, and may be performed alternately or partially with other steps or at least some of the sub-steps or stages of other steps.

It should be noted that step numbers such as S1 and S2 are used herein for the purpose of more clearly and briefly describing corresponding contents, and do not constitute a substantial limitation on the sequence, and those skilled in the art may perform S2 first and then S1 in the specific implementation, but these should be within the protection scope of the present application.

It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

In the following description, suffixes such as "module", "component", or "unit" used to denote elements are used only for the convenience of description of the present application, and have no specific meaning in themselves. Thus, "module", "component" or "unit" may be used mixedly.

The smart terminal may be implemented in various forms. For example, the smart terminal described in the present application may include mobile terminals such as a mobile phone, a tablet computer, a notebook computer, a palmtop computer, a Personal Digital Assistant (PDA), a Portable Media Player (PMP), a navigation device, a wearable device, a smart band, a pedometer, and the like, and fixed terminals such as a Digital TV, a desktop computer, and the like.

The following description will be given taking a mobile terminal as an example, and it will be understood by those skilled in the art that the configuration according to the embodiment of the present application can be applied to a fixed type terminal in addition to elements particularly used for mobile purposes.

Referring to fig. 1, which is a schematic diagram of a hardware structure of a mobile terminal for implementing various embodiments of the present application, the mobile terminal 100 may include: an RF (Radio Frequency) unit 101, a wireless network module 102, an audio output unit 103, an a/V (audio/video) input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, a processor 110, and a power supply 111. Those skilled in the art will appreciate that the mobile terminal architecture shown in fig. 1 is not intended to be limiting of mobile terminals, and that a mobile terminal may include more or fewer components than shown, or some components may be combined, or a different arrangement of components.

The following describes each component of the mobile terminal in detail with reference to fig. 1:

the radio frequency unit 101 may be configured to receive and transmit signals during information transmission and reception or during a call, and specifically, receive downlink information of a base station and then process the downlink information to the processor 110; in addition, the uplink data is transmitted to the base station. Typically, radio frequency unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. Alternatively, the radio frequency unit 101 may also communicate with a network and other devices through wireless communication. The wireless communication may use any communication standard or protocol, including but not limited to GSM (Global System for Mobile communications), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000 ), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division duplex-Long Term Evolution), TDD-LTE (Time Division duplex-Long Term Evolution, time Division Long Term Evolution), 5G, and so on.

The wireless network belongs to a short-distance wireless transmission technology, and the mobile terminal can help a user to receive and send e-mails, browse webpages, access streaming media and the like through the wireless network module 102, and provides wireless broadband internet access for the user. Although fig. 1 shows the wireless network module 102, it is understood that it does not belong to the essential constitution of the mobile terminal, and may be omitted entirely as needed within the scope not changing the essence of the invention.

The audio output unit 103 may convert audio data received by the radio frequency unit 101 or the wireless network module 102 or stored in the memory 109 into an audio signal and output as sound when the mobile terminal 100 is in a call signal reception mode, a call mode, a recording mode, a voice recognition mode, a broadcast reception mode, or the like. Also, the audio output unit 103 may also provide audio output related to a specific function performed by the mobile terminal 100 (e.g., a call signal reception sound, a message reception sound, etc.). The audio output unit 103 may include a speaker, a buzzer, and the like.

The a/V input unit 104 is for receiving an audio or video signal. The a/V input Unit 104 may include a Graphics Processing Unit (GPU) 1041 and a microphone 1042, the Graphics processor 1041 Processing image data of still pictures or video obtained by an image capturing device (e.g., a camera) in a video capturing mode or an image capturing mode. The processed image frames may be displayed on the display unit 106. The image frames processed by the graphic processor 1041 may be stored in the memory 109 (or other storage medium) or transmitted via the radio frequency unit 101 or the wireless network module 102. The microphone 1042 may receive sounds (audio data) via the microphone 1042 in a phone call mode, a recording mode, a voice recognition mode, or the like, and may be capable of processing such sounds into audio data. The processed audio (voice) data may be converted into a format output transmittable to a mobile communication base station via the radio frequency unit 101 in case of a phone call mode. The microphone 1042 may implement various types of noise cancellation (or suppression) algorithms to cancel (or suppress) noise or interference generated in the course of receiving and transmitting audio signals.

The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Optionally, the light sensor includes an ambient light sensor and a proximity sensor, the ambient light sensor may adjust the brightness of the display panel 1061 according to the brightness of ambient light, and the proximity sensor may turn off the display panel 1061 and/or the backlight when the mobile terminal 100 moves to the ear. As one of the motion sensors, the accelerometer sensor can detect the magnitude of acceleration in each direction (generally three axes), detect the magnitude and direction of gravity when stationary, and can be used for applications of recognizing gestures of a mobile phone (such as horizontal and vertical screen switching, related games, magnetometer gesture calibration), vibration recognition related functions (such as pedometers and taps), and the like; as for other sensors such as a fingerprint sensor, a pressure sensor, an iris sensor, a molecular sensor, a gyroscope, a barometer, a hygrometer, a thermometer, and an infrared sensor, which can be configured on the mobile phone, further description is omitted here.

The display unit 106 is used to display information input by a user or information provided to the user. The Display unit 106 may include a Display panel 1061, and the Display panel 1061 may be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like.

The user input unit 107 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the mobile terminal. Alternatively, the user input unit 107 may include a touch panel 1071 and other input devices 1072. The touch panel 1071, also referred to as a touch screen, may collect a touch operation performed by a user on or near the touch panel 1071 (e.g., an operation performed by the user on or near the touch panel 1071 using a finger, a stylus, or any other suitable object or accessory), and drive a corresponding connection device according to a predetermined program. The touch panel 1071 may include two parts of a touch detection device and a touch controller. Optionally, the touch detection device detects a touch orientation of a user, detects a signal caused by a touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts the touch information into touch point coordinates, sends the touch point coordinates to the processor 110, and can receive and execute commands sent by the processor 110. Alternatively, the touch panel 1071 may be implemented in various types, such as resistive, capacitive, infrared, and surface acoustic wave. The user input unit 107 may include other input devices 1072 in addition to the touch panel 1071. Optionally, other input devices 1072 may include, but are not limited to, one or more of a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, etc., and the like, without limitation.

Alternatively, the touch panel 1071 may cover the display panel 1061, and when the touch panel 1071 detects a touch operation thereon or nearby, the touch panel 1071 transmits the touch operation to the processor 110 to determine the type of the touch event, and then the processor 110 provides a corresponding visual output on the display panel 1061 according to the type of the touch event. Although the touch panel 1071 and the display panel 1061 are shown in fig. 1 as two separate components to implement the input and output functions of the mobile terminal, in some embodiments, the touch panel 1071 and the display panel 1061 may be integrated to implement the input and output functions of the mobile terminal, and is not limited herein.

The interface unit 108 serves as an interface through which at least one external device is connected to the mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device having an identification module, an audio input/output (I/O) port, a video I/O port, an earphone port, and the like. The interface unit 108 may be used to receive input (e.g., data information, power, etc.) from external devices and transmit the received input to one or more elements within the mobile terminal 100 or may be used to transmit data between the mobile terminal 100 and external devices.

The memory 109 may be used to store software programs as well as various data. The memory 109 may mainly include a program storage area and a data storage area, and optionally, the program storage area may store an operating system, an application program (such as a sound playing function, an image playing function, and the like) required by at least one function, and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like. Optionally, the memory 109 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.

The processor 110 is a control center of the mobile terminal, connects various parts of the entire mobile terminal using various interfaces and lines, and performs various functions of the mobile terminal and processes data by operating or executing software programs and/or modules stored in the memory 109 and calling data stored in the memory 109, thereby performing overall monitoring of the mobile terminal. Processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor and a modem processor, optionally the application processor primarily handles operating systems, user interfaces, application programs, etc., and the modem processor primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 110.

The mobile terminal 100 may further include a power supply 111 (e.g., a battery) for supplying power to various components, and preferably, the power supply 111 may be logically connected to the processor 110 via a power management system, so as to manage charging, discharging, and power consumption management functions via the power management system.

Although not shown in fig. 1, the mobile terminal 100 may further include a bluetooth module or the like, which is not described in detail herein.

In order to facilitate understanding of the embodiments of the present application, a communication network system on which the mobile terminal of the present application is based is described below.

Referring to fig. 2, fig. 2 is an architecture diagram of a communication Network system provided in an embodiment of the present application, where the communication Network system is an LTE system of a universal mobile telecommunications technology, and the LTE system includes a UE (User Equipment) 201, an e-UTRAN (Evolved UMTS Terrestrial Radio Access Network) 202, an epc (Evolved Packet Core) 203, and an IP service 204 of an operator, which are in communication connection in sequence.

Optionally, the UE201 may be the terminal 100 described above, and is not described herein again.

The E-UTRAN202 includes eNodeB2021 and other eNodeBs 2022, among others. Alternatively, the eNodeB2021 may be connected with other enodebs 2022 through a backhaul (e.g., X2 interface), the eNodeB2021 is connected to the EPC203, and the eNodeB2021 may provide the UE201 access to the EPC 203.

The EPC203 may include an MME (Mobility Management Entity) 2031, an hss (Home Subscriber Server) 2032, other MMEs 2033, an SGW (Serving gateway) 2034, a pgw (PDN gateway) 2035, and a PCRF (Policy and Charging Rules Function) 2036, and the like. Optionally, the MME2031 is a control node that handles signaling between the UE201 and the EPC203, providing bearer and connection management. HSS2032 is used to provide registers to manage functions such as home location register (not shown) and holds subscriber specific information about service characteristics, data rates, etc. All user data may be sent through SGW2034, PGW2035 may provide IP address allocation and other functions for UE201, PCRF2036 is a policy and charging control policy decision point for traffic data flow and IP bearer resources, which selects and provides available policy and charging control decisions for policy and charging enforcement function (not shown).

The IP services 204 may include the internet, intranets, IMS (IP Multimedia Subsystem), or other IP services, among others.

Although the LTE system is described as an example, it should be understood by those skilled in the art that the present application is not limited to the LTE system, but may also be applied to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA, 5G, and future new network systems (e.g. 6G), and the like.

Based on the above mobile terminal hardware structure and communication network system, various embodiments of the present application are provided.

Referring to fig. 3, an image processing method provided in this embodiment of the present application may be applied to a case of removing a highlight area in an image, and the method may be executed by an image processing apparatus provided in this embodiment of the present application, where the apparatus may be implemented in a software and/or hardware manner, and in a specific application, the apparatus may be specifically an intelligent terminal, a server, and the like. The smart terminal may be implemented in various forms, and the smart terminal described in this embodiment may include a mobile terminal such as a mobile phone, a tablet computer, a notebook computer, a palm top computer, a Personal Digital Assistant (PDA), a Portable Media Player (PMP), a wearable device, a smart band, and a pedometer. In this embodiment, taking an execution subject of the image processing method as an example of an intelligent terminal, the image processing method includes the following steps:

s1, responding to preset operation, and determining a target area needing highlight removal in the image to be processed.

And S2, carrying out highlight removing treatment on the target area based on a pre-trained highlight removing model to obtain a target image.

Optionally, the image to be processed may be a preview image or a pre-stored image captured by the intelligent terminal in real time through an image capturing device of the intelligent terminal, such as a camera, or a photo, a video, or the like selected by the intelligent terminal from an image library in the local or cloud end according to a selection operation of the user, or an image sent to the intelligent terminal by other devices. It should be noted that, as will be understood by those skilled in the art, a video includes a series of image frames, each of which may also be referred to as an image, and the image processing apparatus provided in the embodiment of the present application may also apply the image processing method provided in the embodiment of the present application to one or more image frames included in the video, for example, the image processing apparatus may apply the image processing method provided in the embodiment of the present application to each image frame of the video. The target area may be an area where a target object is located, and the target object may be a human body, or may be one or more parts of the human body, such as a face, a hand, and the like; the target object may also be an object, such as an animal, a plant, a building, food, clothing, etc., and may also be one or more parts of an object, such as leaves, a trunk, etc. It is understood that the determination of the target object may be automatically implemented, or may be set in combination with a manual operation of a user. Alternatively, the target area may be one or more areas. Optionally, the preset operation may be set according to actual needs, for example, touch operation, air gesture operation, voice instruction, and the like may be performed.

Optionally, the highlight removing model may be constructed by using a deep learning Network, in this embodiment, the highlight removing model is constructed by using a Generative Adaptive Network (GAN), where the highlight removing model includes a generator and a discriminator, the generator is configured to generate an image with approximately no highlight through a random sample, and the discriminator is configured to discriminate whether the image generated by the generator is true or false and give confidence of the discrimination. Optionally, before the step S1, the method further includes: acquiring a training sample image; optionally, the training sample images comprise non-highlight images and/or highlight images; and training the initial highlight removing model according to the training sample image to obtain the trained highlight removing model. Optionally, when the training sample image is acquired, the light is controlled accordingly, such as controlling the light irradiation position, intensity, and the like, so as to acquire the highlight-free image and/or the highlight image. And training the initial highlight removing model based on the training sample image to obtain the highlight removing model capable of effectively removing highlight areas in the image.

Optionally, before the step S1, the method further includes:

and inputting the image to be processed into the highlight removing model for highlight area identification, and acquiring and displaying at least one highlight area in the image to be processed.

Optionally, the intelligent terminal inputs the image to be processed into the highlight removing model, so that the highlight removing model performs highlight region identification on the image to be processed, thereby obtaining at least one highlight region identified by the highlight removing model from the image to be processed, and further displaying the at least one highlight region in the image to be processed. Thus, the highlight area in the image can be quickly and conveniently determined, and the image processing efficiency is improved.

Optionally, the step S1 includes:

receiving a selection operation for a highlight area and/or an object displayed on the image to be processed, and determining the highlight area selected by the selection operation and/or a highlight area on the selected object as the target area.

It is understood that sometimes a user may only need to perform the highlight removing processing on a part of the highlight area on the image, such as performing the highlight removing processing on the highlight area which significantly affects the image quality, but not performing the highlight removing processing on the highlight area which does not significantly affect the image quality, and/or performing the highlight removing processing only on the highlight area located at a specific position, etc. Taking the to-be-processed image including the portrait as an example, if the portrait area and the non-portrait area in the to-be-processed image both have a highlight area, only the highlight area in the portrait area may need to be subjected to the highlight removal processing, or only the highlight area on the portrait needs to be subjected to the highlight removal processing. Therefore, in order to flexibly control the region where highlight needs to be removed and improve the processing efficiency, the user may select the highlight region displayed on the image to be processed through a touch operation, a blank operation, or the like, and determine the selected highlight region as the target region where highlight needs to be removed. For example, the user may sequentially click on a plurality of highlight areas displayed on the image to be processed by a finger, a stylus, or the like, so as to determine the selected highlight area as a target area from which highlight needs to be removed.

Alternatively, the user may also select an object displayed on the image to be processed through a touch operation, a blank operation, or the like, and determine a highlight region located on the selected object as the target region. For example, the user may click a portrait displayed on the image to be processed with a finger or a stylus pen, so as to determine a highlight area located on the portrait as a target area where highlight needs to be removed, or the user may click a building displayed on the image to be processed with a finger in the space, so as to determine a highlight area located on the building as a target area where highlight needs to be removed.

Optionally, the intelligent terminal may be provided with a selection trigger key, so as to control the to-be-processed image to be in a selected state after the selection trigger key is triggered, for example, control each highlight region to be highlighted, or control each dashed box to be labeled, so as to receive a selection operation input by a user on the highlight region and/or the object displayed on the to-be-processed image. Therefore, the target area needing highlight removal in the image to be processed is determined in different modes, so that the user can selectively highlight and remove the image, and the user experience is further improved.

Optionally, the step S1 includes:

and responding to the triggering operation of an automatic selection key, and selecting a highlight area meeting a preset condition from the image to be processed to determine the highlight area as the target area.

Optionally, the intelligent terminal may also automatically select a target area from the image to be processed, where highlight needs to be removed, based on a user requirement. Optionally, the intelligent terminal may be provided with an automatic selection key, the automatic selection key may be displayed on the display interface including the to-be-processed image through invocation by a user, and when the user inputs a trigger operation on the automatic selection key through a touch operation, a voice, or the like, the intelligent terminal may automatically select a highlight area satisfying a preset condition from the to-be-processed image and determine the highlight area as the target area. Optionally, the meeting of the preset condition may include at least one of: the position of the highlight area is a preset position, the area of the highlight area is larger than or equal to a preset area threshold, and the object where the highlight area is located is a preset type object. Optionally, the preset position may be set according to actual needs, for example, the preset position may be an edge area of an image. It can be understood that when the highlight region is located at the edge of the image to be processed, the effect on the image quality may be considered to be small, and at this time, the highlight removal processing may not be performed on the highlight region, and when the highlight region is located at the center or near the center of the image to be processed, the effect on the image quality may be considered to be large, and at this time, the highlight removal processing may be performed on the highlight region. Optionally, the preset area threshold may be set according to actual needs, for example, may be set as a product of a preset scaling factor and an image area. It is understood that when the area of the highlight region is small, the effect on the image quality may be considered to be small, and in this case, the highlight removal processing may not be performed on the highlight region, and when the area of the highlight region is large, the effect on the image quality may be considered to be large, and in this case, the highlight removal processing may be performed on the highlight region. Optionally, the preset type object may be set according to actual needs, for example, the preset type object may be a portrait, a building, a tree, or the like. It can be understood that if the to-be-processed image is shot for a preset type of object, the picture quality of the preset type of object is usually the most concerned by the user in the to-be-processed image, for example, if the to-be-processed image is a shot portrait, then in the to-be-processed image, the portrait should avoid the existence of a highlight region, and therefore, when the object in which the highlight region is located is the preset type of object, it can be considered that highlight removal processing needs to be performed on the highlight region. Therefore, the highlight removing target area in the image to be processed is automatically determined, so that the user can selectively remove the highlight from the image, and the user experience is further improved.

Optionally, the step S1 includes:

and receiving touch operation on the image to be processed, and determining a closed area surrounded by a touch track corresponding to the touch operation as the target area.

Optionally, the touch operation may be set according to actual needs, including but not limited to single-finger sliding, multi-finger sliding, and the like. It can be understood that, since a user may generally find a highlight area in an image to be processed by naked eyes, in order to quickly determine a target area needing highlight removal from the image to be processed and meet a user requirement, the intelligent terminal may receive a touch operation input by the user on the image to be processed, and determine an enclosed area surrounded by a touch trajectory corresponding to the touch operation as the target area. Optionally, the intelligent terminal may be provided with a selection trigger key, so that after the selection trigger key is triggered, the to-be-processed image is controlled to enter a selection state, for example, a frame of the to-be-processed image is highlighted, so as to receive a touch operation input by a user on the to-be-processed image. Therefore, the highlight removing method and the highlight removing device can be convenient for a user to quickly and accurately determine the highlight-required target area in the image to be processed, so that the user can selectively remove the highlight from the image, and the user experience is further improved.

In summary, in the image processing method provided in the above embodiment, the region in the image that needs to be highlight-removed is determined according to the operation input by the user, so that the user can selectively highlight-remove the image, the image quality is improved, and the user experience is improved.

Optionally, the method further comprises: and responding to the shooting operation, and saving and displaying the target image.

Optionally, when the image to be processed is a preview image, the intelligent terminal stores and displays the target image after receiving the shooting operation. Optionally, after performing highlight removal processing on the target area in the image to be processed, the intelligent terminal may also directly replace the image to be processed with the obtained target image for display, so that a user can quickly know a highlight removal effect of the image. Optionally, the intelligent terminal may further provide a contrast key, and when the user triggers the contrast key, the intelligent terminal may simultaneously display the to-be-processed image and the target image.

Based on the same application concept of the foregoing embodiment, this embodiment describes in detail the technical solutions of the foregoing embodiments by using a specific example, and in this embodiment, the highlight removing model is taken as an example of a portrait highlight removing model for highlight suppression of a portrait.

Referring to fig. 4, the image processing method provided in the embodiment of the present application mainly includes the following operations:

(I) constructing natural light-highlight portrait dataset

(1) Selecting a part of people as a provider of portrait data;

(2) By utilizing the automatic lighting system, the position and the intensity of the light on the portrait are controlled in an all-round 360-degree mode through programming the light, the position of the portrait is controlled, the natural light and highlight data images of the portrait are collected, the situation in a real scene is more truly approximated, and therefore training sample images comprising highlight-free images (namely natural light images) and highlight images are obtained.

(II) constructing a portrait highlight removal model for portrait highlight inhibition based on GAN network

For the aspect of image generation, the application adopts a GAN network in a deep learning network. The U-net architecture is used as a generator to generate images without highlight, the classification network is used as a discriminator to distinguish the truth of the generated images, and the accuracy of the generator is improved, so that the highlight removing function of the highlight images is realized.

(III) deployment of portrait highlight removal model

(1) Designing a corresponding drive;

(2) Deploying the trained portrait highlight removal model to the existing intelligent terminal shooting function, and adding a portrait highlight removal function key;

(3) And designing a UI (user interface) and increasing a preview display function of highlight removal of the portrait.

(IV) detection of portrait highlights

(1) Opening an intelligent terminal camera, sequentially clicking a portrait mode and a highlight removing function in the setting, and seeing the real-time highlight removing effect of the image;

(2) And clicking to photograph to obtain a complete highlight-removed image.

The technical solution of the foregoing embodiment is described in detail by a specific application scenario.

When a user travels to a place with large environmental difference, due to the fact that the current illumination is large, when the intelligent terminal conducts portrait shooting, due to the problem of the illumination, certain highlight always appears on the face, the natural structure of the face is broken, the problem of a highlight area is weakened, the highlight area is very steep, and the naturally shot photo is not good. At this time, through opening the intelligent terminal camera that has the portrait to remove the highlight function, select the portrait to remove the highlight function, shoot again, the highlight in highlight region just can be got rid of to the photo quality that shoots is better.

Optionally, the specific operation steps of using the intelligent terminal to shoot the highlight-removed image are as follows:

the method comprises the steps that firstly, a camera of the intelligent terminal is opened, a highlight removing mode is selected, and the intelligent terminal can automatically perform the face highlight removing function. As shown in fig. 5 (a) and 5 (b), when the user clicks a "highlight removal" mode key displayed on a preview interface of the smart terminal camera, the smart terminal enters a highlight removal mode.

And secondly, clicking a highlight area key at the lower right corner of the screen to see the highlight area of the face under AI detection and highlighting the highlight area.

Optionally, a "highlight region" key may be displayed on a lower right corner of a preview interface of the intelligent terminal camera, and when the user clicks the "highlight region" key on the lower right corner of the screen, the intelligent terminal detects and displays the highlight region of the face, so that the user can see the highlight region of the face under AI detection, and highlight the highlight region at the same time. Fig. 6 (a) shows the highlight region of the face detected in fig. 5 (a), and fig. 6 (b) shows the highlight region of the face detected in fig. 5 (b).

And thirdly, clicking a contrast effect key at the lower right corner of the screen to see the highlight effect of the portrait of the AI, and highlighting the contrast effect. The lower right corner of a preview interface of the intelligent terminal camera can be displayed with a contrast effect key, and when a user clicks the contrast effect key on the lower right corner of the screen, the intelligent terminal performs highlight removing processing on a highlight area of a detected face, so that the user can see the face after highlight removing. Fig. 7 (a) shows the effect of removing the highlight region of the face in fig. 5 (a), and fig. 7 (b) shows the effect of removing the highlight region of the face in fig. 5 (b).

And fourthly, clicking to shoot, and storing the image with the highlight removed from the face.

In summary, in the image processing method provided in the above embodiment, based on the deep learning technology, the automatic lighting system is mainly used to construct the natural light-highlight data set of the portrait, the GAN network is used to train the data set, and the trained generator is extracted as the portrait highlight removal model of the intelligent terminal, so that the highlight removal function of the portrait is realized at the intelligent terminal. The scheme is more advanced in technology than multiple similar schemes, is more real in effect than the similar schemes, is more applicable to application in complex scenes, mainly has the advantage that textures and colors are attached to a normal portrait, and more accords with subjective vision of human eyes. Meanwhile, the user pays more attention to the face image according to the photographing requirement of the user, and the highlight removal of the portrait can not only remove unnatural highlight on the face, but also enrich the portrait details, so that the photo is natural and beautiful, and perfectly meets the market requirement; moreover, the method has the advantages of simple, convenient, quick and intelligent interaction process and improvement of user experience.

The embodiment of the present application further provides an intelligent terminal, the intelligent terminal includes: a memory, and a processor, wherein the memory has a computer program stored thereon, and the computer program, when executed by the processor, implements the steps of the image processing method as described in any one of the above.

An embodiment of the present application further provides a storage medium, where a computer program is stored on the storage medium, and when being executed by a processor, the computer program implements the steps of the image processing method as described in any one of the above.

In the embodiments of the intelligent terminal and the storage medium provided in the embodiments of the present application, all technical features of any one of the embodiments of the image processing method may be included, and the expanding and explaining contents of the specification are basically the same as those of the embodiments of the method, and are not described herein again.

Embodiments of the present application also provide a computer program product, which includes computer program code, when the computer program code runs on a computer, the computer is caused to execute the image processing method as described in the above various possible embodiments.

An embodiment of the present application further provides a chip, which includes a memory and a processor, where the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that a device installed with the chip executes the image processing method described in the foregoing various possible embodiments.

It should be understood that the foregoing scenarios are only examples, and do not constitute a limitation on application scenarios of the technical solutions provided in the embodiments of the present application, and the technical solutions of the present application may also be applied to other scenarios. For example, as a person having ordinary skill in the art can know, with the evolution of the system architecture and the emergence of new service scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

The above-mentioned serial numbers of the embodiments of the present application are merely for description, and do not represent the advantages and disadvantages of the embodiments.

The steps in the method of the embodiment of the application can be sequentially adjusted, combined and deleted according to actual needs.

The units in the device of the embodiment of the application can be combined, divided and deleted according to actual needs.

In the embodiments of the present application, the same or similar term concepts, technical solutions and/or application scenario descriptions are generally only described in detail when they appear for the first time, and when they appear repeatedly later, they are not generally described repeatedly for brevity, and when understanding the content of the technical solutions of the present application, the same or similar term concepts, technical solutions and/or application scenario descriptions, etc. that are not described in detail later, can refer to their previous related detailed descriptions.

In the embodiments of the present application, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or recited in a certain embodiment.

The technical features of the technical solution of the present application may be arbitrarily combined, and for brevity of description, all possible combinations of the technical features in the embodiments are not described, however, as long as there is no contradiction between the combinations of the technical features, the scope of the present application should be considered as being described in the present application.

Through the description of the foregoing embodiments, it is clear to those skilled in the art that the method of the foregoing embodiments may be implemented by software plus a necessary general hardware platform, and certainly may also be implemented by hardware, but in many cases, the former is a better implementation. Based on such understanding, the technical solution of the present application or portions thereof that contribute to the prior art may be embodied in the form of a software product, where the computer software product is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) as above, and includes several instructions for enabling an intelligent terminal (e.g., a mobile phone, a computer, a server, a controlled terminal, or a network device) to execute the method of each embodiment of the present application.

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. The procedures or functions according to the embodiments of the present application are all or partially generated when the computer program instructions are loaded and executed on a computer. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a storage medium or transmitted from one storage medium to another storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The storage medium may be any available medium that can be accessed by a computer or a data storage device including one or more available media integrated servers, data centers, and the like. The usable medium may be a magnetic medium (e.g., floppy Disk, memory Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), among others.

The above description is only a preferred embodiment of the present application, and not intended to limit the scope of the present application, and all modifications of equivalent structures and equivalent processes, which are made by the contents of the specification and the drawings of the present application, or which are directly or indirectly applied to other related technical fields, are included in the scope of the present application.

Claims (10)

1. An image processing method, characterized in that the method comprises:

s1, responding to a preset operation, and determining a target area needing highlight removal in an image to be processed;

and S2, carrying out highlight removing processing on the target area based on a highlight removing model trained in advance to obtain a target image.

2. The method according to claim 1, wherein the step S1 is preceded by:

and inputting the image to be processed into the highlight removing model to perform highlight area identification, and acquiring and displaying at least one highlight area in the image to be processed.

3. Method according to claim 1 or 2, characterized in that said step S1 comprises:

receiving a selection operation for a highlight area and/or an object displayed on the image to be processed, and determining the highlight area selected by the selection operation and/or a highlight area on the selected object as the target area.

4. Method according to claim 1 or 2, characterized in that said step S1 comprises:

and responding to the triggering operation of an automatic selection key, and selecting a highlight area meeting a preset condition from the image to be processed to determine the highlight area as the target area.

5. The method according to claim 4, wherein the predetermined condition is satisfied and comprises at least one of: the position of the highlight area is a preset position, the area of the highlight area is larger than or equal to a preset area threshold, and the object where the highlight area is located is a preset type object.

6. Method according to any one of claims 1 to 3, characterized in that said step S1 comprises:

and receiving touch operation on the image to be processed, and determining a closed area surrounded by a touch track corresponding to the touch operation as the target area.

7. The method according to any one of claims 1 to 3, wherein step S1 is preceded by:

acquiring a training sample image;

and training the initial highlight removing model according to the training sample image to obtain the trained highlight removing model.

8. The method according to any one of claims 1 to 3, further comprising:

and responding to the shooting operation, and saving and displaying the target image.

9. An intelligent terminal, characterized in that, intelligent terminal includes: memory, processor, wherein the memory has stored thereon a computer program which, when being executed by the processor, carries out the steps of the image processing method according to any one of claims 1 to 8.

10. A storage medium, characterized in that the storage medium has stored thereon a computer program which, when being executed by a processor, carries out the steps of the image processing method according to any one of claims 1 to 8.

Images