WO2019233392A1

WO2019233392A1 - Image processing method and apparatus, electronic device, and computer-readable storage medium

Info

Publication number: WO2019233392A1
Application number: PCT/CN2019/089905
Authority: WO
Inventors: 刘耀勇
Original assignee: Oppo广东移动通信有限公司
Priority date: 2018-06-08
Filing date: 2019-06-04
Publication date: 2019-12-12
Also published as: CN108846351A

Abstract

An image processing method, comprising: acquiring an image to be tested; performing scene recognition on the image to be tested, so as to acquire a target scene label of the image to be tested; performing target detection on the image to be tested, so as to acquire a target body label of the image to be tested; determining an image label of the image to be tested according to the target scene label and/or the target body label; and performing image processing on the image to be tested according to the image label.

Description

Image processing method, device, electronic device and computer-readable storage medium

Cross-reference to related applications

This application claims the priority of a Chinese patent application filed on June 8, 2018 with the Chinese Patent Office, application number 201810589446.0, and invention name "Image Processing Method and Apparatus, Electronic Device, and Computer-readable Storage Medium", its entire content Incorporated by reference in this application.

Technical field

The present application relates to the field of computer technology, and in particular, to an image processing method and apparatus, an electronic device, and a computer-readable storage medium.

Background technique

With the rapid development of intelligent electronic equipment and image processing technology, the technology of intelligent electronic equipment for image processing is becoming more and more comprehensive. Optionally, the smart electronic device may classify the images according to the time when the images were taken, classify the images according to the location where the images were taken, classify the images based on the people in the images, and so on. By classifying images, it is convenient for users to view images more conveniently and quickly.

Summary of the Invention

Embodiments of the present application provide an image processing method, apparatus, electronic device, and computer-readable storage medium, which can improve the efficiency of image processing on an image.

An image processing method includes:

Obtaining images to be detected;

Performing scene recognition on the image to be detected, and obtaining a target scene label of the image to be detected;

Performing target detection on the image to be detected, and obtaining a target subject label of the image to be detected;

Determining an image label of the image to be detected according to the target scene label and / or the target subject label;

Performing image processing on the image to be detected according to the image label.

An image processing device includes:

A first acquisition module, configured to acquire an image to be detected;

A recognition module, configured to perform scene recognition on the image to be detected, and obtain a target scene label of the image to be detected;

A detection module, configured to perform target detection on the image to be detected, and obtain a target subject label of the image to be detected;

A determining module, configured to determine an image label of the image to be detected according to the target scene label and / or the target subject label;

A processing module, configured to perform image processing on the image to be detected according to the image label.

An electronic device includes a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor causes the processor to perform the operations of the method as described above.

A computer-readable storage medium has stored thereon a computer program that, when executed by a processor, implements the operations of the method described above.

The image processing method, device, electronic device, and computer-readable storage medium in the embodiments of the present application can determine the image tag corresponding to the image by performing scene recognition and target detection on the image, and perform image processing on the image to be detected according to the image tag. Batch processing is performed on the images to be detected corresponding to the same image tag, and different image processing is performed on the images to be detected corresponding to different image tags, which not only improves the efficiency of image processing of electronic devices, but also automatically processes the images of electronic devices. The effect is more diversified.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to explain the technical solutions in the embodiments of the present application or the prior art more clearly, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are merely These are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without paying creative work.

FIG. 1 is a flowchart of an image processing method according to an embodiment.

FIG. 2 is a flowchart of an image processing method in another embodiment.

FIG. 3 is a flowchart of an image processing method in another embodiment.

FIG. 4 is a structural block diagram of an image processing apparatus in an embodiment.

FIG. 5 is a structural block diagram of an image processing apparatus in another embodiment.

FIG. 6 is a structural block diagram of an image processing apparatus in another embodiment.

FIG. 7 is a schematic diagram of an internal structure of an electronic device in an embodiment.

FIG. 8 is a schematic diagram of an image processing circuit in one embodiment.

Detailed ways

In order to make the purpose, technical solution, and advantages of the present application clearer, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application, and are not used to limit the application.

FIG. 1 is a flowchart of an image processing method according to an embodiment. As shown in FIG. 1, an image processing method includes:

In operation 102, an image to be detected is obtained.

The electronic device may acquire an image to be detected. The image to be detected may be an image captured by the electronic device, an image stored in the electronic device, or an image downloaded by the electronic device. After acquiring the image to be detected, the electronic device may perform scene recognition, target detection, and the like on the image to be detected, and determine a shooting scene corresponding to the image to be detected and a target subject captured in the image to be detected. Further, after acquiring the shooting scene corresponding to the image to be detected and the target subject captured in the image to be detected, classification processing may be performed on the image to be detected according to the shooting scene, and image processing may be performed on the target subject in the image to be detected. .

Operation 104: Perform scene recognition on the image to be detected, and obtain a target scene label of the image to be detected.

The electronic device may perform scene recognition on the image to be detected, and obtain a target scene label of the image to be detected. The electronic device may use image classification technology to perform scene recognition on the image to be detected. Image classification refers to a method of dividing an image or an image area into one of several categories according to the characteristics reflected by the image information. The electronic device performing scene recognition according to the image classification technology includes: the electronic device may pre-store image feature information corresponding to multiple scene tags, and after acquiring the to-be-detected image, the electronic device may use the image feature of the to-be-detected image The information is respectively matched with the stored image feature information, and a scene label corresponding to the successfully matched graphic feature information is obtained as the target scene label of the image to be detected. Optionally, the scene tags pre-stored in the electronic device may include: landscape, beach, blue sky, green grass, snow, night, dark, backlight, sunset, fireworks, spotlight, indoor, macro, text, portrait, baby, cat, Dogs, food, etc. Further, the above scene tags may be divided into a background tag and a foreground tag, and the above background tags may include: landscape, beach, blue sky, green grass, snow scene, night scene, dark, backlight, sunset, fireworks, spotlight, indoor; the above foreground tag Can include: Macro, Text, Portrait, Baby, Cat, Dog, Food, etc.

Wherein, when the electronic device matches the image feature information of the image to be detected with the stored image feature information, if there is only one scene tag that is successfully matched, the electronic device uses the successfully matched scene tag as the Target scene tags; if there are multiple matching scene tags, the electronic device can obtain the confidence of each scene tag, and select a scene tag as the target scene tag according to the confidence of each scene tag.

Operation 106: Perform target detection on the image to be detected, and obtain a target subject label of the image to be detected.

The electronic device may also perform target detection on the image to be detected, and identify and locate a target subject in the image to be detected. The above object detection refers to a method of identifying the type of an object in an image and calibrating the position of the object in the image according to the characteristics reflected in the image information. When the electronic device performs target detection on the image to be detected, the image feature information of the image to be detected can be matched with the feature information corresponding to the stored subject tag, and the successfully matched subject tag is obtained as the target subject tag. The main body tags stored in the electronic device may include: portrait, baby, cat, dog, food, text, blue sky, green grass, beach, fireworks, and the like.

Optionally, when the electronic device performs target detection on the image to be detected, if there is only one subject tag in the image to be detected, the above body tag is used as the target subject label; if the electronic device performs target detection on the image to be detected, if If there are multiple subject tags in the image to be detected, the electronic device may select one of the multiple subject tags as the target subject tag. Among them, the electronic device may select a target body tag having the largest area of the corresponding body region from the plurality of body tags; the electronic device may also select a corresponding body region having the highest definition of the body region from the plurality of body tags as the target body tag, etc. .

After acquiring the target subject label, the electronic device may also obtain position information of the target subject area corresponding to the target subject tag, and mark the target subject area in the image to be detected. For example, the electronic device may mark a target subject region in the image to be detected with a rectangular frame.

Operation 108: Determine an image label of the image to be detected according to the target scene label and / or the target subject label.

When the electronic device obtains the target scene tag and the target subject tag, the electronic device may select one of the target scene tag and the target subject tag as the image tag, and the electronic device may also use the target scene tag and the target subject tag as the image tag. When the electronic device obtains only the target scene label, the electronic device may use the target subject label as the image label. When the electronic device obtains only the target subject label, the target scene label may be used as the image label.

Operation 110: Perform image processing on the image to be detected according to the image label.

After acquiring the image tag of the image to be detected, the electronic device may perform image processing on the image to be detected according to the image tag. Optionally, the electronic device may perform group processing, global image processing, and local image processing on the image to be detected according to the image tag. The above-mentioned grouping processing refers to grouping images to be detected according to image tags, for example, grouping images corresponding to the same image tag into a group. The aforementioned global image processing refers to performing color processing, saturation processing, brightness processing, contrast processing, and other processing on the entire image. The image local processing refers to performing color processing, saturation processing, brightness processing, contrast processing, and other processing on a part of an image.

The electronic device may obtain an image processing strategy corresponding to an image tag when performing image global processing or image local processing on an image to be detected, and perform image processing on the image to be detected according to the image processing strategy. Wherein, when the image label is the same as the target scene label, the electronic device can perform the global image processing on the image to be detected; when the image label is the same as the target body label, the electronic device can find the target body area corresponding to the target body label, and The target subject region in the image to be detected is subjected to local image processing. For example, when the image tag is "landscape", the electronic device may increase the saturation of the image to be detected; when the image tag is "portrait", the electronic device may perform a beauty treatment on the portrait area in the image to be detected.

Generally, when an electronic device performs image processing on an image, it can often only process a single image or a part of a single image according to fixed parameters. The effect on image processing is relatively simple, and the efficiency of image processing is low.

In the method of the embodiment of the present application, the image tags corresponding to the images can be determined by performing scene recognition and target detection on the images, and performing image processing on the images to be detected according to the image tags, can not only implement batch processing of the images to be detected corresponding to the same image tag, It is also possible to implement different image processing on the images to be detected corresponding to different image tags, which not only improves the efficiency of image processing of electronic equipment, but also has more diversified effects of automatic image processing by electronic equipment.

In the embodiment of the present application, the electronic device may use the classification model to perform scene recognition on the image to be detected, and use the detection model to perform target detection on the image to be detected. The above classification model and detection model are both deep learning models. Optionally, an independent classification model and detection model may be set in the electronic device, and the above classification model and detection model are run in parallel. The electronic device inputs the to-be-detected image into the above classification model and detection model, respectively, the above-mentioned classification model can perform scene recognition on the to-be-detected image and output a target scene label of the to-be-detected image; the above-mentioned detection model can perform object detection on the to-be-detected image, and The target subject label of the image to be detected is output. The electronic device may also reuse the above-mentioned classification model and detection model with the basic network, extract features from the to-be-detected image by multiplexing the basic network, and then send the extracted features to the classification model and the detection model, respectively. The classification model can perform scene recognition based on the extracted features; the above detection model can perform target detection based on the extracted features.

In one embodiment, performing scene recognition on the image to be detected and obtaining a target scene label of the image to be detected includes: performing scene recognition on the image to be detected, obtaining multiple scene labels corresponding to the image to be detected and the confidence of each scene label; according to the confidence The degree determines the target scene label.

Due to the richness and complexity of the picture content, the electronic device may output multiple scene labels corresponding to the image to be detected when performing scene recognition on the image to be detected; the above multiple scene labels represent corresponding to the image to be detected detected by the electronic device. Shoot the scene. When the electronic device outputs a plurality of scene labels corresponding to the image to be detected, the electronic device may output the confidence level of each scene label. The above-mentioned confidence level is a value used to indicate the credibility of the output parameter. When the above-mentioned confidence level is higher, it indicates that the confidence level of the scene label is higher. For example, the electronic device outputs multiple scene labels corresponding to the image to be detected may be: "blue sky" confidence 90%, "beach" confidence 85%, and "grassland" confidence 80%, then in the above three scene labels, " The blue sky label has the highest credibility, which means that the shooting scene corresponding to the image to be detected is the closest to "blue sky".

After the electronic device obtains multiple scene labels corresponding to the image to be detected and the confidence level of each scene label, the electronic device may determine the target scene label according to the above confidence level. Optionally, the electronic device may use the scene label with the highest confidence as the target scene label.

In the method of the embodiment of the present application, when scene recognition is performed on an image to be detected, multiple scene tags corresponding to the image to be detected and the confidence levels of each scene tag may be obtained, and then a target scene tag is determined according to the confidence level of each scene tag, and detection is acquired The target scene label has high accuracy, which is beneficial to further processing the image according to the obtained target scene label.

In one embodiment, determining the image label of the image to be detected according to the target scene label and / or the target subject label includes:

(1) If the target scene label cannot be obtained, use the target subject label as the image label.

(2) If the target subject label cannot be obtained, use the target scene label as the image label.

Due to the variety of image types, when performing scene recognition and target detection on the image to be detected, the image to be detected may not include a target scene tag or a target subject tag.

If the target subject label is not included in the image to be detected but only the target scene label, the electronic device may use the target scene label as the image label, and at this time, the image label of the image to be detected is a single label. If the target scene tag is not included in the image to be detected and only includes the target subject tag, the electronic device may use the target subject tag as the image tag, and at this time, the image tag of the image to be detected is a single tag. If the target scene label is not included in the image to be detected, and only the target subject label is included, the electronic device may obtain multiple subject labels obtained by performing target detection on the image to be detected, obtain the subject area corresponding to each subject label, and the electronic device may compare each subject. The area of the area determines the image label based on the area of the body area. The electronic device can sort the body tags according to the area of the body area, and then select the body tags corresponding to the serial number as the image tags according to the number of output image tags. For example, if the number of image tags output by the electronic device is two, the electronic device may select a body tag with the largest body area area and the second largest body area area as the image label.

In the method in the embodiment of the present application, the electronic device may select a target subject label or a target scene label as the image label of the image to be detected, which is helpful for quickly and accurately determining the image label.

In one embodiment, determining the image tag of the image to be detected according to the target scene tag and / or the target subject tag includes: if the obtained target scene tag is different from the target subject tag, obtaining a target subject region corresponding to the target subject tag and the target subject tag Detect the area ratio of the image; determine the image label based on the area ratio.

After the electronic device performs scene recognition on the to-be-detected image to obtain a target scene tag, and performs target detection on the to-be-detected image to obtain a target subject tag, a tag is selected as the image tag from the target subject tag and the target scene tag. When the target scene label is the same as the target subject label, the target subject label is directly obtained as the image label. For example, when the target subject label and the target scene label are both "blue sky", the electronic device outputs an image label as "blue sky". When the target subject label is different from the target scene label, the electronic device may obtain a target subject area corresponding to the target subject label, and determine an image tag of the image to be detected according to an area ratio of the area of the target subject area to an area of the image to be detected.

In one embodiment, determining the image label of the image to be detected according to the area ratio includes: if the area ratio is lower than the first threshold, using the target scene label as the image label; if the area ratio is not lower than the first threshold, obtaining the target scene label's The first confidence level and the second confidence level of the target subject label; determine the image label according to the first confidence level and the second confidence level.

When the area ratio of the target subject area to the image to be detected is lower than the first threshold, it may be determined that the area of the target subject area in the image to be detected is small, and the electronic device may select a target scene tag as the image tag. When the subject area in the image to be detected is small, the electronic device performs a target detection on the subject area and the detection accuracy is low. Therefore, when the subject area in the image to be detected is small, the electronic device can select a target scene label obtained by image classification. As image tag. When the area ratio of the target subject area to the image to be detected is not lower than the first threshold, the electronic device may obtain the first confidence level of the target scene label and the second confidence level of the target subject label, and according to the first confidence level, The image tag is determined by the degree of confidence and the second degree of confidence. The electronic device may select a tag with a high degree of confidence among the first and second degrees of confidence as the image tag. For example, the target scene label output of the electronic device to be detected by the electronic device has a “beach” confidence of 90%, the target subject label has a “cat” confidence of 95%, and the electronic device detects that the target subject area corresponding to the target subject label and the image to be detected If the area ratio is greater than 1/3, the electronic device selects the target subject tag "cat" with high confidence as the image tag of the electronic device. The first threshold may be a value set by a user or a value set by an electronic device, for example, 1/2 or 1/3.

In the method in the embodiment of the present application, when the target subject label obtained by the electronic device is different from the target scene label, the image label may be determined according to the area of the target subject area, and the image label output by the image to be detected is more accurate.

In an embodiment, the method further includes:

Operation 112: Obtain a target subject region corresponding to the target subject tag.

Operation 114: Acquire an image processing strategy corresponding to the target subject label.

Operation 116: Perform corresponding image processing on the target subject area according to the image processing strategy.

The electronic device may perform partial image processing on the image to be detected. Optionally, after the electronic device obtains the target subject label of the image to be detected, the electronic device may identify a target subject region corresponding to the target subject label in the image to be detected. The electronic device may also obtain an image processing strategy corresponding to the target subject tag, and perform image processing on the target subject area in the image to be detected according to the image processing strategy. For example, when the target subject area is "Portrait", the electronic device may perform beauty treatment on the "Portrait"; when the target subject area is "Gourmet", the electronic device may perform saturation enhancement processing on the "Gourmet"; when the target subject is When the area is "text", the electronic device can sharpen and calibrate the "text". The image processing strategy corresponding to the above body tag may be pre-stored in the electronic device or stored in the server. The electronic device can directly obtain a pre-stored image processing strategy corresponding to the target subject tag or search the server for an image processing strategy corresponding to the target subject tag.

In the method of the embodiment of the present application, the electronic device can perform image processing on the target subject area in the image to be detected, which can not only realize local processing of the image, but also process the target subject area according to the image processing strategy, and the image processing method is more intelligent. .

In an embodiment, the method further includes:

In operation 118, the number of browsing times of each image under the same image label is obtained.

Operation 120: Determine a display order of each image according to the number of views.

The electronic device can group the images according to the image tags, and group the images corresponding to the same image tag into a group. The electronic device can display the images corresponding to each group on the electronic device interface, so that users can directly view the images according to the image tags. The electronic device may also count the browsing times of each image under the same image tag. The above browsing times may be the cumulative browsing times or the browsing times counted over a period of time. The electronic device may determine the display order of the foregoing images according to the number of views of each image. Optionally, the electronic device may sequentially display the images in order of the number of browsing times from high to low; the electronic device may also sequentially display the images in order of the number of browsing times from low to high.

In the method of the embodiment of the present application, by counting the number of views of each image under the same label, the display order of each image by the electronic device can be determined, and the method of displaying images is more intelligent.

In one embodiment, an image processing method includes:

(1) Acquire an image to be detected.

(2) Perform scene recognition on the image to be detected, and obtain a target scene label of the image to be detected.

(3) Perform target detection on the image to be detected, and obtain a target subject label of the image to be detected.

(4) Determine the image label of the image to be detected according to the target scene label and / or the target subject label.

(5) Perform image processing on the image to be detected according to the image label.

In one embodiment, determining the image tag of the image to be detected according to the target scene tag and / or the target subject tag includes: if the target scene tag cannot be obtained, using the target subject tag as the image tag; if the target subject tag cannot be obtained, changing The target scene label is used as the image label.

In one embodiment, the method further includes: acquiring a target subject region corresponding to the target subject tag; acquiring an image processing strategy corresponding to the target subject tag; and performing corresponding image processing on the target subject region according to the image processing strategy.

In one embodiment, the above method further includes: obtaining the number of views of each image under the same image tag; and determining the display order of each image according to the number of views.

It should be understood that although the operations in the above flowchart are sequentially displayed in accordance with the directions of the arrows, these operations are not necessarily performed in the order indicated by the arrows. Unless explicitly stated in this article, there is no strict order in which these operations can be performed, and these operations can be performed in other orders. In addition, at least a part of the operations in the above flowchart may include multiple sub-operations or multiple stages. These sub-operations or stages are not necessarily performed at the same time, but may be performed at different times. The execution order is not necessarily sequential, but may be performed in turn or alternately with at least a part of another operation or a sub-operation or stage of another operation.

FIG. 4 is a structural block diagram of an image processing apparatus in an embodiment. As shown in FIG. 4, an image processing apparatus includes:

The first acquiring module 402 is configured to acquire an image to be detected.

The recognition module 404 is configured to perform scene recognition on an image to be detected, and obtain a target scene label of the image to be detected.

The detection module 406 is configured to perform target detection on the image to be detected, and obtain a target subject label of the image to be detected.

A determining module 408 is configured to determine an image label of an image to be detected according to a target scene label and / or a target subject label.

The processing module 410 is configured to perform image processing on an image to be detected according to an image tag.

In one embodiment, the recognition module 404 performs scene recognition on the image to be detected, and obtains target scene tags of the image to be detected includes: scene recognition on the image to be detected, obtaining multiple scene tags corresponding to the image to be detected, and confidence of each scene tag. ; Determine the target scene label based on the confidence.

In one embodiment, the determining module 408 determines the image tag of the image to be detected according to the target scene tag and / or the target subject tag, including: if the target scene tag cannot be obtained, using the target subject tag as the image tag; if the target subject cannot be obtained Label, using the target scene label as the image label.

In one embodiment, the determining module 408 determines the image tag of the image to be detected according to the target scene tag and / or the target subject tag, including: if the obtained target scene tag is different from the target subject tag, obtaining a target subject corresponding to the target subject tag The area ratio of the area to the image to be detected; the image label is determined according to the area ratio.

In one embodiment, the determining module 408 determines the image label of the image to be detected according to the area ratio. If the area ratio is lower than the first threshold, the target scene label is used as the image label. If the area ratio is not lower than the first threshold, the target is obtained. The first confidence level of the scene label and the second confidence level of the target subject label; the image label is determined according to the first confidence level and the second confidence level.

FIG. 5 is a structural block diagram of an image processing apparatus in another embodiment. As shown in FIG. 5, an image processing apparatus includes: a first acquisition module 502, an identification module 504, a detection module 506, a determination module 508, a processing module 510, and a second acquisition module 512. The first acquisition module 502, the identification module 504, the detection module 506, the determination module 508, and the processing module 510 have the same functions as the corresponding modules in FIG. 4.

The second acquisition module 512 is configured to acquire a target subject region corresponding to the target subject tag; and acquire an image processing strategy corresponding to the target subject tag.

The processing module 510 is further configured to perform corresponding image processing on the target subject area according to the image processing strategy.

FIG. 6 is a structural block diagram of an image processing apparatus in another embodiment. As shown in FIG. 6, an image processing device includes: a first acquisition module 602, an identification module 604, a detection module 606, a determination module 608, a processing module 610, a statistics module 612, and a display module 614. The first acquisition module 602, the identification module 604, the detection module 606, the determination module 608, and the processing module 610 have the same functions as the corresponding modules in FIG. 4.

The statistics module 612 is configured to obtain the browsing times of each image under the same image label.

A display module 614 is configured to determine a display order of each image according to the number of views.

The division of each module in the above image processing apparatus is for illustration only. In other embodiments, the image processing apparatus may be divided into different modules as needed to complete all or part of the functions of the above image processing apparatus.

For the specific limitation of the image processing device, refer to the foregoing limitation on the image processing method, and details are not described herein again. Each module in the image processing apparatus may be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules may be embedded in the hardware in or independent of the processor in the computer device, or may be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

The implementation of each module in the image processing apparatus provided in the embodiments of the present application may be in the form of a computer program. The computer program can be run on a terminal or server. The program module constituted by the computer program can be stored in the memory of the terminal or server. When the computer program is executed by a processor, the operations of the image processing method described in the embodiments of the present application are implemented.

FIG. 7 is a schematic diagram of an internal structure of an electronic device in an embodiment. As shown in FIG. 7, the electronic device includes a processor, a memory, and a network interface connected through a system bus. The processor is used to provide computing and control capabilities to support the operation of the entire electronic device. The memory is used to store data, programs, and the like. At least one computer program is stored on the memory, and the computer program can be executed by a processor to implement the image processing method applicable to the electronic device provided in the embodiments of the present application. The memory may include a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The computer program can be executed by a processor to implement an image processing method provided by each of the following embodiments. The internal memory provides a cached operating environment for operating system computer programs in a non-volatile storage medium. The network interface may be an Ethernet card or a wireless network card, and is used to communicate with external electronic devices. The electronic device may be a mobile phone, a tablet computer, or a personal digital assistant or a wearable device.

An embodiment of the present application further provides a computer-readable storage medium. One or more non-volatile computer-readable storage media containing computer-executable instructions, when the computer-executable instructions are executed by one or more processors, causing the processors to execute the image processing methods described in the embodiments of the present application operating.

The embodiment of the present application further provides a computer program product containing instructions, which when executed on a computer, causes the computer to perform operations of the image processing method described in the embodiment of the present application.

An embodiment of the present application further provides an electronic device. The above electronic device includes an image processing circuit. The image processing circuit may be implemented by hardware and / or software components, and may include various processing units that define an ISP (Image Signal Processing) pipeline. FIG. 8 is a schematic diagram of an image processing circuit in one embodiment. As shown in FIG. 8, for ease of description, only aspects of the image processing technology related to the embodiments of the present application are shown.

As shown in FIG. 8, the image processing circuit includes a first ISP processor 830, a second ISP processor 840, and a control logic 850. The first camera 810 includes one or more first lenses 812 and a first image sensor 814. The first image sensor 814 may include a color filter array (such as a Bayer filter). The first image sensor 814 may obtain light intensity and wavelength information captured by each imaging pixel of the first image sensor 814, and provide information that may be captured by the first ISP. A set of image data processed by the processor 830. The second camera 820 includes one or more second lenses 822 and a second image sensor 824. The second image sensor 824 may include a color filter array (such as a Bayer filter). The second image sensor 824 may obtain light intensity and wavelength information captured by each imaging pixel of the second image sensor 824, and provide information that may be captured by the second ISP. A set of image data processed by the processor 840.

The first image collected by the first camera 810 is transmitted to the first ISP processor 830 for processing. After the first ISP processor 830 processes the first image, the first image statistical data (such as the brightness of the image and the contrast value of the image) can be processed. , Image color, etc.) to the control logic 850. The control logic 850 can determine the control parameters of the first camera 810 according to the statistical data, so that the first camera 810 can perform operations such as autofocus and automatic exposure according to the control parameters. The first image may be stored in the image memory 860 after being processed by the first ISP processor 830, and the first ISP processor 830 may also read the image stored in the image memory 860 for processing. In addition, the first image may be directly sent to the display 870 for display after being processed by the ISP processor 830, and the display 870 may also read the image in the image memory 860 for display.

The first ISP processor 830 processes the image data pixel by pixel in a variety of formats. For example, each image pixel may have a bit depth of 8, 10, 12, or 14 bits, and the first ISP processor 830 may perform one or more image processing operations on the image data and collect statistical information about the image data. The image processing operations may be performed with the same or different bit depth accuracy.

The image memory 860 may be a part of a memory device, a storage device, or a separate dedicated memory in an electronic device, and may include a DMA (Direct Memory Access) feature.

When receiving the interface from the first image sensor 814, the first ISP processor 830 may perform one or more image processing operations, such as time-domain filtering. The processed image data may be sent to the image memory 860 for further processing before being displayed. The first ISP processor 830 receives the processing data from the image memory 860, and performs the image data processing in the RGB and YCbCr color spaces on the processing data. The image data processed by the first ISP processor 830 may be output to the display 870 for viewing by a user and / or further processed by a graphics engine or a GPU (Graphics Processing Unit). In addition, the output of the first ISP processor 830 may also be sent to the image memory 860, and the display 870 may read image data from the image memory 860. In one embodiment, the image memory 860 may be configured to implement one or more frame buffers.

The statistical data determined by the first ISP processor 830 may be sent to the control logic 850. For example, the statistical data may include statistical information of the first image sensor 814 such as automatic exposure, automatic white balance, automatic focus, flicker detection, black level compensation, and first lens 812 shading correction. The control logic 850 may include a processor and / or a microcontroller that executes one or more routines (such as firmware), and the one or more routines may determine the control parameters and the first parameters of the first camera 810 according to the received statistical data. A control parameter of an ISP processor 830. For example, the control parameters of the first camera 810 may include gain, integration time of exposure control, image stabilization parameters, flash control parameters, control parameters of the first lens 812 (for example, focal length for focusing or zooming), or a combination of these parameters. The ISP control parameters may include a gain level and a color correction matrix for automatic white balance and color adjustment (eg, during RGB processing), and a first lens 812 shading correction parameter.

Similarly, the second image collected by the second camera 820 is transmitted to the second ISP processor 840 for processing. After the second ISP processor 840 processes the first image, statistical data of the second image (such as image brightness, image (The contrast value of the image, the color of the image, etc.) are sent to the control logic 850. The control logic 850 can determine the control parameters of the second camera 820 according to the statistical data, so that the second camera 820 can perform operations such as autofocus and automatic exposure according to the control parameters . The second image may be stored in the image memory 860 after being processed by the second ISP processor 840, and the second ISP processor 840 may also read the image stored in the image memory 860 for processing. In addition, the second image may be directly sent to the display 870 for display after being processed by the ISP processor 840, and the display 870 may also read the image in the image memory 860 for display. The second camera 820 and the second ISP processor 840 may also implement processing operations as described by the first camera 810 and the first ISP processor 830. The electronic device can implement the image processing method described in the embodiment of the present application according to the image processing technology.

Any reference to memory, storage, database, or other media used in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM), which is used as external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDR, SDRAM), enhanced SDRAM (ESDRAM), synchronous Link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

The above embodiments only express several implementation manners of the present application, and their descriptions are more specific and detailed, but they cannot be understood as limiting the patent scope of the present application. It should be noted that, for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of this application patent shall be subject to the appended claims.

Claims

An image processing method, comprising:

Obtaining images to be detected;

Performing scene recognition on the image to be detected, and obtaining a target scene label of the image to be detected;

Performing target detection on the image to be detected, and obtaining a target subject label of the image to be detected;

Determining an image label of the image to be detected according to at least one of the target scene label and the target subject label; and

Performing image processing on the image to be detected according to the image label.
The method according to claim 1, wherein the performing scene recognition on the image to be detected and obtaining a target scene label of the image to be detected comprises:

Performing scene recognition on the image to be detected, and obtaining a plurality of scene tags corresponding to the image to be detected and the confidence of each scene tag; and

Determining the target scene label according to the confidence.
The method according to claim 1, wherein determining the image tag of the image to be detected according to at least one of the target scene tag and the target subject tag comprises:

If the target scene tag cannot be obtained, use the target subject tag as the image tag; and

If the target subject label cannot be obtained, the target scene label is used as the image label.
The method according to claim 1, wherein determining the image tag of the image to be detected according to at least one of the target scene tag and the target subject tag comprises:

If the acquired target scene label is different from the target subject label, obtaining an area ratio of a target subject area corresponding to the target subject label to the image to be detected; and

Determining the image label according to the area ratio.
The method according to claim 4, wherein the determining an image label of the image to be detected according to the area ratio comprises:

If the area ratio is lower than a first threshold, using the target scene label as the image label;

If the area ratio is not lower than a first threshold, obtaining a first confidence level of the target scene label and a second confidence level of the target subject label; and

Determining the image label according to the first confidence level and the second confidence level.
The method according to any one of claims 1 to 5, wherein the method further comprises:

Obtaining a target subject region corresponding to the target subject tag;

Acquiring an image processing strategy corresponding to the target subject tag; and

Corresponding image processing is performed on the target subject area according to the image processing strategy.
The method according to any one of claims 1 to 5, wherein the method further comprises:

Get the number of views of each image under the same image tag; and

The display order of each image is determined according to the number of views.
An image processing device, comprising:

A first acquisition module, configured to acquire an image to be detected;

A recognition module, configured to perform scene recognition on the image to be detected, and obtain a target scene label of the image to be detected;

A detection module, configured to perform target detection on the image to be detected, and obtain a target subject label of the image to be detected;

A determining module, configured to determine an image label of the image to be detected according to the target scene label and / or the target subject label; and

A processing module, configured to perform image processing on the image to be detected according to the image label.
An electronic device includes a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor causes the processor to perform the following operations:

Obtaining images to be detected;

Performing scene recognition on the image to be detected, and obtaining a target scene label of the image to be detected;

Performing target detection on the image to be detected, and obtaining a target subject label of the image to be detected;

Determining an image label of the image to be detected according to at least one of the target scene label and the target subject label; and

Performing image processing on the image to be detected according to the image label.
The electronic device according to claim 9, wherein the processor is further configured to execute:

Performing scene recognition on the image to be detected, and obtaining a plurality of scene tags corresponding to the image to be detected and the confidence of each scene tag; and

Determining the target scene label according to the confidence.
The electronic device according to claim 9, wherein the processor is further configured to execute:

If the target scene tag cannot be obtained, use the target subject tag as the image tag; and

If the target subject label cannot be obtained, the target scene label is used as the image label.
The electronic device according to claim 9, wherein the processor is further configured to execute:

If the obtained target scene label is different from the target subject label, obtaining an area ratio of a target subject area corresponding to the target subject label to the image to be detected;

Determining the image label according to the area ratio.
The electronic device according to claim 12, wherein the processor is further configured to execute:

If the area ratio is lower than a first threshold, using the target scene label as the image label;

If the area ratio is not lower than a first threshold, obtaining a first confidence level of the target scene label and a second confidence level of the target subject label; and

Determining the image label according to the first confidence level and the second confidence level.
The electronic device according to any one of claims 9 to 13, wherein the processor is further configured to execute:

Obtaining a target subject region corresponding to the target subject tag;

Acquiring an image processing strategy corresponding to the target subject tag; and

Corresponding image processing is performed on the target subject area according to the image processing strategy.
The electronic device according to any one of claims 9 to 13, wherein the processor is further configured to execute:

Get the number of views of each image under the same image tag; and

The display order of each image is determined according to the number of views.
A computer-readable storage medium having stored thereon a computer program, characterized in that, when the computer program is executed by a processor, the operations of the method according to any one of claims 1 to 7 are realized.