CN113177438A - Image processing method, apparatus and storage medium - Google Patents
Image processing method, apparatus and storage medium Download PDFInfo
- Publication number
- CN113177438A CN113177438A CN202110362328.8A CN202110362328A CN113177438A CN 113177438 A CN113177438 A CN 113177438A CN 202110362328 A CN202110362328 A CN 202110362328A CN 113177438 A CN113177438 A CN 113177438A
- Authority
- CN
- China
- Prior art keywords
- displayed
- image
- shooting
- target
- display data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 27
- 238000012545 processing Methods 0.000 claims abstract description 53
- 238000000034 method Methods 0.000 claims abstract description 8
- 238000004422 calculation algorithm Methods 0.000 claims description 38
- 238000003062 neural network model Methods 0.000 claims description 27
- 230000000306 recurrent effect Effects 0.000 claims description 27
- 238000005286 illumination Methods 0.000 abstract description 5
- 238000012544 monitoring process Methods 0.000 description 14
- 238000010586 diagram Methods 0.000 description 12
- 230000009467 reduction Effects 0.000 description 10
- 238000004364 calculation method Methods 0.000 description 7
- 238000004590 computer program Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 238000012937 correction Methods 0.000 description 5
- 238000012986 modification Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 230000003247 decreasing effect Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 239000000463 material Substances 0.000 description 2
- 239000013077 target material Substances 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005375 photometry Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/757—Matching configurations of points or features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/48—Matching video sequences
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Studio Devices (AREA)
Abstract
The invention discloses an image processing method, an image processing device and a storage medium, wherein the method comprises the following steps: after receiving an image frame to be displayed, adjusting the image frame to be displayed according to preset first image display data, and outputting the adjusted image frame to be displayed; acquiring the adjusted shooting characteristic information of the image frame to be displayed; calculating target image display data according to the shooting characteristic information; the first image display data is updated according to the target image display data. The invention realizes the purpose of keeping the definition of the monitored target in real time under the scene with large light and shade change such as low light illumination, high light illumination, backlight and the like, and improves the use scene of the camera.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to an image processing method, an image processing apparatus, and a storage medium.
Background
The security cameras of the monitoring type have main monitoring targets in different scenes, and are required to be capable of accurately monitoring and storing information related to the monitoring targets. In order to improve the quality of the monitoring video Image, technicians introduce an Image Signal Processing (ISP) algorithm into the camera. However, the conventional Image Signal Process (ISP) algorithm is a static algorithm for a set gain and global photometry, and cannot perform parameter adjustment on a moving monitoring target in a scene in a targeted manner, so that a camera cannot effectively record the details of the screen of the monitoring target.
Disclosure of Invention
The embodiment of the application aims to solve the problem that a camera cannot effectively record the picture details of a monitoring target due to the fact that a traditional image signal processing algorithm cannot perform parameter adjustment on the monitoring target moving in a scene in a targeted mode.
The embodiment of the application provides an image processing method, which comprises the following steps:
after receiving an image frame to be displayed, adjusting the image frame to be displayed according to preset first image display data, and outputting the adjusted image frame to be displayed, wherein the first image display data comprises an image processing algorithm and image display parameters;
acquiring the adjusted shooting characteristic information of the image frame to be displayed; the shooting characteristic information comprises a scene type and a shooting target of a shooting scene;
calculating target image display data according to the shooting characteristic information, wherein the target image display data comprises a target image processing algorithm and target image display parameters;
and updating the first image display data according to the target image display data.
In an embodiment, the step of adjusting the image frame to be displayed according to preset first image display data includes:
and adjusting the image display parameters of the image frame to be displayed according to the first image display data.
In an embodiment, the step of obtaining the adjusted shooting feature information of the image frame to be displayed includes:
inputting the adjusted image frame to be displayed into a preset image recognition model so as to recognize at least two preset shooting characteristic information from the adjusted image frame to be displayed;
and determining the adjusted shooting characteristic information of the image frame to be displayed according to the identified preset shooting characteristic information.
In an embodiment, the step of determining the adjusted shooting feature information of the image frame to be displayed according to each piece of the identified preset shooting feature information includes:
acquiring the confidence of each piece of the identified preset shooting characteristic information;
and determining the preset shooting characteristic information with the maximum confidence coefficient as the adjusted shooting characteristic information of the image frame to be displayed.
In one embodiment, the step of calculating target image display data according to the shooting feature information includes:
detecting whether the shooting target exists in the image frame to be displayed;
and when the shooting target exists in the image frame to be displayed, determining target image display data matched with the shooting characteristic information according to a preset recurrent neural network model.
In an embodiment, after the step of detecting whether the shooting target exists in the image frame to be displayed, the method further includes:
when the shooting target does not exist in the image frame to be displayed, acquiring preset second image display data;
determining the second image display data as the target image display data.
In an embodiment, when it is detected that the shooting target exists in the image frame to be displayed, the step of determining target image display data matched with the shooting feature information according to a preset recurrent neural network model further includes:
when the shooting target is detected to exist in the image frame to be displayed and the shooting target is single, the shooting characteristic information is input into the recurrent neural network model, and the target image display data is output.
In an embodiment, when it is detected that the shooting target exists in the image frame to be displayed, the step of determining target image display data matched with the shooting feature information according to a preset recurrent neural network model further includes:
when the shooting targets are detected to be at least two in the image frame to be displayed, marking the shooting target which occupies the largest area proportion of the image frame to be displayed in the at least two shooting targets;
inputting the scene type and the marked shooting target into the recurrent neural network model, and outputting the target image display data.
Further, to achieve the above object, the present invention also provides an image processing apparatus comprising: the image processing system comprises a memory, a processor and an image processing program which is stored on the memory and can run on the processor, wherein the image processing program realizes the steps of the image processing method when being executed by the processor.
Further, to achieve the above object, the present invention also provides a storage medium having stored thereon an image processing program which, when executed by a processor, realizes the steps of the above-described image processing method.
The technical scheme of the image processing method, the image processing device and the storage medium provided by the embodiment of the application at least has the following technical effects or advantages:
the technical scheme that after an image frame to be displayed is received, the image frame to be displayed is adjusted according to preset first image display data, the adjusted image frame to be displayed is output, shooting characteristic information of the adjusted image frame to be displayed is obtained, target image display data is calculated according to the shooting characteristic information, and the first image display data is updated according to the target image display data is adopted, so that the problem that a traditional image signal processing algorithm cannot perform parameter adjustment on a monitoring target moving in a scene in a targeted manner, and therefore the camera cannot effectively record picture details of the monitoring target is solved, the definition of the monitoring target is kept in real time under a scene with large brightness and darkness changes, such as low illumination, high illumination, backlight and the like, the use scene of the camera is improved, and the effectiveness of the monitoring information is improved.
Drawings
FIG. 1 is a schematic diagram of a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a first embodiment of an image processing method according to the present invention;
FIG. 3 is a set of comparison images output after a conventional image signal processing algorithm and the image processing method provided by the present invention process the same image frame to be displayed;
FIG. 4 is a diagram of another set of comparison images output after the same image frame to be displayed is processed by the conventional image signal processing algorithm and the image processing method provided by the present invention;
FIG. 5 is a flowchart illustrating a second embodiment of an image processing method according to the present invention;
FIG. 6 is a flowchart illustrating a third embodiment of an image processing method according to the present invention;
Detailed Description
For a better understanding of the above technical solutions, exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
As shown in fig. 1, fig. 1 is a schematic structural diagram of a hardware operating environment according to an embodiment of the present invention.
Fig. 1 may be a schematic structural diagram of a hardware operating environment of the image processing apparatus.
As shown in fig. 1, the image processing apparatus may include: a processor 1001, such as a CPU, a memory 1005, a user interface 1003, a network interface 1004, a communication bus 1002. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (e.g., a magnetic disk memory). The memory 1005 may alternatively be a storage device separate from the processor 1001.
Those skilled in the art will appreciate that the image processing apparatus configuration shown in fig. 1 is not intended to be limiting of the image processing apparatus and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.
As shown in fig. 1, a memory 1005, which is a storage medium, may include therein an operating system, a network communication module, a user interface module, and an image processing program. Among them, the operating system is a program that manages and controls hardware and software resources of the image processing apparatus, an image processing program, and the execution of other software or programs.
In the image processing apparatus shown in fig. 1, the user interface 1003 is mainly used for connecting a terminal, and performing data communication with the terminal; the network interface 1004 is mainly used for the background server and performs data communication with the background server; the processor 1001 may be used to invoke an image processing program stored in the memory 1005.
In the present embodiment, an image processing apparatus includes: a memory 1005, a processor 1001, and an image processing program stored on the memory 1005 and executable on the processor, wherein:
when the processor 1001 calls the image processing program stored in the memory 1005, the following operations are performed:
after receiving an image frame to be displayed, adjusting the image frame to be displayed according to preset first image display data, and outputting the adjusted image frame to be displayed, wherein the first image display data comprises an image processing algorithm and image display parameters;
acquiring the adjusted shooting characteristic information of the image frame to be displayed; the shooting characteristic information comprises a scene type and a shooting target of a shooting scene;
calculating target image display data according to the shooting characteristic information, wherein the target image display data comprises a target image processing algorithm and target image display parameters;
and updating the first image display data according to the target image display data.
When the processor 1001 calls the image processing program stored in the memory 1005, the following operations are also performed:
and adjusting the image display parameters of the image frame to be displayed according to the first image display data.
When the processor 1001 calls the image processing program stored in the memory 1005, the following operations are also performed:
inputting the adjusted image frame to be displayed into a preset image recognition model so as to recognize at least two preset shooting characteristic information from the adjusted image frame to be displayed;
and determining the adjusted shooting characteristic information of the image frame to be displayed according to the identified preset shooting characteristic information.
When the processor 1001 calls the image processing program stored in the memory 1005, the following operations are also performed:
acquiring the confidence of each piece of the identified preset shooting characteristic information;
and determining the preset shooting characteristic information with the maximum confidence coefficient as the adjusted shooting characteristic information of the image frame to be displayed.
When the processor 1001 calls the image processing program stored in the memory 1005, the following operations are also performed:
detecting whether the shooting target exists in the image frame to be displayed;
and when the shooting target exists in the image frame to be displayed, determining target image display data matched with the shooting characteristic information according to a preset recurrent neural network model.
When the processor 1001 calls the image processing program stored in the memory 1005, the following operations are also performed:
when the shooting target does not exist in the image frame to be displayed, acquiring preset second image display data;
determining the second image display data as the target image display data.
When the processor 1001 calls the image processing program stored in the memory 1005, the following operations are also performed:
when the shooting target is detected to exist in the image frame to be displayed and the shooting target is single, the shooting characteristic information is input into the recurrent neural network model, and the target image display data is output.
When the processor 1001 calls the image processing program stored in the memory 1005, the following operations are also performed:
when the shooting targets are detected to be at least two in the image frame to be displayed, marking the shooting target which occupies the largest area proportion of the image frame to be displayed in the at least two shooting targets;
inputting the scene type and the marked shooting target into the recurrent neural network model, and outputting the target image display data.
The embodiments of the present invention provide an embodiment of an image processing method, and it should be noted that, although a logical sequence is shown in the flowchart, in some cases, the steps shown or described may be executed in a sequence different from that here, and the image processing method is applied to the field of image processing, and is particularly applicable to output processing of monitoring videos, for example, processing video resources collected by a video capture device such as a video camera, and the like.
As shown in fig. 2, in a first embodiment of the present application, an image processing method of the present application includes the steps of:
step S210: after receiving an image frame to be displayed, adjusting the image frame to be displayed according to preset first image display data, and outputting the adjusted image frame to be displayed.
The image frame to be displayed refers to a video frame needing to be output and displayed in the collected video resources, and the video resources are collected by a video collecting device; the first image display data is preset and stored, and comprises an image processing algorithm and image display parameters; the image processing algorithm comprises an automatic white balance algorithm, an automatic exposure algorithm and other algorithms, and the image display parameters comprise gamma value, brightness, contrast, sharpness, wide dynamic, color correction, time domain noise reduction, space domain noise reduction and other parameters.
In the present embodiment, the received image frame to be displayed is an unprocessed video frame, and can be understood as an original video frame. After receiving the image frame to be displayed, performing image processing on the image frame to be displayed, specifically, adjusting image display parameters of the image frame to be displayed according to the first image display data, where the image display parameters refer to parameters corresponding to the image frame to be displayed when the image frame is displayed, and include color temperature parameters, exposure parameters, brightness, contrast and other parameters. For example, the color temperature parameter of the image frame to be displayed is increased or decreased by adopting an automatic white balance algorithm, the exposure parameter of the image frame to be displayed is increased or decreased by adopting an automatic exposure algorithm, and the current gamma value, brightness, contrast, sharpness, wide dynamic, color correction, time domain noise reduction, spatial domain noise reduction and other parameters of the image frame to be displayed are sequentially adjusted (increased or decreased) to be the stored gamma value, brightness, contrast, sharpness, wide dynamic, color correction, time domain noise reduction, spatial domain noise reduction and other parameters based on the stored gamma value, brightness, contrast, sharpness, wide dynamic, color correction, time domain noise reduction, spatial domain noise reduction and other parameters. And after the frame to be displayed is adjusted, outputting and displaying the adjusted frame to be displayed.
Furthermore, the user can clearly see the details of the shooting target through the adjusted image frame to be displayed. As shown in fig. 3, fig. 3 is an image frame to be displayed processed and output in a backlight scene, the image frame to be displayed is a video frame including a person, an image a is a result of processing the image frame to be displayed by a conventional Image Signal Processing (ISP) algorithm, and a face area cannot be seen clearly due to a large-range high-illuminance area, underexposure of an indoor area and the face area, and outdoor clarity, and an image B is a result of adjusting the image frame to be displayed according to the present invention.
Step 220: and acquiring the adjusted shooting characteristic information of the image frame to be displayed.
And acquiring shooting characteristic information from the adjusted image to be displayed according to the output adjusted image to be displayed, and specifically acquiring the shooting characteristic information from the adjusted image to be displayed based on a preset image recognition model. The shooting characteristic information comprises a scene type and a shooting target of a shooting scene, wherein the shooting scene comprises multiple dimensionality types such as weather (such as rainy days, snowy days, fog days and the like), sunshine, light direction and the like; a photographic object refers to an object or thing that can be moved and a classification of the object or thing, for example, the object or thing includes a person, an animal, a vehicle, and the like.
Step 230: and calculating target image display data according to the shooting characteristic information.
In this embodiment, after acquiring the shooting feature information from the adjusted image to be displayed, image display data for adjusting the next image frame to be displayed is calculated by using a preset recurrent neural network model according to the shooting feature information, where the image display data is referred to as target image display data. The target image display data comprises a target image processing algorithm and target image display parameters, the target image processing algorithm comprises an automatic white balance algorithm, an automatic exposure algorithm and other algorithms, and the target image display parameters comprise gamma values, brightness, contrast, sharpness, wide dynamic, color correction, time domain noise reduction, space domain noise reduction and other parameters. And outputting the next image frame to be displayed after the adjusted image frame to be displayed is output. And if the next image frame to be displayed needs to be output and displayed, adjusting the image display parameters of the next image frame to be displayed by using target image display data, and then outputting and displaying.
Step 240: and updating the first image display data according to the target image display data.
Updating the first image display data according to the target image display data specifically is to replace the first image display data with the target image display data, for example, the first image display data is M0, the target image display data is M1, after the first image display data is replaced with the target image display data, M0 is changed to M1, and M1 is the updated first image display data.
And then, when outputting each adjusted image frame to be displayed, acquiring shooting characteristic information from the adjusted image frame to be displayed, calculating target image display data for adjusting the next image frame to be displayed, and updating and adjusting the target image display data of the previous image frame to be displayed by adopting the calculated target image display data. For example, the image frames to be displayed in the video resource are respectively image 1, image 2 and image 3 in sequence, and the image display parameters of the image 1 are adjusted by using the stored first image display data, and then the adjusted image 1 is output; then acquiring 1 st shooting characteristic information from the adjusted image 1, calculating 1 st target image display data of the adjusted image 2 according to the 1 st shooting characteristic information, replacing the first image display data with the 1 st target image display data, namely, changing the first image display data into the 1 st target image display data, then adjusting the image display parameters of the image 2 by using the 1 st target image display data before outputting the image 2, and outputting the adjusted image 2; then, 2 nd shooting characteristic information is obtained from the adjusted image 2, 2 nd target image display parameters of the adjusted image 3 are calculated according to the 2 nd shooting characteristic information, the 1 st target image display data are replaced by the 2 nd target image display data, namely, the 1 st target image display data are changed into the 2 nd target image display data, then, before the image 3 is output, the 2 nd target image display data are used for adjusting the image display parameters of the image 3, and the adjusted image 3 is output. If more image frames to be displayed need to be output in the video resource, the image frames to be displayed need to be output are sequentially adjusted and the display data of the target image is replaced according to the mode.
It should be noted that the first image display data for adjusting the first image frame to be displayed output in the video resource is preset fixed data, which can be understood as initial image display data, the initial image display data is used to adjust the image display parameters of the first image frame to be displayed, and the adjusted first image frame to be displayed is output, and the initial image display data is replaced next.
According to the technical scheme, after the image frame to be displayed is received, the image frame to be displayed is adjusted according to the preset first image display data, the adjusted image frame to be displayed is output, the shooting characteristic information of the adjusted image frame to be displayed is obtained, the target image display data is calculated according to the shooting characteristic information, and the first image display data is updated according to the target image display data.
As shown in fig. 5, in the second embodiment of the present application, based on the first embodiment, the step S220 includes the following steps:
step S221: inputting the adjusted image frame to be displayed into a preset image recognition model so as to recognize at least two preset shooting characteristic information from the adjusted image frame to be displayed.
Step S222: and determining the adjusted shooting characteristic information of the image frame to be displayed according to the identified preset shooting characteristic information.
In step S222, determining the adjusted shooting feature information of the image frame to be displayed according to each piece of the identified preset shooting feature information specifically includes: and acquiring the confidence coefficient of each piece of recognized preset shooting characteristic information, and determining the preset shooting characteristic information with the maximum confidence coefficient as the adjusted shooting characteristic information of the image frame to be displayed.
In this embodiment, the image recognition model is a pre-established convolutional neural network model, and is used for recognizing the scene type and the shooting target of the shooting scene in the image, and is obtained by training massive shooting scene materials and shooting target materials, after the training of the image recognition model is completed, massive shooting scenes, scene types of the shooting scenes, and shooting targets are stored in the image recognition model, and in the massive shooting scenes, the scene types of the shooting scenes, and the shooting targets, each group of the shooting scenes, the scene types of the shooting scenes, and the shooting targets is referred to as preset shooting feature information. And after the adjusted image frame to be displayed is obtained, taking the adjusted image frame to be displayed as the input of an image recognition model, comparing the input adjusted image frame to be displayed with mass preset shooting characteristic information by the image recognition model, and outputting a comparison result. The comparison result comprises at least two pieces of preset shooting characteristic information recognized from the adjusted image frame to be displayed and the confidence coefficient of each piece of preset shooting characteristic information. The confidence level refers to a probability that the total parameter value falls in a certain area of the sample statistical value, and in this embodiment, the confidence level refers to a probability that the shooting feature information identified from the adjusted image frame to be displayed by the image recognition model is the preset shooting feature information. Further, after the comparison result is output, the confidence degrees of all the preset shooting feature information in the comparison result are sorted, and the preset shooting feature information with the maximum confidence degree is selected as the shooting feature information of the image frame to be displayed.
According to the technical scheme, the adjusted image frame to be displayed is input into the preset image recognition model, so that at least two pieces of preset shooting characteristic information are recognized from the adjusted image frame to be displayed, the confidence of each piece of recognized preset shooting characteristic information is obtained, the preset shooting characteristic information with the maximum confidence is determined as the shooting characteristic information of the adjusted image frame to be displayed, and the accuracy of obtaining the shooting characteristic information is improved.
As shown in fig. 6, in the third embodiment of the present application, based on the first embodiment, step S230 includes the following steps:
step S231: and detecting whether the shooting target exists in the image frame to be displayed.
In this embodiment, the present invention mainly adjusts an image frame to be displayed, which includes a single shooting target. Specifically, after the received image frame to be displayed, whether a shooting target exists in the received image frame to be displayed is detected, if a shooting target exists in the received image frame to be displayed, step S232 is executed, otherwise step S233 is executed. Wherein the shooting target is set by a user in a self-defined way, for example, the user sets that the shooting target is a pedestrian, and step S232 or step S233 is selected to be executed according to whether the received image frame to be displayed is detected to have the pedestrian.
Step S232: and when the shooting target exists in the image frame to be displayed, determining target image display data matched with the shooting characteristic information according to a preset recurrent neural network model.
In this embodiment, the recurrent neural network model is also pre-established, and is used for calculating and adjusting the target image display data of the next output image frame to be displayed according to the shooting feature information. The recurrent neural network model is obtained by training massive scene type materials, shooting target materials, image processing algorithms and image display parameters, and after the recurrent neural network model is trained, massive scene types and data of the shooting targets related to the image processing algorithms and the image display parameters are stored in the recurrent neural network model, wherein the image processing algorithms and the image display parameters related to the scene types and the shooting targets are target image processing algorithms and target image display parameters. For example, if the 1 st shooting feature information (scene type and shooting target) is acquired from the 1 st image frame to be displayed, the recurrent neural network model can calculate target image display data (target image processing algorithm and target image display parameters) matching the 1 st shooting feature information according to the 1 st shooting feature information, and the target image display data is used for adjusting the next output 2 nd image frame to be displayed.
Further, step S232 specifically includes:
step a: when the shooting target is detected to exist in the image frame to be displayed and the shooting target is single, the shooting characteristic information is input into a recurrent neural network model, and the target image display data is output.
In the present embodiment, the number of the photographic objects is acquired when it is detected that the photographic objects exist in the image frame to be displayed. If one shooting target exists, the acquired scene type and the shooting target are used as the input of the recurrent neural network model, the recurrent neural network model outputs a calculation result according to the scene type and the shooting target, and the calculation result comprises a target image processing algorithm and target image display parameters which are related to the scene type and the shooting target, and the matching degree of each group of target image processing algorithms and target image display parameters which are related to the scene type and the shooting target. And taking a group of target image processing algorithms and target image display parameters which are associated with the scene type and the shooting target and have the maximum matching degree in the calculation results as target image display data.
Step b: when the fact that at least two shooting targets exist in the image frame to be displayed is detected, marking the shooting target which occupies the largest area proportion of the image frame to be displayed in the at least two shooting targets.
Step c: inputting the scene type and the marked shooting target into the recurrent neural network model, and outputting the target image display data.
In the present embodiment, the number of the photographic objects is acquired when it is detected that the photographic objects exist in the image frame to be displayed. If at least two shooting targets exist, the area of each shooting target is obtained from the image frame to be displayed, the area proportion of each shooting target in the image frame to be displayed is calculated, the shooting target which occupies the largest area proportion of the image frame to be displayed is marked, the scene type and the marked shooting target are used as the input of a recurrent neural network model, the recurrent neural network model outputs a calculation result according to the scene type and the marked shooting target, and the calculation result comprises a plurality of groups of target image processing algorithms and target image display parameters which are related to the scene type and the marked shooting target, and the matching degree of each group of target image processing algorithms and target image display parameters which are related to the scene type and the marked shooting target. And taking a group of target image processing algorithms and target image display parameters which are associated with the scene type and the marked shooting target and have the maximum matching degree in the calculation results as target image display data.
Step S233: and when the shooting target does not exist in the image frame to be displayed, acquiring preset second image display data.
Step S234: determining the second image display data as the target image display data.
In the present embodiment, after the received image frame to be displayed, if there is no photographing target in the received image frame to be displayed, the second image display data is directly set and stored in advance. When the second image display data is a fixed group of data, the group of data is used as target image display data to adjust the next image frame to be displayed, which needs to be output; and when the second image display data are a plurality of groups of data, acquiring a group of second image display data matched with the scene type according to the scene type, and adjusting the next image frame to be displayed, which needs to be output, by adopting the group of second image display data matched with the scene type.
Specifically, for example, after the 1 st video frame is output, when the 2 nd video frame needs to be output, the image display parameters of the 2 nd video frame are adjusted through the updated first image display data (i.e., the target image display data), after the 2 nd video frame is adjusted, the adjusted 2 nd video frame is encoded (also referred to as image compression) to generate an encoded video frame, and the encoded video frame is transmitted to the display end for display, where the display end can clearly display details of the shooting target in the 2 nd video frame.
As shown in fig. 4, fig. 4 is a to-be-displayed image frame processed and output in a low-light-level scene, where the image C can be seen as the 2 nd video frame before the adjustment, and the image D can be seen as the 2 nd video frame after the adjustment. The image frame to be displayed is a video frame containing a vehicle, the image C is a result of processing the image frame to be displayed by a traditional Image Signal Processing (ISP) algorithm, a video acquisition device such as a camera can turn on a light supplement lamp to shoot under low illumination, and the whole image in the image C is clear but the license plate area reflects light, so that the license plate of the vehicle cannot be seen clearly. And the image D is the result of the adjustment of the image frame to be displayed, after the license plate area is detected, the stored first image display data is used for adjusting the image frame to be displayed, the brightness of the whole image of the adjusted image frame to be displayed is reduced, but the key information license plate is clear and visible.
Further, when more image frames to be displayed are subsequently output, each subsequently output image frame to be displayed is processed by adopting the processing mode of the 2 nd video frame. The first image display data used for adjusting the current image frame to be displayed is obtained by calculating the image frame to be displayed which is already output.
According to the technical scheme, whether the shooting target exists in the image frame to be displayed or not is detected, when the shooting target exists in the image frame to be displayed, target image display data matched with the shooting characteristic information is determined according to a preset recurrent neural network model, when the shooting target does not exist in the image frame to be displayed, preset second image display data is obtained, and the second image display data is determined to be the target image display data, so that the accuracy of the target image display data obtained through calculation is improved.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It should be noted that in the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (10)
1. An image processing method, characterized in that the image processing method comprises:
after receiving an image frame to be displayed, adjusting the image frame to be displayed according to preset first image display data, and outputting the adjusted image frame to be displayed, wherein the first image display data comprises an image processing algorithm and image display parameters;
acquiring the adjusted shooting characteristic information of the image frame to be displayed; the shooting characteristic information comprises a scene type and a shooting target of a shooting scene;
calculating target image display data according to the shooting characteristic information, wherein the target image display data comprises a target image processing algorithm and target image display parameters;
and updating the first image display data according to the target image display data.
2. The image processing method as claimed in claim 1, wherein the step of adjusting the image frame to be displayed according to the preset first image display data comprises:
and adjusting the image display parameters of the image frame to be displayed according to the first image display data.
3. The image processing method according to claim 1, wherein the step of obtaining the adjusted shooting feature information of the image frame to be displayed comprises:
inputting the adjusted image frame to be displayed into a preset image recognition model so as to recognize at least two preset shooting characteristic information from the adjusted image frame to be displayed;
and determining the adjusted shooting characteristic information of the image frame to be displayed according to the identified preset shooting characteristic information.
4. The image processing method according to claim 3, wherein the step of determining the adjusted capturing feature information of the image frame to be displayed according to each of the recognized preset capturing feature information comprises:
acquiring the confidence of each piece of the identified preset shooting characteristic information;
and determining the preset shooting characteristic information with the maximum confidence coefficient as the adjusted shooting characteristic information of the image frame to be displayed.
5. The image processing method according to claim 4, wherein the step of calculating target image display data based on the photographing characteristic information includes:
detecting whether the shooting target exists in the image frame to be displayed;
and when the shooting target exists in the image frame to be displayed, determining target image display data matched with the shooting characteristic information according to a preset recurrent neural network model.
6. The image processing method according to claim 5, wherein after the step of detecting whether the photographic target exists in the image frame to be displayed, the method further comprises:
when the shooting target does not exist in the image frame to be displayed, acquiring preset second image display data;
determining the second image display data as the target image display data.
7. The image processing method according to claim 5, wherein the step of determining target image display data matching the photographing feature information according to a preset recurrent neural network model when the photographing target is detected to exist in the image frame to be displayed, further comprises:
when the shooting target is detected to exist in the image frame to be displayed and the shooting target is single, the shooting characteristic information is input into the recurrent neural network model, and the target image display data is output.
8. The image processing method according to claim 5, wherein the step of determining target image display data matching the photographing feature information according to a preset recurrent neural network model when the photographing target is detected to exist in the image frame to be displayed, further comprises:
when the shooting targets are detected to be at least two in the image frame to be displayed, marking the shooting target which occupies the largest area proportion of the image frame to be displayed in the at least two shooting targets;
inputting the scene type and the marked shooting target into the recurrent neural network model, and outputting the target image display data.
9. An image processing apparatus characterized by comprising: memory, a processor and an image processing program stored on the memory and executable on the processor, the image processing program when executed by the processor implementing the steps of the image processing method according to any one of claims 1 to 8.
10. A storage medium having stored thereon an image processing program which, when executed by a processor, implements the steps of the image processing method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110362328.8A CN113177438B (en) | 2021-04-02 | 2021-04-02 | Image processing method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110362328.8A CN113177438B (en) | 2021-04-02 | 2021-04-02 | Image processing method, device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113177438A true CN113177438A (en) | 2021-07-27 |
| CN113177438B CN113177438B (en) | 2024-05-31 |
Family
ID=76922967
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110362328.8A Active CN113177438B (en) | 2021-04-02 | 2021-04-02 | Image processing method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113177438B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113408380A (en) * | 2021-06-07 | 2021-09-17 | 深圳小湃科技有限公司 | Video image adjusting method, device and storage medium |
| CN113949910A (en) * | 2021-10-15 | 2022-01-18 | 深圳小湃科技有限公司 | Set top box resolution matching method and device, set top box and storage medium |
| CN114038370A (en) * | 2021-11-05 | 2022-02-11 | 深圳Tcl新技术有限公司 | Display parameter adjusting method and device, storage medium and display equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106331518A (en) * | 2016-09-30 | 2017-01-11 | 北京旷视科技有限公司 | Image processing method and device and electronic system |
| CN108900769A (en) * | 2018-07-16 | 2018-11-27 | Oppo广东移动通信有限公司 | Image processing method, device, mobile terminal and computer readable storage medium |
| WO2019120016A1 (en) * | 2017-12-20 | 2019-06-27 | Oppo广东移动通信有限公司 | Image processing method and apparatus, storage medium, and electronic device |
| CN110163810A (en) * | 2019-04-08 | 2019-08-23 | 腾讯科技（深圳）有限公司 | A kind of image processing method, device and terminal |
| WO2020057198A1 (en) * | 2018-09-20 | 2020-03-26 | Oppo广东移动通信有限公司 | Image processing method and device, electronic device and storage medium |
| CN111416950A (en) * | 2020-03-26 | 2020-07-14 | 腾讯科技（深圳）有限公司 | Video processing method and device, storage medium and electronic equipment |
-
2021
- 2021-04-02 CN CN202110362328.8A patent/CN113177438B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106331518A (en) * | 2016-09-30 | 2017-01-11 | 北京旷视科技有限公司 | Image processing method and device and electronic system |
| WO2019120016A1 (en) * | 2017-12-20 | 2019-06-27 | Oppo广东移动通信有限公司 | Image processing method and apparatus, storage medium, and electronic device |
| CN108900769A (en) * | 2018-07-16 | 2018-11-27 | Oppo广东移动通信有限公司 | Image processing method, device, mobile terminal and computer readable storage medium |
| WO2020057198A1 (en) * | 2018-09-20 | 2020-03-26 | Oppo广东移动通信有限公司 | Image processing method and device, electronic device and storage medium |
| CN110163810A (en) * | 2019-04-08 | 2019-08-23 | 腾讯科技（深圳）有限公司 | A kind of image processing method, device and terminal |
| CN111416950A (en) * | 2020-03-26 | 2020-07-14 | 腾讯科技（深圳）有限公司 | Video processing method and device, storage medium and electronic equipment |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113408380A (en) * | 2021-06-07 | 2021-09-17 | 深圳小湃科技有限公司 | Video image adjusting method, device and storage medium |
| CN113949910A (en) * | 2021-10-15 | 2022-01-18 | 深圳小湃科技有限公司 | Set top box resolution matching method and device, set top box and storage medium |
| CN114038370A (en) * | 2021-11-05 | 2022-02-11 | 深圳Tcl新技术有限公司 | Display parameter adjusting method and device, storage medium and display equipment |
| CN114038370B (en) * | 2021-11-05 | 2023-10-13 | 深圳Tcl新技术有限公司 | Display parameter adjustment method and device, storage medium and display equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113177438B (en) | 2024-05-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11995800B2 (en) | Artificial intelligence techniques for image enhancement | |
| US11138478B2 (en) | Method and apparatus for training, classification model, mobile terminal, and readable storage medium | |
| CN108764208B (en) | Image processing method and device, storage medium and electronic equipment | |
| CN108805103B (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| CN108764370B (en) | Image processing method, image processing device, computer-readable storage medium and computer equipment | |
| US10896323B2 (en) | Method and device for image processing, computer readable storage medium, and electronic device | |
| CN110276767B (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| CN113177438B (en) | Image processing method, device and storage medium | |
| CN111292264A (en) | Image high dynamic range reconstruction method based on deep learning | |
| CN108805198B (en) | Image processing method, image processing device, computer-readable storage medium and electronic equipment | |
| CN110610463A (en) | Image enhancement method and device | |
| CN108717530B (en) | Image processing method, image processing device, computer-readable storage medium and electronic equipment | |
| CN110349163B (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| CN110443766B (en) | Image processing method and device, electronic equipment and readable storage medium | |
| CN104182721A (en) | Image processing system and image processing method capable of improving face identification rate | |
| CN108848306B (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| WO2019223513A1 (en) | Image recognition method, electronic device and storage medium | |
| CN111654643B (en) | Exposure parameter determination method and device, unmanned aerial vehicle and computer readable storage medium | |
| WO2023125750A1 (en) | Image denoising method and apparatus, and storage medium | |
| US9940543B2 (en) | Control of computer vision pre-processing based on image matching using structural similarity | |
| CN113408380B (en) | Video image adjustment method, device and storage medium | |
| CN108600631B (en) | Image processing method, image processing device, computer-readable storage medium and electronic equipment | |
| CN113067980A (en) | Image acquisition method and device, electronic equipment and storage medium | |
| CN108881740B (en) | Image method and device, electronic equipment and computer readable storage medium | |
| CN111080543B (en) | Image processing method and device, electronic equipment and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |