WO2022183321A1 - Procédé de détection, appareil et dispositif électronique - Google Patents

Procédé de détection, appareil et dispositif électronique Download PDF

Info

Publication number
WO2022183321A1
WO2022183321A1 PCT/CN2021/078478 CN2021078478W WO2022183321A1 WO 2022183321 A1 WO2022183321 A1 WO 2022183321A1 CN 2021078478 W CN2021078478 W CN 2021078478W WO 2022183321 A1 WO2022183321 A1 WO 2022183321A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
image processing
algorithm
image data
detection
Prior art date
Application number
PCT/CN2021/078478
Other languages
English (en)
Chinese (zh)
Inventor
邱珏沁
柳海波
闻明
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to PCT/CN2021/078478 priority Critical patent/WO2022183321A1/fr
Priority to CN202180093086.5A priority patent/CN116888621A/zh
Publication of WO2022183321A1 publication Critical patent/WO2022183321A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/60—Image enhancement or restoration using machine learning, e.g. neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • the embodiments of the present application relate to the technical field of artificial intelligence, and in particular, to an image detection method, apparatus, and electronic device.
  • AI Artificial Intelligence
  • machine learning methods are usually used to construct initial models of various structures, such as neural network models, support vector machine models, and decision tree models. Then, the initial model is trained by training samples to realize functions such as image detection, speech recognition, etc.
  • the neural network is usually trained to obtain a perception model to realize image detection tasks such as scene recognition, object detection or image segmentation.
  • the sample image data used for training the neural network and the image data to be detected are usually collected by different camera modules. Due to the significant differences in the manufacturing process, photoelectric response function and noise level of different camera modules, there is a large deviation between the detection results of the perception model and the real results in the image detection process. Therefore, when a new camera module is combined with an already trained perception model, how to efficiently improve the accuracy of the detection result of the perception model is a problem that needs to be solved.
  • the image detection method, device and electronic device provided by the present application can improve the accuracy of image detection model inference when a new photographing device is combined with an already trained image detection model.
  • an embodiment of the present application provides an image detection method, the image detection method includes: collecting image data to be detected by a first camera device; processing the image data to be detected by using an image processing algorithm to generate a processing The processed image is input into the image detection model to obtain the detection result; wherein, the parameters of the image processing algorithm are the annotation information of the first sample image data collected by the first camera device by comparing It is obtained by adjusting the detection result of the first sample image data with the image detection model and based on the comparison result.
  • the parameters of the image processing algorithms used to execute multiple image processing processes are adjusted based on the image detection model, so that the image obtained after image processing is performed on the image collected by the first camera device is
  • the style is consistent with the style of the sample image data for training the image detection model, thereby reducing the difference between the image data collected by the camera device and the feature distribution in the high-dimensional space of the sample image data for training the image detection model.
  • the image detection model is obtained by performing neural network training on the second sample image data collected by the second camera device.
  • the parameters of the image processing algorithm are determined by the following steps: comparing the detection result with the annotation information of the first sample image data to obtain the comparison Result: Based on the error between the detection result and the annotation information of the sample image data, the parameters of the image processing algorithm are iteratively adjusted; when the preset conditions are satisfied, the parameters of the image processing algorithm are saved.
  • the preset conditions here may include but are not limited to: the error is less than or equal to a preset threshold, or the number of iterations is greater than or equal to a preset threshold.
  • the comparison result is an error
  • the iteratively adjusting the parameters of the image processing algorithm based on the comparison result includes: based on the comparison result and the first The error between the annotation information of the sample image data, construct a target loss function, wherein the target loss function includes the parameters to be adjusted in the image processing algorithm; based on the target loss function, using the back propagation algorithm and Gradient descent algorithm, iteratively adjust the parameters of the image processing algorithm.
  • the processing of the image data and the first sample image data includes at least one of the following: dark current correction, lens shading correction, demosaicing, and white balance correction , tone mapping, contrast enhancement, image edge enhancement, or image noise reduction.
  • the image processing algorithm is executed by an image signal processor; and the parameters of the image processing algorithm include at least one of the following: each pixel of the image in the lens shading correction algorithm The distance from the optical center of the camera; the boundary coordinates of the neutral color region in the image in the white balance correction algorithm; the target brightness and target saturation in the tone mapping algorithm, and the filter kernel parameters used to generate the low-pass filtered image; contrast ratio The contrast threshold in the enhancement algorithm; the edge enhancement factor in the image edge enhancement algorithm; and the spatial domain Gaussian parameter and the pixel value domain Gaussian parameter in the image noise reduction algorithm.
  • the image processing algorithm is executed by a trained image processing model; the parameters of the image processing algorithm further include: a neural network for generating the image processing model weight factor.
  • the annotation information of the first sample image data is manually annotated; and the method further includes: converting the first sample image data into suitable Human-annotated color images.
  • the image detection model is used to perform at least one of the following detection tasks: labeling of detection frames, recognition of target objects, prediction of confidence levels, and prediction of motion trajectories of target objects .
  • an embodiment of the present application provides a parameter adjustment method for image processing.
  • the parameter adjustment method for image processing includes: using an image processing algorithm to perform image processing on first sample image data to generate a first image data, wherein the first sample image data is collected by a first camera; input the first image data into a pre-trained image detection model to obtain a detection result; compare the detection result with the first The error between the annotation information of a sample image data is obtained, and a comparison result is obtained; based on the comparison result, the parameters of the image processing algorithm are adjusted.
  • the image detection model is obtained by performing neural network training on the second sample image data collected by the second camera device.
  • the comparison result is an error
  • the iteratively adjusting the parameters of the image processing algorithm based on the comparison result includes: based on the detection result and the first
  • the error between the annotation information of the sample image data is used to construct a target loss function, wherein the target loss function includes the parameters to be adjusted in the image processing algorithm; based on the target loss function, a back-propagation algorithm is used. and a gradient descent algorithm that iteratively adjusts the parameters of the image processing algorithm.
  • the image processing algorithm includes at least one of the following: dark current correction, lens shading correction, demosaicing, white balance correction, tone mapping, contrast enhancement, and image edge enhancement Or image noise reduction.
  • the parameters of the image processing algorithm include at least one of the following: the distance between each pixel of the image in the lens shading correction algorithm and the optical center of the camera; The boundary coordinates of the color region in the image; the target brightness, target saturation and the filter kernel parameters used to generate the low-pass filtered image in the tone mapping algorithm; the contrast threshold in the contrast enhancement algorithm; the edge in the image edge enhancement algorithm enhancement factor; and the spatial domain Gaussian parameter and the pixel value domain Gaussian parameter in the image noise reduction algorithm.
  • the image processing algorithm is executed by a trained image processing model; the parameters of the image processing algorithm include: a neural network used to generate the image processing model. weight factor.
  • the labeling information of the first sample image data is manually labelled; and the method further includes: converting the first sample image data into Human-annotated color images.
  • the image detection model is used to perform at least one of the following detection tasks: labeling a detection frame, identifying a target object, predicting a confidence level, and predicting a motion trajectory of the target object .
  • an embodiment of the present application provides an image detection device, the image detection device includes: a collection module configured to collect image data to be detected through a first camera device; a processing module configured to use an image processing algorithm to The image data to be detected is processed to generate a processed image; the detection module is configured to input the processed image into an image detection model to obtain a detection result; wherein, the parameters of the image processing algorithm are obtained through It is obtained by comparing the annotation information of the first sample image data collected by the first camera device with the detection result of the first sample image data by the image detection model, and adjusting based on the comparison result.
  • the image detection model is obtained by performing neural network training on the second sample image data collected by the second camera device.
  • the parameters of the image processing algorithm are determined by a parameter adjustment module
  • the parameter adjustment module includes: a comparison sub-module configured to The detection result of the image data is compared with the annotation information of the first sample image data to obtain the comparison result; the adjustment sub-module is configured to iteratively adjust the parameters of the image processing algorithm based on the comparison result; save the sub-module The module is configured to save the parameters of the image processing algorithm when the preset condition is satisfied.
  • the comparison result is an error
  • the adjustment sub-module is further configured to: based on the detection result of the first sample image data and the first sample
  • the error between the annotation information of the image data is used to construct a target loss function, wherein the target loss function includes the parameters to be updated in the image processing algorithm; based on the target loss function, back-propagation algorithm and gradient descent are used. algorithm that iteratively updates the parameters of the image processing algorithm.
  • the image processing algorithm includes at least one of the following image processing procedures: dark current correction, lens shading correction, demosaicing, white balance correction, tone mapping, contrast enhancement, image Edge enhancement and image noise reduction.
  • the parameters of the image processing algorithm include at least one of the following: the distance between each pixel of the image in the lens shading correction algorithm and the optical center of the camera; The boundary coordinates of the color region in the image; the target brightness, target saturation and the filter kernel parameters used to generate the low-pass filtered image in the tone mapping algorithm; the contrast threshold in the contrast enhancement algorithm; the edge in the image edge enhancement algorithm enhancement factor; and the spatial domain Gaussian parameter and the pixel value domain Gaussian parameter in the image noise reduction algorithm.
  • the image processing algorithm is executed by a trained image processing model; the parameters of the image processing algorithm include: a neural network used to generate the image processing model. weight factor.
  • the annotation information of the first sample image data is manually annotated; and the method further includes: converting the first sample image data into Human-annotated color images.
  • the image detection model is used to perform at least one of the following detection tasks: labeling a detection frame, identifying a target object, predicting a confidence level, and predicting a motion trajectory of the target object .
  • an embodiment of the present application provides an electronic device, the electronic device includes: a first camera device, configured to collect image data to be detected; an image signal processor, configured to use an image processing algorithm to detect the image data to be detected.
  • the image data is processed to generate a processed image; an artificial intelligence processor is used to input the processed image into an image detection model to obtain a detection result; wherein, the parameters of the image processing algorithm are obtained by comparing the first It is obtained by adjusting the annotation information of the first sample image data collected by a camera device and the detection result of the first sample image data by the image detection model and based on the comparison result.
  • an embodiment of the present application provides an image detection device, the image detection device includes one or more processors and a memory; the memory is coupled to the processor, and the memory is used to store one or more programs ; the one or more processors are configured to run the one or more programs to implement the method according to the first aspect.
  • an embodiment of the present application provides a parameter adjustment apparatus for image processing
  • the parameter adjustment apparatus for image processing includes one or more processors and a memory; the memory is coupled to the processor, and the The memory is used to store one or more programs; the one or more processors are used to execute the one or more programs to implement the method according to the second aspect.
  • embodiments of the present application provide a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by at least one processor, is used to implement the first aspect or the second aspect the method described.
  • an embodiment of the present application provides a computer program product, which is used to implement the method according to the first aspect or the second aspect when the computer program product is executed by at least one processor.
  • FIG. 1 is a schematic structural diagram of a terminal provided by an embodiment of the present application.
  • FIG. 2 is a schematic structural diagram of an image processing process performed in combination with an ISP and an AI processor provided by an embodiment of the present application;
  • FIG. 3 is a schematic structural diagram of a vehicle provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of a system architecture including an electronic device for parameter debugging of an image processing algorithm provided by an embodiment of the present application;
  • FIG. 5 is a flowchart of a parameter debugging method of an image processing algorithm provided by an embodiment of the present application.
  • FIG. 7 is a flowchart of an image detection method provided by an embodiment of the present application.
  • FIG. 8 is a schematic structural diagram of an image processing apparatus provided by an embodiment of the present application.
  • FIG. 9 is a schematic structural diagram of a parameter debugging apparatus provided by an embodiment of the present application.
  • the corresponding apparatus may include one or more units, such as functional units, to perform one or more of the described method steps (eg, one unit performs one or more steps) , or units, each of which performs one or more of the steps), even if such unit or units are not explicitly described or illustrated in the figures.
  • the corresponding method may contain a step to perform the functionality of the one or more units (eg, a step to perform the one or more units) functionality, or steps, each of which performs the functionality of one or more of the plurality of units), even if such one or more steps are not explicitly described or illustrated in the figures.
  • the image detection method described in the present application can be applied in the field of computer vision, in a scene where an image detection model obtained by training sample images collected by other photographing equipment needs to be combined with a new photographing equipment.
  • the electronic device 100 may be a user equipment (User Equipment, UE), such as various types of devices such as a mobile phone, a tablet computer, a smart screen, or an image capturing device.
  • UE User Equipment
  • the electronic device 100 may also be a vehicle.
  • a camera 101 may be provided in the electronic device 100 for capturing image data.
  • the electronic device 100 may also include or be integrated into a module, chip, chip set, circuit board or component in the electronic device, and the chip or chip set or the circuit board equipped with the chip or chip set can be driven by necessary software Work.
  • the electronic device 100 includes one or more processors, such as an image signal processor (ISP, Image Signal Processor) 102 and an AI processor 103 .
  • the one or more processors can be integrated in one or more chips, and the one or more chips can be regarded as a chipset, when one or more processors are integrated in the same chip
  • the chip is also called a system on a chip (SOC).
  • the electronic device 100 also includes one or more other necessary components, such as memory and the like.
  • the camera device 101 shown in FIG. 1 may be a monocular camera.
  • the camera device 101 may further include multi-camera cameras, and these cameras may be physically combined in one camera device, or may be physically separated into multiple camera devices. Multiple images are captured at the same time by a multi-eye camera, and can be processed according to these images to obtain an image to be detected.
  • the camera 101 may also be in other situations, which are not specifically limited in the embodiments of the present application.
  • the camera 101 can collect image data in real time, or collect image data periodically. The period is such as 3s, 5s, 10s and so on.
  • the camera device 101 may also collect image data in other ways, which are not specifically limited in this embodiment of the present application. After the camera 101 collects the image data, it can transmit the image data to the ISP 102 .
  • the ISP 102 shown in FIG. 1 can set up a plurality of hardware modules or run necessary software programs to process the image data and communicate with the AI processor 103 .
  • ISP102 can be used alone as a component or integrated in other digital logic devices, including but not limited to: CPU (Central Processing Unit, Central Processing Unit), GPU (Graphics Processing Unit, Graphics Processing Unit) or DSP (Digital Signal Processing Unit) processor, Digital Signal Processing).
  • the CPU, GPU and DSP are all processors within a system-on-chip.
  • ISP102 can perform multiple image processing processes, which may include but are not limited to: dark current correction, response nonlinearity correction, shading correction, demosaicing, white balance correction, tone mapping, contrast enhancement, edge enhancement, Noise reduction, color correction, and more.
  • the ISP 102 executes the above-mentioned multiple image processing processes by running the image processing algorithm.
  • Each image processing process in the above-mentioned multiple image processing processes can be regarded as an independent image processing process, and thus, the image processing algorithm for executing each image processing process can be regarded as independent.
  • the ISP 102 may include multiple logic modules. For example, it includes, but is not limited to, a dark current correction module, a response nonlinearity correction module, a shading correction module, a demosaicing module, and the like.
  • Each logic module is used to perform an image detection process.
  • Each logic module may use its own specific hardware structure, and multiple logic modules may also share a set of hardware structures, which is not limited in this embodiment of the present application.
  • the one or more image processing processes are typically performed sequentially. For example, after the image data acquired by the camera 101 is provided to the ISP, processing procedures such as dark current correction, response nonlinear correction, shading correction, demosaicing, white balance correction, etc. may be sequentially performed. It should be noted that this embodiment of the present application does not limit the sequence of the image processing processes performed by the ISP. For example, white balance correction may be performed first, and then demosaicing may be performed.
  • the AI processor 103 shown in FIG. 1 may include a special neural processor such as a neural network processor (Neural-network Processing Unit, NPU), including but not limited to a convolutional neural network processor, a tensor processor, or a neural processing unit. engine.
  • NPU Neuro-network Processing Unit
  • the AI processor can be used alone as a component or integrated in other digital logic devices, including but not limited to: CPU, GPU or DSP.
  • the AI processor 103 may run an image detection model, and the image detection model is obtained by training a deep neural network based on the sample image data set S1. This image detection model can perform specific detection tasks.
  • the specific detection task may include, but is not limited to, labeling of detection frames, recognition of target objects, prediction of confidence levels, prediction of motion trajectories of target objects, or image segmentation, and the like.
  • the image detection model is deployed in the AI processor 103 shown in FIG. 1 after the offline end training is completed.
  • the offline end here can be regarded as a server device or a device for model training.
  • the AI processor 103 may also perform one or more image processing operations, the one or more image processing operations Operations may include, but are not limited to: demosaicing, white balance correction, tone mapping, contrast enhancement, edge enhancement, noise reduction, color correction.
  • the AI processor 103 may also run one or more image processing models, where each image processing model is used to execute a specific image processing process.
  • the image data obtained from the camera 101 may undergo multiple image processing processes to generate a final image processing result, and the AI processor 103 may perform one or more of the above image processing processes, That is, corresponding to one or more of the above-mentioned image processing operations, the ISP 102 may also perform one or more of the above-mentioned image processing procedures.
  • the AI processor 103 and the ISP 102 may perform different image processing processes.
  • the AI processor 103 and the ISP 102 may also perform the same image processing process, such as performing further enhancement processing, which is not limited in this embodiment.
  • each image processing model may be obtained by training a neural network by using a machine learning method based on the sample image data set S3.
  • the sample image data set S3 includes: a plurality of sample image data H, and reference image data I corresponding to each sample image data H in the plurality of sample image data H.
  • the reference image data I is used for image data comparison, and the image data to be compared with the reference image data I is: image data obtained after the neural network to be trained performs image processing on the sample image data H.
  • the reference image data I and its corresponding sample image data H present the same scene.
  • the same scene presented here can be understood as: the target objects presented by the reference image data I and the corresponding sample image data H and the positions of the target objects in the image are all the same.
  • the image processing process performed by the image processing model is different, and the sample image dataset S3 used for training the image processing model is also different.
  • each sample image data H included in the sample image data set S3 is a single-channel raw image format of a*b*1 size (RAW, Raw Image Format) image data
  • each reference image data I corresponding to each sample image data is the RGB image data of a*b*3 size, wherein, a is the vertical pixel value of the image, and b is the horizontal pixel value of the image, 1 is single channel (eg R channel, G channel or B channel), 3 is three channel (RGB channel).
  • each sample image data H included in the sample image data set S3 is single-channel RAW image data of a*b*1 size, which is the same as each sample image data H.
  • Each reference image data I corresponding to one sample image data is single-channel image data of a*b*1 size, the sample image data set S3.
  • the reference image data I and its corresponding sample image data H have the same scene but different white balance values.
  • the following describes the training process of the image processing model by taking the image processing process in which the image processing model performs demosaicing as an example.
  • Each sample image data H included in the sample image data set S3 is respectively input to the neural network to be trained to obtain processed image data.
  • the loss function may include, but is not limited to, a mean absolute error (MAE) loss function or a mean square error (MSE) loss function, for example.
  • the loss function includes the weight coefficients of each layer of the neural network to be trained.
  • the back-propagation algorithm and the gradient descent algorithm are used to iteratively adjust the weight coefficient values of each layer network in the neural network to be trained, until the processed image data output by the image processing model and the reference image data I The error between them is less than or equal to the preset threshold, or the number of iterations is less than the preset threshold, and the weight coefficient values of each layer of the neural network to be trained are saved.
  • the neural network to be trained is an image processing model.
  • the ISP102 can be set with multiple ports, and the AI processor 103 can also be set with multiple ports.
  • the ISP102 can pass the processed image data A through multiple ports.
  • One of the ports is provided to the AI processor 103 , the AI processor 103 processes the image data A to generate image data B, and provides the image data B to the ISP 102 through one of the multiple ports.
  • the AI processor 103 performing demosaicing as an example, the combination of the ISP 102 and the AI processor 103 will be described with reference to FIG. 2 .
  • FIG. 2 In FIG.
  • the ISP 102 acquires image data from the camera 101, performs the three image processing processes of dark current correction, response nonlinear correction, and shading correction on the acquired image data, and generates image data A, which is provided to AI through port Vio The input port Vai of the processor 103 .
  • the image processing model run by the AI processor 103 performs demosaic processing on the image data A to generate image data B, and provides the image data B to the input port Vii of the ISP 102 through the output port Vao.
  • the ISP 102 performs subsequent image processing procedures such as white balance correction and color correction on the image data B input through the input port Vii, and generates image data C that is input to the AI processor 103 .
  • the image detection model run by the AI processor 103 can perform image detection processing on the image data C.
  • the AI processor 103 may include one or more.
  • the image processing model for performing image processing and the image processing model for performing image detection The image detection model can be set in the same AI processor 103.
  • the image processing model used to perform image processing and the image detection model used to perform image detection can be set to different AI processing models. in the device 103.
  • the parameters in the image processing algorithm run by the ISP 102 and the parameters in the image processing model run by the AI processor 103 described in the embodiments of the present application are based on the sample image data set S2 collected by the camera 101 and The image detection results of the image detection model are obtained by debugging.
  • the image processing algorithm run by the ISP 102 and the debugging method of each parameter in the image processing model refer to the embodiment shown in FIG. 5 below.
  • the sample image data set S1 used for training the image detection model running in the AI processor 103 is collected through big data, and the camera used for collecting the sample image data set S1 is the same as the camera shown in FIG. 1 .
  • 101 is a different camera device. Since there are significant differences in the manufacturing process, photoelectric response function, noise level and other characteristics of different camera devices, the style of the sample image data D in the sample image data set S1 is different from the image data collected by the camera device 101. There are differences in the styles of the images obtained after processing, which in turn leads to significant differences in the feature distributions in the high-dimensional space between the image data collected by the camera device 101 and the sample image data D in the sample image data set S1, resulting in the deployment of image detection models.
  • the AI processor 103 during the detection process of the image data collected by the camera 101 , the deviation between the detection result and the real result is relatively large, which reduces the detection accuracy of the image detection model deployed in the AI processor 103 .
  • the embodiment of the present application adjusts the parameters of the image processing algorithms used for executing multiple image processing processes (or adjusts the parameters of the image processing algorithms used for executing multiple image processing processes). parameters and parameters of the image processing model), so that the style of the image obtained by performing image processing on the image collected by the camera 101 is consistent with the style of the sample image data D in the sample image data set S1, thereby reducing the
  • the difference between the image data collected by the camera 101 and the feature distribution of the sample image data D in the sample image data set S1 in the high-dimensional space is conducive to improving the accuracy of the image detection model inference.
  • the embodiment of the present application does not require any modification to the trained image detection model, which saves the time and computing power overhead required for retraining and fine-tuning the image detection model; Adjust the parameters of the image processing algorithm used to perform multiple image processing processes. Since the image processing algorithm does not perform the image detection process, the training can be completed with fewer training samples, thereby reducing the need for manual labeling of training samples. quantity, shortening the commissioning cycle when combining image detection models with new cameras.
  • FIG. 3 shows a schematic structural diagram of a vehicle 300 provided by an embodiment of the present application.
  • Components coupled to or included in vehicle 300 may include control system 10 , propulsion system 20 , and sensor system 30 . It should be understood that the vehicle 300 may also include more systems, which will not be repeated here.
  • the control system 10 may be configured to control the operation of the vehicle 300 and its components.
  • the ISP 102 and the AI processor 103 shown in FIG. 1 can be set in the control system 10.
  • the control system 10 can also include devices such as a central processing unit, a memory, and the like, and the memory is used to store the instructions and data required for the operation of each processor .
  • Propulsion system 20 may be used for vehicle 300 to provide powered motion, which may include, but is not limited to, an engine/motor, energy source, transmission, and wheels.
  • the sensor system 104 may include, but is not limited to, a global positioning system, an inertial measurement unit, a lidar sensor, or a millimeter-wave radar sensor.
  • the camera device 101 shown in FIG. 1 may be provided in the sensor system 30 .
  • the components and systems of vehicle 300 may be coupled together through a system bus, network, and/or other connection mechanism to operate in interconnection with other components within and/or outside of their respective systems. In specific work, various components in the vehicle 300 cooperate with each other to realize various automatic driving functions.
  • the automatic driving function may include, but is not limited to, blind spot detection, parking assist or lane change assist, and the like.
  • the camera device 101 may periodically collect image data, and provide the collected image data to the ISP 102 .
  • the ISP102 (or the image processing model in the ISP102 and the AI processor 103) processes the image data by performing multiple image processing processes, and converts it into image data that can be recognized or calculated by the image detection model running in the AI processor 103.
  • the AI processor 103 enables the AI processor 103 to realize the reasoning or detection of a specific task, and generate the detection result.
  • Other components in the control system 10 for example, a CPU that executes decisions) control other devices or components to perform automatic driving functions based on the detection results of the AI processor 103 .
  • the manufacturer of the vehicle may not produce some parts by itself.
  • the trained image detection model is ordered through manufacturer A, and the camera device is ordered through manufacturer B.
  • the training method described in the embodiments of the present application can be used to debug the parameters of the image processing algorithm or the image processing model used to execute the image processing flow. For another example, when a manufacturer upgrades certain models of vehicles, it is necessary to replace a camera device of a different model from the previously configured camera device.
  • the training method described in the embodiments of the present application can also be used to Debug the parameters of the image processing algorithm or image processing model used to execute the image processing flow.
  • the parameter debugging of the image processing algorithm or the image processing model may be completed at the offline end (or in other words, the training is completed in the server or the device used for model training).
  • the image processing algorithm can be deployed in the ISP of the terminal.
  • the image processing model can be deployed in the AI processor 103 .
  • FIG. 4 shows a schematic diagram 400 of a system architecture including an electronic device for parameter debugging of an image processing algorithm provided by an embodiment of the present application.
  • the system architecture 400 includes a camera device 101 , a parameter debugging device 401 , a storage device 402 and a display device 403 .
  • the camera 101 is used for collecting a plurality of sample image data E, and storing the collected sample image data E in the storage device 402 .
  • the imaging device 101 and the imaging device 101 shown in FIG. 1 are the same (or the same) imaging device.
  • the storage device 402 may include, but is not limited to, read-only memory or random access memory, and the like. It is used to store sample image data E.
  • the storage device 402 may also store executable programs and data of an image processing algorithm for executing the image processing process, and executable programs and data of an image detection model for executing the image detection.
  • the parameter debugging device 401 can run the image processing algorithm and the image detection model, and the parameter debugging device 401 can also call the sample image data E, the executable program and data of the image processing algorithm for executing the image processing process, and the data for the image processing from the storage device 101.
  • the executable program and data of the image detection model that performs image detection to debug the parameters of the image processing algorithm.
  • the parameter debugging device 401 may also store the data generated by the operation and the debugging result after each parameter debugging of the image processing algorithm into the storage device 402 .
  • the parameter debugging device 401 and the storage device 402 may also be provided with I/O ports for data interaction with the display device 403 .
  • the display device 403 may include a display device such as a screen to mark the sample image data E.
  • the parameter debugging device 401 may acquire sample image data E from the storage device 402 , perform image processing on the sample image data E, and provide the sample image data E to the display device 403 for presentation in the display device 403 .
  • the user will mark the sample image data E through the display device 403 , and store the marking information of the sample image data E in the storage device 402 .
  • the sample image data E output by the camera 101 is a high bit depth (for example, 16bit, 20bit or 24bit) single-channel Linear RAW image data, the dynamic range of which is much larger than the dynamic range that can be displayed by the monitor.
  • the sample image data E is a color filter array (CFA, color filter array) image, which does not have color information, so it is difficult for the annotator to extract the image from the camera.
  • Each target object is identified in the sample image data E output by the device 101 .
  • the parameter debugging device 401 also runs an image processing algorithm T, and the image processing algorithm T is used to process the sample image data E to generate a color that can be presented on a display and has suitable brightness and color.
  • An image such as an RGB image, is convenient for the annotator to annotate the target object presented in the sample image data E.
  • the image processing flow performed by the image processing algorithm T may include, but is not limited to: system error correction, global tone mapping, demosaicing or white balance correction.
  • Each parameter in the image processing algorithm T does not need to be adjusted, which can be achieved by using traditional image processing algorithms.
  • the image processing algorithm T described in the embodiments of the present application is used to process the sample image data E to generate image data that can be displayed on the display for annotators to mark;
  • the image processing algorithm is used to process the sample image data E to generate image data for image detection by the image detection model, and its parameters need to be adjusted.
  • FIG. 5 shows a flow 500 of a method for debugging parameters of an image processing algorithm provided by an embodiment of the present application.
  • the execution body of the parameter debugging method for image processing described in the embodiments of the present application may be the parameter debugging device 401 shown in FIG. 4 .
  • the parameter debugging method for image processing includes the following steps:
  • Step 501 based on the sample image data set S2 , use an image processing algorithm to process each sample image data E in the sample image data set S2 to generate a plurality of image data F.
  • the sample image data set S2 includes a plurality of sample image data E and label information of each sample image data E.
  • each sample image data E in the sample image data set S2 is collected by the camera 101 as shown in FIG. 1 .
  • the annotation information of the sample image data E is annotated based on the detection content performed by the image detection model. For example, when the image detection model is used to perform target detection, the annotation information of the sample image data E may include the target object and the position of the target object in the second sample image; when the image detection model is used to perform pedestrian intent detection, the sample image The annotation information of the data E may include the target object and the action information of the target object.
  • the image processing algorithms herein are used to perform one or more image processing procedures.
  • the one or more image processing procedures include, but are not limited to, dark current correction, response nonlinearity correction, lens shading correction, demosaicing, white balance correction, tone mapping, noise reduction, contrast enhancement or edge enhancement, and the like. It should be noted that the one or more image processing processes are usually performed sequentially. This embodiment of the present application does not specifically limit the execution order of the image processing process.
  • step 502 the image data F is detected by using the image detection model, and a detection result is generated.
  • the image detection model may perform at least one of the following detections: object detection, lane line detection, or pedestrian intent detection, and the like.
  • the image detection model is obtained by training a deep neural network based on the image dataset S1. It should be noted that the image data D in the image data set S1 is collected by other imaging devices different from the imaging device 101 .
  • the training method of the image detection model is a traditional technology, and details are not described here.
  • Step 503 based on the detection result and the label information of the sample image data E, adjust the parameters of the image processing algorithm.
  • the parameters of the image processing algorithm may be adjusted by using a machine learning method.
  • the second possible implementation manner is described in detail below.
  • a loss function is constructed based on the error between the detection result of each sample image data E in the sample image data set S2 and the label information of the sample image data E.
  • the loss function may include, but is not limited to, a cross-entropy function and the like.
  • the parameters of the image processing module for executing one or more image processing procedures in the image processing algorithm are adjusted by using the back-propagation algorithm and the gradient descent algorithm.
  • the gradient descent algorithm may specifically include, but is not limited to, optimization algorithms such as SGD and Adam.
  • the chain rule can be used to calculate the gradient of the preset loss function with respect to each parameter in the image processing algorithm.
  • the image processing algorithms used to execute each image processing flow are independent of each other.
  • the image processing flow performed by the image processing algorithm is dark current correction, response nonlinear correction, lens shading correction, demosaicing, white balance correction, noise reduction, contrast enhancement or edge enhancement, etc.
  • the parameters that are propagated and adjusted first are the parameters of the image processing algorithm for performing edge enhancement, and then the parameters of the image processing algorithms that are adjusted sequentially are the image processing parameters for performing contrast enhancement.
  • the embodiments of the present application may include more or less image processing processes, and accordingly may include more or less parameters to be adjusted.
  • the embodiment of the present application does not limit the order of the image processing flow performed by the image processing algorithm, when using the backpropagation algorithm to adjust the parameters of the image processing algorithm, the first adjusted parameter and the last adjusted parameter.
  • Step 504 Determine whether the loss value of the preset loss function is less than or equal to a preset threshold. If the loss value of the preset loss function is less than or equal to the preset threshold, the parameters of the image processing algorithm are saved; if the loss value of the preset loss function is greater than the preset threshold, step 505 is executed.
  • Step 505 Determine whether the number of times of iteratively adjusting the parameters of the image processing algorithm is greater than or equal to a preset threshold. If the number of times of iteratively adjusting the parameters of the image processing algorithm is greater than or equal to the preset threshold, the parameters of the image processing algorithm are saved, and if the number of times of iteratively adjusting the parameters of the image processing algorithm is less than the preset threshold, continue to execute steps 501-505.
  • the above-mentioned one or more image processing procedures may all be implemented by traditional image processing algorithms.
  • the one or more image processing procedures include dark current correction, response nonlinearity correction, lens shading correction, demosaicing, white balance correction, tone mapping , Noise Reduction, Contrast Enhancement and Edge Enhancement.
  • the parameters of the image processing algorithm to be adjusted may include, but are not limited to: parameters of an image processing algorithm for performing lens shading correction, parameters for an image processing algorithm for performing white balance correction, parameters for performing tone mapping Parameters of an image processing algorithm, parameters of an image processing algorithm for performing contrast enhancement, parameters for an image processing algorithm for performing edge enhancement, and parameters for an image processing algorithm for performing noise reduction.
  • parameters of an image processing algorithm for performing lens shading correction parameters for an image processing algorithm for performing white balance correction
  • parameters for performing tone mapping Parameters of an image processing algorithm parameters of an image processing algorithm for performing contrast enhancement
  • parameters for an image processing algorithm for performing edge enhancement parameters for an image processing algorithm for performing noise reduction.
  • Lens shading correction is used to correct the illuminance attenuation caused by the increase of the incident angle of the chief ray in the edge area of the image. It uses a polynomial to fit the illuminance attenuation surface, where the independent variable of the polynomial is the distance between each pixel of the image and the optical center of the camera device . Therefore, in the image processing algorithm for performing lens shading correction, the parameter to be adjusted is the distance between each pixel of the image and the optical center of the camera, that is, the value of the independent variable in the polynomial.
  • the execution process of white balance correction is as follows: first, the neutral color pixel search algorithm is used to screen the neutral color area in the image, and based on the screening result, the boundary coordinates of the neutral color area in the image are determined. Then, the pixel values in the filtered neutral regions are weighted using the luminance channel of the image to generate a binarized neutral pixel mask. The individual (near) neutral pixels are then weighted averaged using this neutral pixel mask to obtain an estimate of the color of the light source in the image. Finally, by calculating the ratio between the RGB channels of the light source color, the white balance correction coefficient corresponding to the image is obtained, and the white balance correction coefficient is applied to the original image, that is, the white balance corrected image is obtained. Therefore, in the image processing algorithm for performing white balance correction, the parameter to be adjusted is the boundary coordinates of the neutral color area in the image.
  • Tone mapping is used to receive a linear image with high bit depth, convert the linear image into a nonlinear image, and complete the compression of the image bit depth, and output an 8-bit image.
  • the trainable parameter is the ⁇ parameter; when using the logarithmic transformation algorithm to compress the dynamic range of the linear image, the trainable parameter is the base of the logarithmic transformation; when using a more complex order
  • a tone mapping model such as a retinax model based on the dynamic range response of the human eye, can be trained as a target luminance parameter (key), a target saturation parameter (saturation), and a filter kernel parameter for generating a low-pass filtered image.
  • Contrast Enhancement is used to enhance the contrast of an image.
  • the CLAHE contrast limited adaptive histogram equalization, contrast limited adaptive histogram equalization
  • the CLAHE algorithm contains two adjustable parameters: the contrast threshold parameter and the sub-patch size used for histogram statistics.
  • the size of the sub-image block may be fixed, and only the contrast threshold parameter may be adjusted. Further, the size of the sub-image block can be fixed to the size of the input image.
  • the Y-channel image in the received image is first subjected to Gaussian filtering to obtain a low-pass Y-channel image Y L ; the difference image between the original Y-channel image and the low-pass Y-channel image Y L
  • the high-frequency signal usually corresponds to the edge area in the image; by amplifying the intensity of the high-frequency signal and superimposing it into the low-pass Y-channel image Y L , it can be
  • the parameter to be adjusted is the edge enhancement factor ⁇ .
  • the bilateral filter noise reduction algorithm is usually used.
  • the trainable parameters can include: the spatial Gaussian kernel parameter ⁇ s used to control the relationship between the noise reduction intensity and the spatial distance, and the pixel used to control the relationship between the noise reduction intensity and the response value difference Value domain Gaussian kernel parameter ⁇ r.
  • part of the image processing flow in the above one or more image processing flows may be implemented by an image processing model.
  • the above-mentioned image processing processes such as demosaicing, white balance correction, tone mapping, noise reduction, contrast enhancement and edge enhancement are implemented by an image processing model.
  • the image processing model may include a variety of image processing models, each of which is used to perform a specific image processing operation.
  • a noise-cancelling image processing model is used to perform a noise-cancelling image processing operation
  • a demosaicing image processing model is used to perform a demosaicing image processing operation.
  • Each image processing model may be obtained by using a traditional neural network training method, and using training samples to train a multi-layer neural network, and the training method will not be repeated in this embodiment of the present application.
  • the adjusted image processing algorithm is the weight matrix (the weight matrix formed by the vectors W of many layers) of all layers forming the image processing model. It should be noted that the above one or more image processing models are pre-trained at the offline end.
  • the parameters of the image processing algorithm are adjusted using the parameter debugging method described in this application, it is only necessary to fine-tune the parameters of the neural network used to form the image processing model, so that the image processing model can obtain the style of the image after processing the image. features, similar to the style features of the sample image data used to train the image detection model. Thereby, the detection accuracy of the image detection model is improved.
  • FIG. 6 is a schematic diagram of a specific application of the parameter debugging method for image processing according to an embodiment of the present application.
  • the execution subject of the parameter debugging method for image processing described in FIG. 6 may be the parameter debugging device 401 shown in FIG. 4 .
  • Step 601 using the camera 101 to collect a plurality of sample image data E.
  • the plurality of sample image data E are single-channel linear RAW image data of high bit depth (eg, 16 bits, 20 bits, or 24 bits).
  • Step 602 using the image processing algorithm T to process the sample image data E, to generate the processed image data F for presentation on the display screen.
  • the image data F is a color image, such as an RGB image.
  • Step 603 Manually label the image data F to obtain labeling information of each sample image data E. Since the detection performed by the image detection model is target detection, the annotation information of the sample image data includes the category of the target object presented by the sample image data and the position in the sample image data.
  • Step 604 using an image processing algorithm to process the sample image data to generate image data D.
  • Step 605 Input the image data D into the image detection model to obtain an image detection result, wherein the image detection result includes the position area of the preset target object in the sample image data and the probability value of the preset target object.
  • Step 606 Construct a loss function based on the image detection result and the labeling information of the sample image data E.
  • N( ⁇ ) is used to represent the image detection model, so we have in Represents the loss function of the image detection model.
  • Y out represents the image data input by the image detection model, that is, the image data finally output by the image processing algorithm by executing multiple image processing procedures.
  • Step 607 Determine whether the loss value of the loss function reaches a preset threshold. If the preset threshold is not reached, step 508 is performed, and if the preset threshold is reached, the parameters of the image processing algorithm are saved.
  • Step 608 using the back-propagation algorithm and the gradient descent algorithm to adjust the parameters of the image detection algorithm.
  • ⁇ (t+1) is the value of the current value of ⁇ (t) after stepping a distance in the opposite direction of its gradient
  • ⁇ represents the learning rate, which is used to control the step size of each iteration.
  • the value of the edge enhancement factor ⁇ after one iteration is:
  • the adjustment process of the parameters of the image detection algorithm for executing other image processing procedures may be similar to the adjustment process of the parameters of the image detection algorithm for edge enhancement, and details are not repeated here.
  • an embodiment of the present application further provides an image detection method.
  • FIG. 7 shows a process 700 of an image detection method provided by an embodiment of the present application.
  • the execution subject of the image detection method described in FIG. 7 may be the ISP processor and the AI processor described in FIG. 1 .
  • the image detection method includes the following steps:
  • Step 701 the image data to be detected is collected by the camera 101 .
  • Step 702 using an image processing algorithm to process the image data to be detected to generate a processed image.
  • Step 703 Input the processed image into the image detection model to obtain a detection result.
  • the parameters of the image processing algorithm described in step 702 may be obtained after debugging using the parameter debugging method for image processing as described in FIG. 5 .
  • the image detection model is used to perform at least one of the following detection tasks: labeling of detection frames, recognition of target objects, prediction of confidence levels, and prediction of motion trajectories of target objects.
  • the electronic device includes corresponding hardware and/or software modules for executing each function.
  • the present application can be implemented in hardware or in the form of a combination of hardware and computer software in conjunction with the algorithm steps of each example described in conjunction with the embodiments disclosed herein. Whether a function is performed by hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functionality for each particular application in conjunction with the embodiments, but such implementations should not be considered beyond the scope of this application.
  • the above one or more processors may be divided into functional modules according to the foregoing method examples.
  • each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module.
  • the above-mentioned integrated modules can be implemented in the form of hardware. It should be noted that, the division of modules in this embodiment is schematic, and is only a logical function division, and there may be other division manners in actual implementation.
  • FIG. 8 shows a possible schematic diagram of the composition of the image detection apparatus 800 involved in the above embodiment.
  • the image detection apparatus 800 may include: Collection module 801 , processing module 802 and detection module 803 .
  • the acquisition module 801 is configured to collect image data to be detected through the first camera device; the processing module 802 is configured to process the image data to be detected by using an image processing algorithm to generate a processed image; the detection module 803 , which is configured to input the processed image into an image detection model to obtain a detection result; wherein, the parameters of the image processing algorithm are the annotations of the first sample image data collected by comparing the first camera device The information and the detection result of the image detection model on the first sample image data are obtained by adjusting based on the comparison result.
  • the image detection model is obtained by performing neural network training on the second sample image data collected by the second camera device.
  • the parameters of the image processing algorithm are determined by a parameter adjustment module, and the parameter adjustment module (not shown in the figure) includes: a comparison sub-module (not shown in the figure), which is is configured to compare the detection result of the first sample image data with the annotation information of the first sample image data to obtain the comparison result; the adjustment sub-module (not shown in the figure) is configured to be based on The comparison result iteratively adjusts the parameters of the image processing algorithm; the saving sub-module (not shown in the figure) is configured to save the parameters of the image processing algorithm when a preset condition is satisfied.
  • a comparison sub-module not shown in the figure
  • the comparison result is an error
  • the adjustment sub-module (not shown in the figure) is further configured to: based on the detection result of the first sample image data and the first The error between the annotation information of the sample image data is used to construct a target loss function, wherein the target loss function includes the parameters to be updated in the image processing algorithm; based on the target loss function, using the back propagation algorithm and A gradient descent algorithm that iteratively updates the parameters of the image processing algorithm.
  • the image processing algorithm includes at least one of the following: dark current correction, lens shading correction, demosaicing, white balance correction, tone mapping, contrast enhancement, image edge enhancement, and image noise reduction.
  • the parameters of the image processing algorithm include at least one of the following: the distance between each pixel of the image in the lens shading correction algorithm and the optical center of the camera device; the neutral color area in the white balance correction algorithm is in the image the boundary coordinates in the tone mapping algorithm; the target brightness, target saturation, and filter kernel parameters used to generate the low-pass filtered image in the tone mapping algorithm; the contrast threshold in the contrast enhancement algorithm; the edge enhancement factor in the image edge enhancement algorithm; and the image The spatial domain Gaussian parameter and the pixel value domain Gaussian parameter in the noise reduction algorithm.
  • the image processing algorithm is executed by a trained image processing model; the parameters of the image processing algorithm include: weight coefficients of the neural network used to generate the image processing model.
  • the annotation information of the first sample image data is manually annotated; and the apparatus further includes: a conversion module (not shown in the figure) configured to convert the first sample image data into The sample image data is converted into color images suitable for human annotation.
  • a conversion module (not shown in the figure) configured to convert the first sample image data into The sample image data is converted into color images suitable for human annotation.
  • the image detection model is used to perform at least one of the following detection tasks: labeling of detection frames, recognition of target objects, prediction of confidence levels, and prediction of motion trajectories of target objects.
  • the image detection apparatus 800 provided in this embodiment is configured to execute the image detection method executed by the electronic device 100, and can achieve the same effect as the above-mentioned implementation method.
  • the modules corresponding to FIG. 8 can be implemented in software, hardware or a combination of the two.
  • each module can be implemented in software to drive the ISP 102 and the AI processor 103 in the electronic device 100 shown in FIG. 1 .
  • each module may include a corresponding processor and a corresponding driver software.
  • FIG. 9 shows a possible schematic diagram of the composition of the parameter debugging apparatus 900 for image processing involved in the above embodiment.
  • the image processing parameter debugging apparatus 900 may include: a processing module 901 , a detection module 902 , a comparison module 903 and an adjustment module 904 .
  • the processing module 901 is configured to perform image processing on the first sample image data by using an image processing algorithm to generate first image data, wherein the first sample image data is collected by a first camera; the detection module 902 , is configured to input the first image data into a pre-trained image detection model to obtain a detection result; a comparison module 903 is configured to compare the detection result with the annotation information of the first sample image data The error is obtained, and a comparison result is obtained; the adjustment module 904 is configured to adjust the parameters of the image processing algorithm based on the comparison result.
  • the image detection model is obtained by performing neural network training on the second sample image data collected by the second camera device.
  • the comparison result is an error
  • the adjustment module is configured to: construct a target based on the error between the detection result and the annotation information of the first sample image data A loss function, wherein the target loss function includes parameters to be adjusted in the image processing algorithm; based on the target loss function, the parameters of the image processing algorithm are iteratively adjusted by using a backpropagation algorithm and a gradient descent algorithm.
  • the image processing algorithm includes at least one of the following: dark current correction, lens shading correction, demosaicing, white balance correction, tone mapping, contrast enhancement, image edge enhancement, or image noise reduction.
  • the parameters of the image processing algorithm include at least one of the following: the distance between each pixel of the image in the lens shading correction algorithm and the optical center of the camera device; the neutral color area in the white balance correction algorithm is in the image the boundary coordinates in the tone mapping algorithm; the target brightness, target saturation, and filter kernel parameters used to generate the low-pass filtered image in the tone mapping algorithm; the contrast threshold in the contrast enhancement algorithm; the edge enhancement factor in the image edge enhancement algorithm; and the image The spatial domain Gaussian parameter and the pixel value domain Gaussian parameter in the noise reduction algorithm.
  • the image processing algorithm is executed by a trained image processing model; the parameters of the image processing algorithm include: weight coefficients of the neural network used to generate the image processing model.
  • the annotation information of the first sample image data is manually annotated; and the apparatus further includes: a conversion module (not shown in the figure) configured to convert the first sample image data into The sample image data is converted into color images suitable for human annotation.
  • a conversion module (not shown in the figure) configured to convert the first sample image data into The sample image data is converted into color images suitable for human annotation.
  • the image detection model is used to perform at least one of the following detection tasks: labeling of detection frames, recognition of target objects, prediction of confidence levels, and prediction of motion trajectories of target objects.
  • the image detection apparatus 800 may include at least one processor and memory. Wherein, at least one processor can call all or part of the computer program stored in the memory to control and manage the actions of the electronic device 100, for example, it can be used to support the electronic device 100 to perform the steps performed by the above-mentioned modules.
  • the memory may be used to support the execution of the electronic device 100 by storing program codes and data, and the like.
  • the processor may implement or execute various exemplary logic modules described in conjunction with the present disclosure, which may be a combination of one or more microprocessors that implement computing functions, such as, but not limited to, the image signal shown in FIG. 1 . Processor 101 and AI Processor 103.
  • the microprocessor combination may also include a central processing unit, a controller, and the like.
  • the processor may include other programmable logic devices, transistor logic devices, or discrete hardware components in addition to the processors shown in FIG. 1 .
  • the memory may include random access memory (RAM) and read only memory (ROM), among others.
  • the random access memory may include volatile memory (such as SRAM, DRAM, DDR (Double Data Rate SDRAM, Double Data Rate SDRAM) or SDRAM, etc.) and non-volatile memory.
  • the RAM can store data (such as image processing algorithms, etc.) and parameters required for the operation of the ISP102 and the AI processor 103, intermediate data generated by the operation of the ISP102 and the AI processor 103, image data processed by the ISP102, and the AI processor 103.
  • Executable programs of the ISP 102 and the AI processor 103 may be stored in the read-only memory ROM. Each of the above components can perform their own work by loading an executable program.
  • the executable program stored in the memory can execute the image detection method as described in FIG. 7 .
  • the image detection apparatus 900 may include at least one processor and storage device. Wherein, at least one processor can call all or part of the computer program stored in the memory to control and manage the actions of the parameter debugging device 401 as shown in FIG. step.
  • the memory may be used to support the execution of the parameter debugging device 401 to store program codes and data, and the like.
  • the processor can implement or execute various exemplary logic modules described in conjunction with the disclosure of the present application, which can be one or more microprocessor combinations that implement computing functions, including but not limited to a central processing unit and a controller, etc. .
  • the processor may also include other programmable logic devices, transistor logic devices, or discrete hardware components, or the like.
  • the memory may include random access memory (RAM), read only memory ROM, and the like.
  • the random access memory can include volatile memory (such as SRAM, DRAM, DDR (Double Data Rate SDRAM, Double Data Rate SDRAM) or SDRAM, etc.) and non-volatile memory.
  • the RAM may store data (such as image processing algorithms) and parameters required for the operation of the parameter debugging device 401, intermediate data generated by the parameter debugging device 401, and output results after the parameter debugging device 401 runs.
  • An executable program of the parameter debugging device 401 may be stored in the read-only memory ROM. Each of the above components can perform their own work by loading an executable program.
  • the executable program stored in the memory can execute the parameter adjustment method described in FIG. 5 or FIG. 6 .
  • This embodiment further provides a computer-readable storage medium, where computer instructions are stored in the computer-readable storage medium, and when the computer instructions are executed on the computer, the computer executes the above-mentioned related method steps to realize the image detection in the above-mentioned embodiment.
  • the image detection method of the apparatus 800, or the parameter adjustment method of the parameter adjustment apparatus 900 in the above-mentioned embodiment is implemented.
  • This embodiment also provides a computer program product, when the computer program product is run on a computer, it causes the computer to execute the above-mentioned relevant steps, so as to realize the image detection method of the image detection apparatus 800 in the above-mentioned embodiment, or to realize the above-mentioned embodiment.
  • the computer-readable storage medium or computer program product provided in this embodiment is used to execute the corresponding method provided above. Therefore, for the beneficial effect that can be achieved, reference may be made to the corresponding method provided above. The beneficial effects will not be repeated here.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.
  • the integrated unit if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a readable storage medium.
  • a readable storage medium includes several instructions to make a device (which may be a single chip microcomputer, a chip, etc.) or a processor (processor) to execute all or part of the steps of the methods in the various embodiments of the present application.
  • the aforementioned readable storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc. that can store program codes. medium.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Studio Devices (AREA)
  • Image Analysis (AREA)

Abstract

Les modes de réalisation de la présente demande concernent un procédé de détection, un appareil et un dispositif électronique. Le procédé de détection d'image consiste à : acquérir, au moyen d'un premier appareil de caméra, des données d'image à détecter ; utiliser un algorithme de traitement d'image pour traiter lesdites données d'image afin de générer une image traitée ; et entrer l'image traitée dans un modèle de détection d'image afin d'obtenir un résultat de détection, les paramètres de l'algorithme de traitement d'image étant obtenus en comparant les informations d'annotation des premières données d'image d'échantillon acquises par le premier appareil de caméra avec le résultat de détection du modèle de détection d'image sur les premières données d'image d'échantillon, puis en effectuant un ajustement d'après le résultat de la comparaison. Selon la présente demande, le procédé de détection d'image améliore la précision d'inférence d'un modèle de détection d'image lorsqu'un nouvel appareil photographique est combiné à un modèle de détection d'image appris.
PCT/CN2021/078478 2021-03-01 2021-03-01 Procédé de détection, appareil et dispositif électronique WO2022183321A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2021/078478 WO2022183321A1 (fr) 2021-03-01 2021-03-01 Procédé de détection, appareil et dispositif électronique
CN202180093086.5A CN116888621A (zh) 2021-03-01 2021-03-01 图像检测方法、装置和电子设备

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/078478 WO2022183321A1 (fr) 2021-03-01 2021-03-01 Procédé de détection, appareil et dispositif électronique

Publications (1)

Publication Number Publication Date
WO2022183321A1 true WO2022183321A1 (fr) 2022-09-09

Family

ID=83153711

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/078478 WO2022183321A1 (fr) 2021-03-01 2021-03-01 Procédé de détection, appareil et dispositif électronique

Country Status (2)

Country Link
CN (1) CN116888621A (fr)
WO (1) WO2022183321A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117830295A (zh) * 2024-02-21 2024-04-05 广州搏辉特自动化设备有限公司 缠绕机自动调节缠绕参数的控制方法、系统、设备及介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150178544A1 (en) * 2012-06-15 2015-06-25 Seref Sagiroglu System for estimating gender from fingerprints
CN108171748A (zh) * 2018-01-23 2018-06-15 哈工大机器人（合肥）国际创新研究院 一种面向机器人智能抓取应用的视觉识别与定位方法
CN109961443A (zh) * 2019-03-25 2019-07-02 北京理工大学 基于多期ct影像引导的肝脏肿瘤分割方法及装置
CN110458095A (zh) * 2019-08-09 2019-11-15 厦门瑞为信息技术有限公司 一种有效手势的识别方法、控制方法、装置和电子设备
CN112101328A (zh) * 2020-11-19 2020-12-18 四川新网银行股份有限公司 一种深度学习中识别并处理标签噪声的方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150178544A1 (en) * 2012-06-15 2015-06-25 Seref Sagiroglu System for estimating gender from fingerprints
CN108171748A (zh) * 2018-01-23 2018-06-15 哈工大机器人（合肥）国际创新研究院 一种面向机器人智能抓取应用的视觉识别与定位方法
CN109961443A (zh) * 2019-03-25 2019-07-02 北京理工大学 基于多期ct影像引导的肝脏肿瘤分割方法及装置
CN110458095A (zh) * 2019-08-09 2019-11-15 厦门瑞为信息技术有限公司 一种有效手势的识别方法、控制方法、装置和电子设备
CN112101328A (zh) * 2020-11-19 2020-12-18 四川新网银行股份有限公司 一种深度学习中识别并处理标签噪声的方法

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117830295A (zh) * 2024-02-21 2024-04-05 广州搏辉特自动化设备有限公司 缠绕机自动调节缠绕参数的控制方法、系统、设备及介质

Also Published As

Publication number Publication date
CN116888621A (zh) 2023-10-13

Similar Documents

Publication Publication Date Title
US11882357B2 (en) Image display method and device
US20220188999A1 (en) Image enhancement method and apparatus
US20200234414A1 (en) Systems and methods for transforming raw sensor data captured in low-light conditions to well-exposed images using neural network architectures
WO2021164234A1 (fr) Procédé de traitement d'image et dispositif de traitement d'image
CN108780508B (zh) 用于归一化图像的系统和方法
WO2020001196A1 (fr) Procédé de traitement d'images, dispositif électronique et support d'informations lisible par ordinateur
CN110276767A (zh) 图像处理方法和装置、电子设备、计算机可读存储介质
US11776129B2 (en) Semantic refinement of image regions
CN113065645B (zh) 孪生注意力网络、图像处理方法和装置
US11967040B2 (en) Information processing apparatus, control method thereof, imaging device, and storage medium
CN112581379A (zh) 图像增强方法以及装置
US20220070369A1 (en) Camera Image Or Video Processing Pipelines With Neural Embedding
CN113379609B (zh) 一种图像处理方法、存储介质及终端设备
CN115239581A (zh) 一种图像处理方法及相关装置
WO2022183321A1 (fr) Procédé de détection, appareil et dispositif électronique
US20230222639A1 (en) Data processing method, system, and apparatus
WO2022193132A1 (fr) Procédé et appareil de détection d'image, et dispositif électronique
US20230132230A1 (en) Efficient Video Execution Method and System
US20230239553A1 (en) Multi-sensor imaging color correction
CN112991236B (zh) 一种基于模板的图像增强方法及装置
CN115731115A (zh) 一种数据处理方法以及装置
WO2023028866A1 (fr) Procédé et appareil de traitement d'image, et véhicule
Wei et al. Enhanced Object Detection by Integrating Camera Parameters into Raw Image-Based Faster R-CNN
US20240031512A1 (en) Method and system for generation of a plurality of portrait effects in an electronic device
WO2022041506A1 (fr) Procédé d'estimation de profondeur d'image, équipement terminal et support d'enregistrement lisible par ordinateur

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21928412

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 202180093086.5

Country of ref document: CN

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21928412

Country of ref document: EP

Kind code of ref document: A1