WO2024082183A1 - Procédé et appareil de réglage de paramètre, et terminal intelligent - Google Patents

Procédé et appareil de réglage de paramètre, et terminal intelligent Download PDF

Info

Publication number
WO2024082183A1
WO2024082183A1 PCT/CN2022/126256 CN2022126256W WO2024082183A1 WO 2024082183 A1 WO2024082183 A1 WO 2024082183A1 CN 2022126256 W CN2022126256 W CN 2022126256W WO 2024082183 A1 WO2024082183 A1 WO 2024082183A1
Authority
WO
WIPO (PCT)
Prior art keywords
parameter set
sub
isp
image
isp parameter
Prior art date
Application number
PCT/CN2022/126256
Other languages
English (en)
Chinese (zh)
Inventor
杨攀
张兴亚
邱守谦
王超
崔泽波
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to PCT/CN2022/126256 priority Critical patent/WO2024082183A1/fr
Publication of WO2024082183A1 publication Critical patent/WO2024082183A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules

Definitions

  • the present application relates to the field of image processing technology, and in particular to a parameter adjustment method, device and intelligent terminal.
  • image recognition has been widely used in various fields, such as transportation, intelligent security, autonomous driving, mobile terminals, etc.
  • image recognition is the core of autonomous driving.
  • Image quality can directly affect the accuracy of image recognition, and image signal processing (ISP) parameters determine the quality of image.
  • ISP parameters can ultimately determine the accuracy of image recognition.
  • ISP parameters are adjusted manually. That is, engineers adjust ISP parameters according to experience, apply the adjusted ISP parameters to image signal processing, and determine the image quality and subjective effect of the image obtained according to the ISP parameters. If the image obtained by processing with the adjusted ISP parameters does not meet the set indicators of image quality and subjective effect, the engineer adjusts the ISP parameters again based on experience until the image obtained according to the adjusted ISP parameters meets the set indicators of image quality and subjective effect, and the ISP parameters that meet the set indicators are used as the final ISP parameters.
  • the above method requires engineers to repeatedly adjust the ISP parameters, which will result in high labor costs and low efficiency.
  • the present application provides a parameter adjustment method, device and intelligent terminal, which can autonomously adjust ISP parameters without investing a lot of manpower costs, thereby improving the efficiency of ISP parameter adjustment, and can also improve the image recognition rate while ensuring human eye vision.
  • the present application provides a parameter adjustment method, in which a second ISP parameter set is determined based on a training image set and a first image signal processing ISP parameter set, the first ISP parameter set being the ISP parameter set corresponding to the smart terminal, a test image set is processed based on the second ISP parameter set, and a first quality score and a first recognition rate are determined based on the processed test image set, the first quality score indicating the image quality of the processed test image set, and the first recognition rate indicating the image recognition status of the processed test image set, if the first quality score is greater than the first reference quality score and the first recognition rate is greater than the first reference recognition rate, the ISP parameter set corresponding to the smart terminal is adjusted from the first ISP parameter set to the second ISP parameter set, the first reference quality score indicating the image quality of the test image set after being processed by the first ISP parameter set, and the first reference recognition rate indicating the image recognition status of the test image set after being processed by the first ISP parameter set.
  • the present application automatically determines the second ISP parameters based on the training image set and the first ISP parameter set, without manually adjusting the ISP parameters repeatedly, thus reducing the labor cost.
  • the first quality score represents the image quality of the test image set after being processed by the second ISP parameter set
  • the first recognition rate represents the image recognition of the test image set after being processed by the second ISP parameter set
  • the first reference quality score represents the image quality of the test image set after being processed by the first ISP parameter set
  • the first reference recognition rate represents the image recognition of the test image set after being processed by the first ISP parameter set.
  • the first quality score is greater than the first reference quality score, and the first recognition rate is greater than the first reference recognition rate, it means that the image quality after being processed by the second ISP parameter set is better than that of the first ISP parameter set, and the image recognition rate after being processed by the second ISP parameter set is also higher than that of the first ISP parameter set.
  • adjusting the ISP parameter set corresponding to the smart terminal from the first ISP parameter set to the second ISP parameter set can not only ensure that the quality of the image obtained by the smart terminal according to the second ISP parameter set is better than the image quality obtained according to the first ISP parameter set, but also ensure that the recognition rate of the image obtained by the smart terminal according to the second ISP parameter set is also better than the recognition rate of the image obtained according to the first ISP parameter set.
  • the method provided in the present application can not only autonomously adjust ISP parameters and improve the efficiency of ISP parameter adjustment without investing a lot of manpower costs, but also improve the image recognition rate while ensuring human eye vision.
  • the training image set includes N sub-training sets
  • the first ISP parameter set includes M parameters
  • each of the N sub-training sets corresponds to a scene
  • N and M are both integers greater than 1.
  • the following operations are performed for each of the N sub-training sets: one of the sub-training sets is used as a target sub-training set, and a first sub-parameter set corresponding to a target scene in the first ISP parameter set is adjusted to obtain a second sub-parameter set corresponding to a target scene in a second ISP parameter set, the target scene is a scene corresponding to the target sub-training set, the first sub-parameter set includes at least one parameter of the M parameters, and the second sub-parameter set includes at least one parameter of the M parameters.
  • the N sub-training sets have one-to-one corresponding N scenes, each of the N scenes corresponds to a sub-parameter set in the first ISP parameter set, and each sub-parameter set includes at least one parameter of the M parameters.
  • the sub-training set is called a target sub-training set
  • the scene corresponding to the target sub-training set is called a target scene
  • the sub-parameter set corresponding to the target scene in the first ISP parameter is called a first sub-parameter set.
  • the sub-parameter set corresponding to the target scene in the second ISP parameter i.e., the second sub-parameter set
  • the sub-parameter set corresponding to each scene in the second ISP parameter set can be obtained.
  • the implementation process of adjusting the first sub-parameter set corresponding to the target scene in the first ISP parameter set to obtain the second sub-parameter set corresponding to the target scene in the second ISP parameter set includes: adjusting the first sub-parameter set corresponding to the target scene in the first ISP parameter set to multiple candidate sub-parameter sets, keeping the remaining parameters unchanged to obtain multiple candidate ISP parameter sets, and determining multiple candidate quality scores corresponding to the target sub-training set based on the multiple candidate ISP parameter sets, the multiple candidate quality scores indicating the image quality of the target sub-training set after being processed by the multiple candidate ISP parameter sets, and determining the parameter corresponding to the target scene in the candidate ISP parameter set corresponding to the maximum candidate quality score among the multiple candidate quality scores as the second sub-parameter set.
  • the parameter corresponding to the target scene in the candidate ISP parameter set corresponding to the maximum candidate quality score among multiple candidate quality scores is determined as the second sub-parameter set. In this way, it can be ensured that the image processing effect of the second sub-parameter set finally determined is the best among the multiple candidate ISP parameter sets, further ensuring the adjustment effect of the ISP parameter set.
  • the test image set includes N sub-test sets, which correspond one-to-one to the N sub-training sets.
  • the intelligent terminal may also determine a second quality score, the second quality score indicating the image quality of the target sub-training set after being processed by the first ISP parameter set.
  • the first sub-parameter set corresponding to the target scene in the first ISP parameter set is adjusted in the above manner, the second reference quality score indicating the image quality of the target sub-test set after being processed by the first ISP parameter set, and the target sub-test set is the sub-test set corresponding to the target scene. If the second quality score is greater than or equal to the second reference quality score, the first sub-parameter set corresponding to the target scene in the first ISP parameter set is not adjusted.
  • the second quality score is less than the second reference quality score, it means that the processing effect of the images in the target sub-training set by the first ISP parameter set is not good, and the ISP parameters corresponding to the target scene need to be adjusted, so the step of adjusting the first sub-parameter set corresponding to the target scene in the first ISP parameter set is performed. If the second quality score is greater than or equal to the second reference quality score, it means that the processing effect of the images in the target sub-training set by the first ISP parameter set is good, and the ISP parameters corresponding to the target scene do not need to be adjusted, so the step of adjusting the first sub-parameter set corresponding to the target scene in the first ISP parameter set is not performed.
  • N sub-parameter sets can be obtained. Since each sub-parameter set includes at least one parameter of the M parameters, the union of the N sub-parameter sets may include each parameter of the M parameters, or may only include some parameters of the M parameters. In different cases, the methods for determining the second ISP parameter set based on the N sub-parameter sets are different, which will be introduced below.
  • the union of the N sub-parameter sets includes each parameter in the M parameters.
  • a second ISP parameter set can be determined based on the N sub-parameter sets.
  • the sub-parameter sets corresponding to different scenarios may or may not have intersections.
  • the N sub-parameter sets can be directly merged to obtain a second ISP parameter set.
  • the parameters with intersections are called first-category parameters, and the parameters without intersections are called second-category parameters.
  • the parameter values of the first-category parameters in the N sub-parameter sets are combined to obtain multiple parameter value combinations, and the multiple parameter value combinations are merged with the parameter values of the second-category parameters in the N sub-parameter sets to obtain multiple second ISP parameter sets.
  • the union of the N sub-parameter sets includes some parameters in the M parameters.
  • the second ISP parameter set is determined based on the N sub-parameter sets and the first ISP parameter set.
  • the sub-parameter sets corresponding to different scenarios may or may not have intersections.
  • the N sub-parameter sets can be directly merged to obtain a merged parameter set, and the merged parameter set can be merged again with other parameters in the first ISP parameter set except the merged parameter set to obtain a second ISP parameter set.
  • the parameters with intersections are called first-category parameters, and the parameters without intersections are called second-category parameters.
  • the parameter values of the first-category parameters in the N sub-parameter sets are combined to obtain multiple parameter value combinations, and the multiple parameter value combinations are merged with the parameter values of the second-category parameters in the N sub-parameter sets to obtain multiple merged parameter sets.
  • the merged parameter set is merged again with other parameters in the first ISP parameter set except the merged parameter set to obtain multiple second ISP parameter sets.
  • the smart terminal before determining the second ISP parameter set based on the training image set and the first ISP parameter set, can also obtain an image corresponding to the current environment, process the image corresponding to the current environment based on the first ISP parameter set, and perform scene recognition based on the processed image to obtain a scene recognition result, which indicates whether there is a key scene in the current environment. If the scene recognition result indicates that there is a key scene in the current environment, the image corresponding to the current environment is stored in the training image set.
  • the training image set includes N sub-training sets, each of which corresponds to a scene. If the scene indicated by the scene recognition result is a key scene, the image corresponding to the current environment is stored in the sub-training set of the corresponding scene.
  • the images in the training image set can be images taken by the camera in real time. Since different users have different behavioral habits, the images taken by the camera are also different. Therefore, adjusting the ISP parameter set based on the images taken by the camera in real time can make the final adjusted ISP parameters adaptive.
  • the images in the training image set can also be images obtained by other means.
  • the images in the training image set are images that have not been processed by the ISP parameter set.
  • the adjusted ISP parameter set can be applied to the images that have not been processed by the ISP parameter set, so as to determine the quality of the adjusted ISP parameter set.
  • the number of the second ISP parameter sets may be one or more. If there are multiple second ISP parameter sets, the test image set is processed based on each second ISP parameter set in the multiple second ISP parameter sets, thereby obtaining multiple processed test image sets.
  • the processed test image set may be one or more.
  • the methods for determining the first quality score and the first recognition rate are different, which will be introduced below.
  • the first quality score and the first recognition rate are determined based on the processed test image set.
  • the quality scores and recognition rates of the multiple processed test image sets are determined, the maximum quality score among the quality scores of the multiple processed test image sets is used as the first quality score, and the recognition rate corresponding to the maximum quality score is used as the first recognition rate.
  • the ISP parameter set corresponding to the smart terminal is not adjusted from the first ISP parameter set to the second ISP parameter set, that is, the ISP parameter set corresponding to the smart terminal remains unchanged.
  • the first quality score is greater than the first reference quality score and the first recognition rate is greater than the first reference recognition rate, it means that the image quality after being processed by the second ISP parameter is better than the first ISP parameter, and the image recognition rate after being processed by the second ISP parameter is also higher than the first ISP parameter, so the ISP parameter set corresponding to the smart terminal can be adjusted from the first ISP parameter set to the second ISP parameter set.
  • the first quality score is not greater than the first reference quality score and ⁇ or the first recognition rate is not greater than the first reference recognition rate, it means that the image quality after being processed by the second ISP parameter is not as good as the first ISP parameter, and the image recognition rate after being processed by the second ISP parameter is also lower than the first ISP parameter, so the ISP parameter set corresponding to the smart terminal is not adjusted from the first ISP parameter set to the second ISP parameter set.
  • the first reference recognition rate may be updated to the first recognition rate.
  • the first reference recognition rate is updated to the first recognition rate and the first reference quality score is updated to the first quality score. If the first quality score is not greater than the first reference quality score and/or the first recognition rate is not greater than the first reference recognition rate, the first reference recognition rate and the first reference quality score are kept unchanged. This ensures that the ISP parameter set adjusted subsequently is better than the previous ISP parameter set, thereby further improving the efficiency of ISP parameter adjustment and the recognition rate of the image.
  • a parameter adjustment device which has the function of implementing the parameter adjustment method in the first aspect.
  • the parameter adjustment device includes at least one module, which is used to implement the parameter adjustment method provided in the first aspect.
  • a parameter adjustment device comprising a processor and a memory, the memory being used to store a computer program for executing the parameter adjustment method provided in the first aspect.
  • the processor is configured to execute the computer program stored in the memory to implement the parameter adjustment method described in the first aspect.
  • the parameter adjustment device may further include a communication bus, and the communication bus is used to establish a connection between the processor and the memory.
  • a smart terminal comprising the parameter adjustment device described in the second or third aspect.
  • a computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the computer executes the parameter adjustment method described in the first aspect.
  • a computer program product comprising instructions is provided, and when the instructions are executed on a computer, the computer executes the steps of the parameter adjustment method described in the first aspect.
  • a computer program is provided, and when the computer program is executed on a computer, the computer executes the parameter adjustment method described in the first aspect.
  • FIG1 is a schematic diagram of the structure of an intelligent vehicle provided by an exemplary embodiment of the present application.
  • FIG2 is a schematic diagram of the structure of a parameter adjustment device provided by an exemplary embodiment of the present application.
  • FIG3 is a flow chart of a parameter adjustment method provided by an exemplary embodiment of the present application.
  • FIG4 is a flow chart of a parameter adjustment method provided by another exemplary embodiment of the present application.
  • FIG. 5 is a schematic diagram of the structure of a parameter adjustment device provided by an exemplary embodiment of the present application.
  • image recognition has been widely used in various fields, such as transportation, intelligent security, autonomous driving, mobile terminal, etc.
  • a smart vehicle can be equipped with a camera to capture the environment around the smart vehicle to obtain images around the smart vehicle, process the images around the smart vehicle based on the ISP parameter set, and then recognize the processed images to determine the scene of the image, objects in the image, etc.
  • the smart vehicle determines people, cars, lane lines, bicycles, traffic signs, etc. in the image.
  • the smart vehicle can realize driving reminders, assisted driving, and autonomous driving based on the identified scenes and objects. Therefore, image recognition is the core of realizing autonomous driving, and image quality can directly affect the accuracy of image recognition, and ISP parameters determine the quality of image quality. In other words, ISP parameters can ultimately determine the accuracy of image recognition.
  • the ISP parameters can be adjusted manually. However, engineers are required to adjust the ISP parameters repeatedly, which will result in high labor costs and low efficiency. After the image is processed by the adjusted ISP parameters, although the human eye perceives the image quality of the image well, the accuracy of image recognition through the intelligent terminal is not good. Therefore, the embodiment of the present application provides a parameter adjustment method, which can realize the autonomous adjustment of ISP parameters without investing a lot of manpower costs, thereby improving the efficiency of ISP parameter adjustment, and can also ensure that the image conforms to the human eye vision and improve the image recognition rate.
  • the method provided by the embodiment of the present application is briefly introduced. Please refer to Figure 1.
  • the intelligent vehicle is equipped with a computing platform, for example, a mobile data center (MDC) and a camera is installed.
  • the computing platform is equivalent to the brain of the intelligent vehicle and can process various types of data.
  • the computing platform is located inside the intelligent vehicle and is powered by the battery of the intelligent vehicle.
  • the computing platform can support the power supply of several (usually several to more than a dozen) cameras to support the operation of the cameras.
  • the camera is a peripheral of the computing platform and is installed on the windshield of the intelligent vehicle or on the outside of the intelligent vehicle.
  • the camera can be used to capture the environment around the smart vehicle to obtain images around the smart vehicle.
  • the images captured by the camera include raw data (RAW) and embedded bitmap data (EBD).
  • the RAW and EBD are serialized by a serializer and transmitted to a deserializer for deserialization through a mobile industry processor interface (MIPI) protocol.
  • the deserialized data is then processed by applying an ISP parameter set to obtain a processed image.
  • the processed image format may be YUV420NV12, YUV420NV21, etc.
  • the processed image is identified by an image perception model/algorithm to determine the scene of the image, and the images of key scenes are stored in a training image set of a memory.
  • a test image set is also built into the memory.
  • the smart vehicle performs image quality evaluation on the training image set and the test image set based on an image quality evaluation system, and then adjusts the ISP parameters based on the image quality evaluation results.
  • the image quality assessment system is located on a system-on-chip (SoC) of a computing platform, and the image quality assessment system includes multiple image quality assessment algorithms, which include but are not limited to clarity, color, noise, white balance, wide dynamic and other image quality assessment algorithms.
  • SoC system-on-chip
  • the method provided in the embodiment of the present application can be executed by any intelligent terminal with image signal processing function, for example, the intelligent terminal can be a personal computer (PC), a mobile phone, a personal digital assistant (PDA), a handheld computer PPC (pocket PC), a tablet computer, a server, a robot, an intelligent driving device, a vehicle-mounted computing platform, etc.
  • the intelligent driving device in the present application can include land vehicles, water vehicles, air vehicles, industrial equipment, agricultural equipment, or entertainment equipment, etc.
  • the intelligent driving device can be a vehicle, which is a vehicle in a broad sense, and can be a vehicle (such as a commercial vehicle, a passenger car, a motorcycle, a flying car, a train, etc.), an industrial vehicle (such as a forklift, a trailer, a tractor, etc.), an engineering vehicle (such as an excavator, a bulldozer, a crane, etc.), agricultural equipment (such as a mower, a harvester, etc.), amusement equipment, a toy vehicle, etc.
  • the embodiment of the present application does not specifically limit the type of vehicle.
  • the intelligent driving device can be a vehicle such as an airplane or a ship.
  • the parameter adjustment device can be deployed on an intelligent terminal.
  • the parameter adjustment device includes at least one processor 201, a communication bus 202, a memory 203 and at least one communication interface 204.
  • the processor 201 may be a general-purpose central processing unit (CPU), a network processor (NP), a microprocessor, or may be one or more integrated circuits for implementing the solution of the present application, such as an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof.
  • the above-mentioned PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.
  • the communication bus 202 is used to transmit information between the above components.
  • the communication bus 202 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
  • the memory 203 may be a read-only memory (ROM), a random access memory (RAM), an electrically erasable programmable read-only memory (EEPROM), an optical disc (including a compact disc read-only memory (CD-ROM), a compressed optical disc, a laser disc, a digital versatile disc, a Blu-ray disc, etc.), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.
  • the memory 203 may exist independently and be connected to the processor 201 via the communication bus 202.
  • the memory 203 may also be integrated with the processor 201.
  • the communication interface 204 uses any transceiver-like device for communicating with other devices or communication networks.
  • the communication interface 204 includes a wired communication interface and may also include a wireless communication interface.
  • the wired communication interface may be, for example, an Ethernet interface.
  • the Ethernet interface may be an optical interface, an electrical interface, or a combination thereof.
  • the wireless communication interface may be a wireless local area network (WLAN) interface, a cellular network communication interface, or a combination thereof, etc.
  • WLAN wireless local area network
  • the processor 201 may include one or more CPUs, such as CPU0 and CPU1 shown in FIG2 .
  • the processor 201 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content to be displayed on the display screen.
  • the processor 201 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning and intelligent driving.
  • AI artificial intelligence
  • the parameter adjustment device may include multiple processors, such as processor 201 and processor 205 shown in Figure 2. Each of these processors may be a single-core processor or a multi-core processor.
  • the processor here may refer to one or more devices, circuits, and/or processing cores for processing data (such as computer program instructions).
  • the parameter adjustment device may further include an output device 206 and an input device 207.
  • the output device 206 communicates with the processor 201 and may display information in a variety of ways.
  • the output device 206 may be a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector.
  • the input device 207 communicates with the processor 201 and may receive user input in a variety of ways.
  • the input device 207 may be a mouse, a keyboard, a touch screen device, or a sensor device.
  • the memory 203 is used to store the program code 210 for executing the solution of the present application, and the processor 201 can execute the program code 210 stored in the memory 203.
  • the program code 210 may include one or more software modules, and the parameter adjustment device can implement the parameter adjustment method provided in the embodiment of FIG. 3 below through the processor 201 and the program code 210 in the memory 203.
  • FIG3 is a flow chart of a parameter adjustment method provided in an embodiment of the present application.
  • the execution subject of the method is a smart terminal. Please refer to FIG3, the method includes the following steps.
  • Step 301 Determine a second ISP parameter set based on a training image set and a first ISP parameter set, where the first ISP parameter set is an ISP parameter set corresponding to the smart terminal.
  • the training image set includes N sub-training sets
  • the first ISP parameter set includes M parameters
  • each of the N sub-training sets corresponds to a scene
  • N and M are both integers greater than 1.
  • the following operations are performed: one of the sub-training sets is used as a target sub-training set, and a first sub-parameter set corresponding to a target scene in the first ISP parameter set is adjusted to obtain a second sub-parameter set corresponding to a target scene in a second ISP parameter set, the target scene is a scene corresponding to the target sub-training set, the first sub-parameter set includes at least one parameter of the M parameters, and the second sub-parameter set includes at least one parameter of the M parameters.
  • the N sub-training sets have one-to-one corresponding N scenes, each of the N scenes corresponds to a sub-parameter set in the first ISP parameter set, and each sub-parameter set includes at least one parameter of the M parameters.
  • the sub-training set is called a target sub-training set
  • the scene corresponding to the target sub-training set is called a target scene
  • the sub-parameter set corresponding to the target scene in the first ISP parameter is called a first sub-parameter set.
  • the sub-parameter set corresponding to the target scene in the second ISP parameter i.e., the second sub-parameter set
  • the sub-parameter set corresponding to each scene in the second ISP parameter set can be obtained.
  • the intelligent terminal stores a correspondence between the sub-training set and the scene, i.e., a first correspondence, and also stores a correspondence between the scene and the sub-parameter set, i.e., a second correspondence.
  • the scene corresponding to the target sub-training set i.e., the target scene
  • the sub-parameter set corresponding to the target scene i.e., the first sub-parameter set
  • the intelligent terminal can also store the correspondence between the sub-training set, the scene and the sub-parameter set. In this way, based on the target sub-training set, the sub-parameter set corresponding to the target scene, that is, the first sub-parameter set, can be directly determined from the correspondence between the three.
  • the first ISP parameter set is the ISP parameter set currently being applied by the intelligent terminal, and the current adjustment of the ISP parameters may be the first time or not. If the current adjustment of the ISP parameters is the first time, the parameter value corresponding to each parameter in the first ISP parameter set may be set in advance. If the current adjustment of the ISP parameters is not the first time, the parameter value corresponding to each parameter in the first ISP parameter set is obtained after the last adjustment of the ISP parameters.
  • the M parameters included in the first ISP parameter set are various parameters used for image processing.
  • the M parameters may be automatic tone remapping (ATR), dynamic range compression (DRC), gamma correction (GAMMA), raw noise fall (RAWNF), YUV noise fall (YUVNF), color correction matrix (CCM), white balance gain (AWB), etc., which is not limited in the embodiments of the present application.
  • the N scenes corresponding to the above-mentioned N sub-training sets can be key scenes for image recognition.
  • the N scenes are entering and exiting a tunnel during the day, backlighting on a sunny day, traffic intersections at night, traffic lights, viaducts, highways, and the like.
  • the sub-parameter set corresponding to each scene in the N scenes refers to a set of parameters that can affect the scene, and the sub-parameter sets corresponding to different scenes may or may not have intersections.
  • Table 1 the second corresponding relationship is shown in Table 1 below.
  • the sub-parameter set corresponding to the scene of entering and exiting a tunnel during the day includes ATR, DRC, and GAMMA
  • the sub-parameter set corresponding to the scene of backlighting on a sunny day includes DRC and GAMMA
  • the sub-parameter set corresponding to the scene of a traffic intersection at night includes RAWNF and YUVNF
  • the sub-parameter set corresponding to the scene of a traffic light includes CCM and AWB.
  • the sub-parameter sets corresponding to the scene of entering and exiting a tunnel during the day and the scene of backlighting on a sunny day have intersections
  • the sub-parameter sets corresponding to other scenes do not have intersections.
  • the implementation process of adjusting the first sub-parameter set corresponding to the target scene in the first ISP parameter set to obtain the second sub-parameter set corresponding to the target scene in the second ISP parameter set includes: adjusting the first sub-parameter set corresponding to the target scene in the first ISP parameter set to multiple candidate sub-parameter sets, keeping the remaining parameters unchanged to obtain multiple candidate ISP parameter sets, based on the multiple candidate ISP parameter sets, determining multiple candidate quality scores corresponding to the target sub-training set, the multiple candidate quality scores indicating the image quality of the target sub-training set after being processed by the multiple candidate ISP parameter sets, and determining the parameter corresponding to the target scene in the candidate ISP parameter set corresponding to the maximum candidate quality score among the multiple candidate quality scores as the second sub-parameter set.
  • the intelligent terminal stores a correspondence between parameters, adjustment ranges, and adjustment steps, i.e., a third correspondence.
  • the adjustment range and adjustment step corresponding to each parameter in the first sub-parameter set can be determined from the third correspondence.
  • the parameter is used as the target parameter, and based on the adjustment range and adjustment step corresponding to the target parameter, multiple candidate parameter values corresponding to the target parameter are determined.
  • the multiple candidate parameter values corresponding to each parameter in the first sub-parameter set are combined to obtain multiple candidate sub-parameter sets.
  • multiple candidate parameter values corresponding to the target parameter can be determined according to the following formula (1).
  • Fn refers to the n+1th candidate parameter value among multiple candidate parameter values corresponding to the target parameter
  • min refers to the minimum value in the adjustment range corresponding to the target parameter
  • max refers to the maximum value in the adjustment range corresponding to the target parameter
  • step refers to the adjustment step corresponding to the target parameter
  • N refers to a natural number.
  • the adjustment range of the target parameter is 1 to 1.6 and the adjustment step is 0.2
  • the implementation process of combining multiple candidate parameter values corresponding to each parameter in the first sub-parameter set includes: for any parameter in the first sub-parameter set, arbitrarily selecting a candidate parameter value from the multiple candidate parameter values corresponding to the parameter, selecting a candidate parameter value corresponding to each parameter in the first sub-parameter set in the same manner, and using the candidate parameter value selected for each parameter in the first sub-parameter set as a candidate sub-parameter set.
  • Multiple candidate sub-parameter sets can be obtained in the same manner, and the multiple candidate sub-parameter sets are different.
  • the first sub-parameter set includes two parameters, and the first parameter corresponds to two candidate parameter values, namely 1 and 1.5.
  • the second parameter corresponds to three candidate parameter values, namely 2, 4, and 6.
  • the candidate parameter values corresponding to the two parameters are combined to obtain multiple candidate sub-parameter sets, namely (1, 2), (1, 4), (1, 6), (1.5, 2), (1.5, 4), and (1.5, 6).
  • the implementation process of adjusting the first sub-parameter set corresponding to the target scenario in the first ISP parameter set to multiple candidate sub-parameter sets, and keeping the remaining parameters unchanged to obtain multiple candidate ISP parameter sets includes: for any one of the multiple candidate sub-parameter sets, replacing the first sub-parameter set corresponding to the target scenario in the first ISP parameter set with the candidate sub-parameter set, and keeping the other parameters in the first ISP parameter set except the first sub-parameter set unchanged, thereby obtaining a candidate ISP parameter set.
  • each candidate sub-parameter set in the multiple candidate sub-parameter sets is processed in the above manner, multiple candidate ISP parameter sets can be obtained, and each candidate ISP parameter set corresponds to a candidate sub-parameter set.
  • the implementation process of determining multiple candidate quality scores corresponding to the target sub-training set based on the multiple candidate ISP parameter sets includes: based on the multiple candidate ISP parameter sets, processing each image in the target sub-training set respectively to obtain multiple processed target sub-training sets corresponding one-to-one to the multiple candidate ISP parameter sets; based on the multiple processed target sub-training sets, determining multiple candidate quality scores, and the multiple candidate quality scores corresponding one-to-one to the multiple processed target sub-training sets.
  • the intelligent terminal can determine the image quality score of each image in the processed target sub-training set according to a relevant algorithm to obtain multiple image quality scores, and determine the candidate quality score corresponding to the processed target sub-training set based on the multiple image quality scores. In the same way, multiple candidate quality scores can be obtained, and the multiple candidate quality scores correspond to the multiple processed target sub-training sets one by one.
  • the mode of the multiple image quality scores may be determined, and the mode may be determined as the candidate quality score corresponding to the processed target sub-training set.
  • the average value of the multiple image quality scores may be determined, and the average value may be determined as the candidate quality score corresponding to the processed target sub-training set.
  • the multiple image quality scores may also be processed in other ways to obtain the candidate quality score corresponding to the processed target sub-training set, and the embodiments of the present application are not limited to this.
  • the parameter corresponding to the target scene in the candidate ISP parameter set corresponding to the maximum candidate quality score among multiple candidate quality scores is determined as the second sub-parameter set. In this way, it can be ensured that the image processing effect of the second sub-parameter set finally determined is the best among the multiple candidate ISP parameter sets, further ensuring the adjustment effect of the ISP parameter set.
  • the test image set includes N sub-test sets, which correspond one-to-one to the N sub-training sets.
  • the intelligent terminal may also determine a second quality score, the second quality score indicating the image quality of the target sub-training set after being processed by the first ISP parameter set.
  • the first sub-parameter set corresponding to the target scene in the first ISP parameter set is adjusted in the above manner, the second reference quality score indicating the image quality of the target sub-test set after being processed by the first ISP parameter set, and the target sub-test set is the sub-test set corresponding to the target scene. If the second quality score is greater than or equal to the second reference quality score, the first sub-parameter set corresponding to the target scene in the first ISP parameter set is not adjusted.
  • the second quality score is less than the second reference quality score, it means that the processing effect of the images in the target sub-training set by the first ISP parameter set is not good, and the ISP parameters corresponding to the target scene need to be adjusted, so the step of adjusting the first sub-parameter set corresponding to the target scene in the first ISP parameter set is performed. If the second quality score is greater than or equal to the second reference quality score, it means that the processing effect of the images in the target sub-training set by the first ISP parameter set is good, and the ISP parameters corresponding to the target scene do not need to be adjusted, so the step of adjusting the first sub-parameter set corresponding to the target scene in the first ISP parameter set is not performed.
  • the process of determining the second quality score and the process of determining the second reference quality score are similar to the process of determining the candidate quality score. For details, please refer to the corresponding content above, which will not be repeated here.
  • test image set is set in advance, and can be adjusted according to different needs in different situations.
  • the first sub-parameter set can be directly adjusted according to the above method to obtain the second sub-parameter set.
  • target sub-training set processed by the first ISP parameter set can also be scored to determine the processing effect of the first ISP parameter set on the images in the target sub-training set, and then determine whether the first sub-parameter set needs to be adjusted.
  • N sub-parameter sets can be obtained. Since each sub-parameter set includes at least one parameter of the M parameters, the union of the N sub-parameter sets may include each parameter of the M parameters, or may only include some parameters of the M parameters. In different cases, the methods for determining the second ISP parameter set based on the N sub-parameter sets are different, which will be introduced below.
  • the union of the N sub-parameter sets includes each parameter in the M parameters.
  • a second ISP parameter set can be determined based on the N sub-parameter sets.
  • the sub-parameter sets corresponding to different scenarios may or may not have intersections.
  • the N sub-parameter sets can be directly merged to obtain a second ISP parameter set.
  • the parameters with intersections are called first-category parameters, and the parameters without intersections are called second-category parameters.
  • the parameter values of the first-category parameters in the N sub-parameter sets are combined to obtain multiple parameter value combinations, and the multiple parameter value combinations are merged with the parameter values of the second-category parameters in the N sub-parameter sets to obtain multiple second ISP parameter sets.
  • the parameter value is merged with the parameter value of the second type of parameters in the N sub-parameter sets to obtain a second ISP parameter set.
  • each parameter value of the first type of parameters is merged with the parameter value of the second type of parameters in the N sub-parameter sets to obtain multiple second ISP parameter sets.
  • each parameter value combination is merged with the parameter value of the second-category parameters in the N sub-parameter sets to obtain a second ISP parameter set.
  • each parameter value combination is merged with the parameter value of the second-category parameters in the N sub-parameter sets to obtain multiple second ISP parameter sets.
  • the implementation method of combining multiple parameter values corresponding to each parameter in the first category of parameters is similar to the implementation method of combining multiple candidate parameter values corresponding to each parameter in the first sub-parameter set mentioned above. Please refer to the relevant content above and will not be repeated here.
  • the union of the N sub-parameter sets includes some parameters in the M parameters.
  • the second ISP parameter set is determined based on the N sub-parameter sets and the first ISP parameter set.
  • the sub-parameter sets corresponding to different scenarios may or may not have intersections.
  • the N sub-parameter sets can be directly merged to obtain a merged parameter set, and the merged parameter set can be merged again with other parameters in the first ISP parameter set except the merged parameter set to obtain a second ISP parameter set.
  • the parameters with intersections are called first-category parameters, and the parameters without intersections are called second-category parameters.
  • the parameter values of the first-category parameters in the N sub-parameter sets are combined to obtain multiple parameter value combinations, and the multiple parameter value combinations are merged with the parameter values of the second-category parameters in the N sub-parameter sets to obtain multiple merged parameter sets.
  • the merged parameter set is merged again with other parameters in the first ISP parameter set except the merged parameter set to obtain multiple second ISP parameter sets.
  • the second ISP parameter set based on the training image set and the first ISP parameter set before determining the second ISP parameter set based on the training image set and the first ISP parameter set, it is also possible to obtain an image corresponding to the current environment, process the image corresponding to the current environment based on the first ISP parameter set, and perform scene recognition based on the processed image to obtain a scene recognition result, which indicates whether there is a key scene in the current environment. If the scene recognition result indicates that there is a key scene in the current environment, the image corresponding to the current environment is stored in the training image set.
  • the smart terminal has a camera that can capture the environment around the smart terminal to obtain an image corresponding to the current environment. In this way, the smart terminal can obtain an image corresponding to the current environment.
  • the recognition can be performed through a neural network model, that is, the processed image is input into the neural network model to obtain the scene name output by the neural network model, that is, the scene recognition result.
  • the neural network model Before using the neural network model for recognition, the neural network model needs to be trained. That is, multiple sample images and the scene name corresponding to each image are obtained, the image is used as the input of the neural network model, and the scene name is used as the output of the neural network model to train the neural network model.
  • the smart terminal stores multiple key scenes, so that the smart terminal can determine whether the scene indicated by the scene recognition result exists in the multiple key scenes. If the scene indicated by the scene recognition result exists in the multiple key scenes, the scene indicated by the scene recognition result is determined to be a key scene, and the image corresponding to the current environment is stored in the training image set. If the scene indicated by the scene recognition result does not exist in the multiple key scenes, the scene indicated by the scene recognition result is determined not to be a key scene, and the image corresponding to the current environment is not stored in the training image set.
  • the training image set includes N sub-training sets, and each sub-training set corresponds to a scene. If the scene indicated by the scene recognition result is a key scene, the image corresponding to the current environment is stored in the sub-training set of the corresponding scene.
  • the images in the training image set can be images taken by the camera in real time. Since different users have different behavioral habits, the images taken by the camera are also different. Therefore, adjusting the ISP parameter set based on the images taken by the camera in real time can make the final adjusted ISP parameters adaptive.
  • the images in the training image set can also be images obtained by other means, and the embodiments of the present application are not limited to this.
  • the intelligent terminal can also determine in real time the number of images in each sub-training set in the training image set. If the number of images in each sub-training set in the training image set reaches the image number threshold, the above step 301 can be executed. Of course, if there is a sub-training set in the training image set that reaches the image number threshold, the intelligent terminal can also process the sub-training set first to obtain the sub-parameter set corresponding to the sub-training set in the second ISP parameter set until all sub-training sets in the training image set are processed. In other words, the intelligent terminal can process each sub-training set only after all sub-training sets in the training image set reach the image number threshold. It is also possible to directly process a sub-training set when the number of images in a sub-training set reaches the image number threshold without waiting for the number of images in other sub-training sets to reach the image number threshold.
  • the image quantity threshold is set in advance, and the image quantity threshold corresponding to each sub-training set can be the same or different, and can be adjusted according to different requirements in different situations.
  • the embodiment of the present application does not limit this.
  • the images in the training image set are images that have not been processed by the ISP parameter set.
  • the adjusted ISP parameter set can be applied to the images that have not been processed by the ISP parameter set, so as to determine the quality of the adjusted ISP parameter set.
  • Step 302 Process the test image set based on the second ISP parameter set.
  • the number of the second ISP parameter sets may be one or more. If there are multiple second ISP parameter sets, the test image set is processed based on each second ISP parameter set in the multiple second ISP parameter sets, thereby obtaining multiple processed test image sets.
  • a test image set can be obtained through a recharge interface, which can be expressed as: recharge(&test_type, &scene, &camera_type, &FOV, &picture_path).
  • recharge represents the recharge interface
  • &test_type is used to determine whether it is a training image set or a test image set, where 1 represents a training image set, 0 represents a test image set
  • &scene represents a scene, such as entering a tunnel, inside a tunnel, an overpass, backlighting, a highway, a desert, snowy days, rainy days, etc.
  • &camera_type represents the camera type
  • &FOV represents the camera wide-angle type, for example, close range (FOV120), medium range (FOV60), long range (FOV30), fisheye (FOV180), etc.
  • &picture_path represents the recharged image path directory.
  • Step 303 Determine a first quality score and a first recognition rate based on the processed test image set, where the first quality score indicates the image quality of the processed test image set, and the first recognition rate indicates the image recognition status of the processed test image set.
  • the processed test image set may be one or more.
  • the methods for determining the first quality score and the first recognition rate are different, which will be introduced below.
  • the first quality score and the first recognition rate are determined based on the processed test image set.
  • the average of the quality scores of the N subtest sets can be determined as the first quality score.
  • each subtest set corresponds to a weight. The quality scores of the N subtest sets are multiplied by their respective corresponding weights and then added to obtain the first quality score.
  • the process of determining the quality score of each sub-test set in the processed test image set is similar to the process of determining the candidate quality score of the processed target sub-training set mentioned above. Please refer to the corresponding content in the above text and will not be repeated here.
  • the mode of the recognition rates of the N subtest sets may be determined, and the mode may be determined as the first recognition rate.
  • the average value of the recognition rates of the N subtest sets may be determined, and the average value may be determined as the first recognition rate.
  • the first recognition rate may also be determined in other ways, and the embodiments of the present application do not limit this.
  • the quality scores and recognition rates of the multiple processed test image sets are determined, the maximum quality score among the quality scores of the multiple processed test image sets is used as the first quality score, and the recognition rate corresponding to the maximum quality score is used as the first recognition rate.
  • Step 304 If the first quality score is greater than the first reference quality score and the first recognition rate is greater than the first reference recognition rate, the ISP parameter set corresponding to the smart terminal is adjusted from the first ISP parameter set to the second ISP parameter set, the first reference quality score indicates the image quality of the test image set after being processed by the first ISP parameter set, and the first reference recognition rate indicates the image recognition status of the test image set after being processed by the first ISP parameter set.
  • the process of determining the first reference quality score and the first reference recognition rate is similar to the process of determining the first quality score and the first recognition rate in the first case of step 303. Please refer to the corresponding content above and will not be repeated here.
  • the ISP parameter set corresponding to the smart terminal is not adjusted from the first ISP parameter set to the second ISP parameter set, that is, the ISP parameter set corresponding to the smart terminal remains unchanged.
  • the first quality score is greater than the first reference quality score and the first recognition rate is greater than the first reference recognition rate, it means that the image quality after being processed by the second ISP parameter is better than the first ISP parameter, and the image recognition rate after being processed by the second ISP parameter is also higher than the first ISP parameter, so the ISP parameter set corresponding to the smart terminal can be adjusted from the first ISP parameter set to the second ISP parameter set.
  • the first quality score is not greater than the first reference quality score and ⁇ or the first recognition rate is not greater than the first reference recognition rate, it means that the image quality after being processed by the second ISP parameter is not as good as the first ISP parameter, and the image recognition rate after being processed by the second ISP parameter is also lower than the first ISP parameter, so the ISP parameter set corresponding to the smart terminal is not adjusted from the first ISP parameter set to the second ISP parameter set.
  • one second ISP parameter set may be determined through the above step 301, or multiple second ISP parameter sets may be determined.
  • the ISP parameter set corresponding to the smart terminal may be directly adjusted from the first ISP parameter set to the second ISP parameter set.
  • the ISP parameter set corresponding to the smart terminal may be adjusted from the first ISP parameter set to the second ISP parameter set corresponding to the first quality score.
  • the first reference recognition rate may be updated to the first recognition rate.
  • the first reference recognition rate is updated to the first recognition rate and the first reference quality score is updated to the first quality score. If the first quality score is not greater than the first reference quality score and/or the first recognition rate is not greater than the first reference recognition rate, the first reference recognition rate and the first reference quality score are kept unchanged. This ensures that the ISP parameter set adjusted subsequently is better than the previous ISP parameter set, thereby further improving the efficiency of ISP parameter adjustment and the recognition rate of the image.
  • the smart terminal can trigger an update prompt, which is used to prompt the user to update the ISP parameter set corresponding to the smart terminal. If the smart terminal receives an update instruction triggered by the user, it means that the user agrees to update the ISP parameter set corresponding to the smart terminal. At this time, the smart terminal adjusts the corresponding ISP parameter set from the first ISP parameter set to the second ISP parameter set.
  • the smart terminal can directly adjust the corresponding ISP parameter set from the first ISP parameter set to the second ISP parameter set, or ask the user whether to update the ISP parameter set. Only when the user agrees to update, the smart terminal will adjust the corresponding ISP parameter set from the first ISP parameter set to the second ISP parameter set.
  • the above content is to determine whether the first quality score is greater than the first reference quality score and whether the first recognition rate is greater than the first reference recognition rate after the first quality score and the first recognition rate are determined.
  • the first quality score can also be determined first, and when the first quality score is greater than the first reference quality score, the first recognition rate can be determined to determine whether the first recognition rate is greater than the first reference recognition rate.
  • the embodiments of the present application do not limit this.
  • the camera captures the environment around the smart vehicle to obtain an image corresponding to the current environment, and then the computing platform processes the image corresponding to the current environment based on the first ISP parameter set, and recognizes the processed image. Based on the scene recognition result obtained by recognition, the image corresponding to the current environment is stored in a training image set.
  • a second ISP parameter set is determined based on the training image set, and then the images in the test image set are processed based on the second ISP parameter set to determine a first quality score and a first recognition rate of the test image set.
  • an ISP parameter update prompt is triggered to remind the user to update the ISP parameters.
  • the first ISP parameter set is adjusted to the second ISP parameter set.
  • the embodiment of the present application automatically determines the second ISP parameter based on the training image set and the first ISP parameter set, and does not require manual repeated adjustment of the ISP parameter, thereby reducing the labor cost.
  • the first quality score represents the image quality of the test image set after being processed by the second ISP parameter set
  • the first recognition rate represents the image recognition of the test image set after being processed by the second ISP parameter set
  • the first reference quality score represents the image quality of the test image set after being processed by the first ISP parameter set
  • the first reference recognition rate represents the image recognition of the test image set after being processed by the first ISP parameter set.
  • the first quality score is greater than the first reference quality score, and the first recognition rate is greater than the first reference recognition rate, it means that the image quality after being processed by the second ISP parameter set is better than that of the first ISP parameter set, and the image recognition rate after being processed by the second ISP parameter set is also higher than that of the first ISP parameter set.
  • adjusting the ISP parameter set corresponding to the smart terminal from the first ISP parameter set to the second ISP parameter set can not only ensure that the quality of the image obtained by the smart terminal according to the second ISP parameter set is better than the image quality obtained according to the first ISP parameter set, but also ensure that the recognition rate of the image obtained by the smart terminal according to the second ISP parameter set is also better than the recognition rate of the image obtained according to the first ISP parameter set. That is to say, the method provided in the present application can not only autonomously adjust the ISP parameters without investing a lot of manpower costs, improve the efficiency of ISP parameter adjustment, but also improve the recognition rate of images on the basis of ensuring human eye vision.
  • the method provided in the embodiment of the present application can also update the first reference recognition rate, or the first reference recognition rate and the first reference quality score, so that the ISP parameter set adjusted subsequently can be better than the previous ISP parameter set, thereby further improving the efficiency of ISP parameter adjustment and the recognition rate of images.
  • the method provided in the embodiment of the present application can also capture images in real time, and then adjust the ISP parameter set based on the real-time captured image, that is, different users capture different images, and then the ISP parameter set adjusted based on the captured image is different, therefore, the final adjusted ISP parameters are adaptive, and can adjust better ISP parameters for different users for the user.
  • FIG5 is a schematic diagram of the structure of a parameter adjustment device provided in an embodiment of the present application, and the parameter adjustment device can be implemented as part or all of the intelligent terminal by software, hardware or a combination of both.
  • the device includes: a first determination module 501, a first processing module 502, a second determination module 503 and an adjustment module 504.
  • the first determination module 501 is used to determine the second ISP parameter set based on the training image set and the first image signal processing ISP parameter set, where the first ISP parameter set is the ISP parameter set corresponding to the smart terminal.
  • the detailed implementation process refers to the corresponding content in the above embodiments, which will not be repeated here.
  • the first processing module 502 is used to process the test image set based on the second ISP parameter set.
  • the detailed implementation process refers to the corresponding content in the above embodiments, which will not be repeated here.
  • the second determination module 503 is used to determine a first quality score and a first recognition rate based on the processed test image set, wherein the first quality score indicates the image quality of the processed test image set, and the first recognition rate indicates the image recognition of the processed test image set.
  • the detailed implementation process refers to the corresponding content in the above embodiments, which will not be repeated here.
  • the adjustment module 504 is used to adjust the ISP parameter set corresponding to the intelligent terminal from the first ISP parameter set to the second ISP parameter set if the first quality score is greater than the first reference quality score and the first recognition rate is greater than the first reference recognition rate, wherein the first reference quality score indicates the image quality of the test image set after being processed by the first ISP parameter set, and the first reference recognition rate indicates the image recognition of the test image set after being processed by the first ISP parameter set.
  • the detailed implementation process refers to the corresponding contents in the above-mentioned embodiments, which will not be repeated here.
  • the training image set includes N sub-training sets
  • the first ISP parameter set includes M parameters
  • each of the N sub-training sets corresponds to a scene
  • N and M are both integers greater than 1;
  • the first determining module 501 is specifically used for:
  • the target sub-training set Take one of the sub-training sets as the target sub-training set, adjust the first sub-parameter set corresponding to the target scene in the first ISP parameter set, so as to obtain the second sub-parameter set corresponding to the target scene in the second ISP parameter set, where the target scene is the scene corresponding to the target sub-training set, and the first sub-parameter set includes at least one parameter among the M parameters.
  • the first determining module 501 is specifically configured to:
  • the parameters corresponding to the target scene in the candidate ISP parameter set corresponding to the maximum candidate quality score among the multiple candidate quality scores are determined as the second sub-parameter set.
  • the test image set includes N sub-test sets, and the N sub-test sets correspond one-to-one to the N sub-training sets;
  • the first determining module 501 is specifically used for:
  • the step of adjusting the first sub-parameter set corresponding to the target scene in the first ISP parameter set is executed, and the second reference quality score indicates the image quality of the target sub-test set after being processed by the first ISP parameter set, and the target sub-test set is the sub-test set corresponding to the target scene.
  • the device further comprises:
  • An updating module is used to update the first reference recognition rate to the first recognition rate if the first quality score is greater than the first reference quality score and the first recognition rate is greater than the first reference recognition rate, or to update the first reference recognition rate to the first recognition rate and the first reference quality score to the first quality score.
  • the device further comprises:
  • An acquisition module is used to obtain an image corresponding to the current environment
  • a second processing module configured to process an image corresponding to the current environment based on the first ISP parameter set
  • a recognition module configured to perform scene recognition based on the processed image to obtain a scene recognition result, wherein the scene recognition result indicates whether a key scene exists in the current environment;
  • the storage module is used to store the image corresponding to the current environment into the training image set if the scene recognition result indicates that there is a key scene in the current environment.
  • the embodiment of the present application automatically determines the second ISP parameter based on the training image set and the first ISP parameter set, without manually adjusting the ISP parameter repeatedly, thereby reducing the labor cost.
  • the first quality score represents the image quality of the test image set after being processed by the second ISP parameter set
  • the first recognition rate represents the image recognition of the test image set after being processed by the second ISP parameter set
  • the first reference quality score represents the image quality of the test image set after being processed by the first ISP parameter set
  • the first reference recognition rate represents the image recognition of the test image set after being processed by the first ISP parameter set.
  • the first quality score is greater than the first reference quality score, and the first recognition rate is greater than the first reference recognition rate, it means that the image quality after being processed by the second ISP parameter set is better than that of the first ISP parameter set, and the image recognition rate after being processed by the second ISP parameter set is also higher than that of the first ISP parameter set.
  • adjusting the ISP parameter set corresponding to the smart terminal from the first ISP parameter set to the second ISP parameter set can not only ensure that the quality of the image obtained by the smart terminal according to the second ISP parameter set is better than the image quality obtained according to the first ISP parameter set, but also ensure that the recognition rate of the image obtained by the smart terminal according to the second ISP parameter set is also better than the recognition rate of the image obtained according to the first ISP parameter set. That is to say, the method provided in the present application can not only autonomously adjust the ISP parameters without investing a lot of manpower costs, improve the efficiency of ISP parameter adjustment, but also improve the recognition rate of images on the basis of ensuring human eye vision.
  • the method provided in the embodiment of the present application can also update the first reference recognition rate, or the first reference recognition rate and the first reference quality score, so that the ISP parameter set adjusted subsequently can be better than the previous ISP parameter set, thereby further improving the efficiency of ISP parameter adjustment and the recognition rate of images.
  • the method provided in the embodiment of the present application can also capture images in real time, and then adjust the ISP parameter set based on the real-time captured image, that is, different users capture different images, and then the ISP parameter set adjusted based on the captured image is different, therefore, the final adjusted ISP parameters are adaptive, and can adjust better ISP parameters for different users.
  • the parameter adjustment device provided in the above embodiment only uses the division of the above functional modules as an example when performing parameter adjustment.
  • the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.
  • the parameter adjustment device provided in the above embodiment and the parameter adjustment method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.
  • the computer program product includes one or more computer instructions.
  • the computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device.
  • the computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.
  • the computer instructions can be transmitted from a website site, computer, server or data center by wired (for example: coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (for example: infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center.
  • the computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that includes one or more available media integrated.
  • the available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a digital versatile disc (DVD)), or a semiconductor medium (e.g., a solid state disk (SSD)).
  • the computer-readable storage medium mentioned in the embodiment of the present application may be a non-volatile storage medium, in other words, a non-transient storage medium.
  • the information including but not limited to user device information, user personal information, etc.
  • data including but not limited to data for analysis, stored data, displayed data, etc.
  • signals involved in the embodiments of the present application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions.
  • the training image set, test image set and first ISP parameter set involved in the embodiments of the present application are all obtained with full authorization.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

La présente demande se rapporte au domaine technique du traitement d'image, et divulgue un procédé et un appareil de réglage de paramètre, et un terminal intelligent. Le procédé consiste à : sur la base d'un ensemble d'images d'apprentissage et d'un premier ensemble de paramètres de traitement de signal d'image (ISP), déterminer un second ensemble de paramètres ISP ; traiter un ensemble d'images de test sur la base du second ensemble de paramètres ISP ; déterminer un premier score de qualité et un premier taux de reconnaissance sur la base de l'ensemble d'images de test traité ; et si le premier score de qualité est supérieur à un premier score de qualité de référence et le premier taux de reconnaissance est supérieur à un premier taux de reconnaissance de référence, régler l'ensemble de paramètres ISP correspondant au terminal intelligent pour passer du premier ensemble de paramètres ISP au second ensemble de paramètres ISP. Selon le procédé fourni par la présente demande, non seulement des paramètres ISP peuvent être réglés de manière autonome sans investir beaucoup de coût humain, ce qui permet d'améliorer l'efficacité de réglage de paramètre ISP, mais également le taux de reconnaissance d'images peut être amélioré tout en garantissant la vision de l'œil humain.
PCT/CN2022/126256 2022-10-19 2022-10-19 Procédé et appareil de réglage de paramètre, et terminal intelligent WO2024082183A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/126256 WO2024082183A1 (fr) 2022-10-19 2022-10-19 Procédé et appareil de réglage de paramètre, et terminal intelligent

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/126256 WO2024082183A1 (fr) 2022-10-19 2022-10-19 Procédé et appareil de réglage de paramètre, et terminal intelligent

Publications (1)

Publication Number Publication Date
WO2024082183A1 true WO2024082183A1 (fr) 2024-04-25

Family

ID=90736517

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/126256 WO2024082183A1 (fr) 2022-10-19 2022-10-19 Procédé et appareil de réglage de paramètre, et terminal intelligent

Country Status (1)

Country Link
WO (1) WO2024082183A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160284095A1 (en) * 2015-03-27 2016-09-29 Edmond Chalom Machine learning of real-time image capture parameters
WO2019152499A1 (fr) * 2018-01-30 2019-08-08 Qualcomm Incorporated Systèmes et procédé pour réglage de processeur de signal d'image utilisant une image de référence
WO2019152534A1 (fr) * 2018-01-30 2019-08-08 Qualcomm Incorporated Systèmes et procédés de réglage de processeur de signal d'image
WO2019233341A1 (fr) * 2018-06-08 2019-12-12 Oppo广东移动通信有限公司 Procédé et appareil de traitement d'images, support d'informations lisible par ordinateur, et dispositif électronique
CN113705597A (zh) * 2021-03-05 2021-11-26 腾讯科技（北京）有限公司 一种图像处理方法、装置、计算机设备以及可读存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160284095A1 (en) * 2015-03-27 2016-09-29 Edmond Chalom Machine learning of real-time image capture parameters
WO2019152499A1 (fr) * 2018-01-30 2019-08-08 Qualcomm Incorporated Systèmes et procédé pour réglage de processeur de signal d'image utilisant une image de référence
WO2019152534A1 (fr) * 2018-01-30 2019-08-08 Qualcomm Incorporated Systèmes et procédés de réglage de processeur de signal d'image
WO2019233341A1 (fr) * 2018-06-08 2019-12-12 Oppo广东移动通信有限公司 Procédé et appareil de traitement d'images, support d'informations lisible par ordinateur, et dispositif électronique
CN113705597A (zh) * 2021-03-05 2021-11-26 腾讯科技（北京）有限公司 一种图像处理方法、装置、计算机设备以及可读存储介质

Similar Documents

Publication Publication Date Title
US12033373B2 (en) Dynamic image recognition model updates
CN111114554B (zh) 行驶轨迹预测方法、装置、终端及存储介质
CN109635825A (zh) 车辆属性检测方法、装置及存储介质
CN113246991B (zh) 用于无人车的远程驾驶端的数据传输方法和装置
CN111127358B (zh) 图像处理方法、装置及存储介质
WO2023138537A1 (fr) Procédé et appareil de traitement d'images, dispositif terminal et support de stockage
CN112840636A (zh) 一种图像处理方法和装置
CN109353345A (zh) 车辆控制方法、装置、设备、介质和车辆
CN110070082B (zh) 车牌识别方法、装置、设备及存储介质
CN113705427B (zh) 基于车规级芯片SoC的疲劳驾驶监测预警方法及系统
CN111695615A (zh) 基于人工智能的车辆定损方法、装置、电子设备及介质
WO2024082183A1 (fr) Procédé et appareil de réglage de paramètre, et terminal intelligent
CN114187515A (zh) 图像分割方法和图像分割装置
CN117454929A (zh) 神经网络阈值确定与动态调整的方法、装置、车辆及介质
CN111160206A (zh) 一种交通环境元素视觉感知方法及装置
WO2023155032A1 (fr) Procédé de traitement d'image et appareil de traitement d'image
CN115984723A (zh) 道路破损检测方法、系统、装置、存储介质及计算机设备
CN113807209B (zh) 车位检测方法、装置、电子设备及存储介质
CN118115530A (zh) 车辆轨迹生成方法、装置、电子设备及存储介质
CN112435475B (zh) 一种交通状态检测方法、装置、设备及存储介质
CN113421191A (zh) 图像处理方法、装置、设备及存储介质
CN110647898B (zh) 图像处理方法、装置、电子设备及计算机存储介质
EP4390836A1 (fr) Amélioration d'image ciblée lors de l'adressage de dégradations provoquées par des conditions environnementales
CN118254807B (zh) 驾驶行为优化方法、装置、车辆及存储介质
WO2023149174A1 (fr) Dispositif de traitement d'informations, procédé de traitement d'informations et programme

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22962373

Country of ref document: EP

Kind code of ref document: A1