WO2023045626A1 - 图像采集方法、装置、终端、计算机可读存储介质及计算机程序产品 - Google Patents

图像采集方法、装置、终端、计算机可读存储介质及计算机程序产品 Download PDF

Info

Publication number
WO2023045626A1
WO2023045626A1 PCT/CN2022/112739 CN2022112739W WO2023045626A1 WO 2023045626 A1 WO2023045626 A1 WO 2023045626A1 CN 2022112739 W CN2022112739 W CN 2022112739W WO 2023045626 A1 WO2023045626 A1 WO 2023045626A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
image
acquisition
image data
parameters
Prior art date
Application number
PCT/CN2022/112739
Other languages
English (en)
French (fr)
Inventor
朱文波
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Publication of WO2023045626A1 publication Critical patent/WO2023045626A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/67—Focus control based on electronic image sensor signals

Definitions

  • the present application relates to the field of computer technology, and in particular to an image acquisition method, device, terminal, computer readable storage medium and computer program product.
  • electronic devices recognize gestures and faces of users through image acquisition and image recognition.
  • the image acquisition will be performed with the set acquisition parameters, and then the identification processing module will identify and process the acquired image data.
  • the images collected with the set collection parameters may fail to meet the requirements of recognition processing, thereby affecting the accuracy of image recognition.
  • Embodiments of the present application provide an image acquisition method, device, terminal, computer-readable storage medium, and computer program product.
  • the application scene is determined, and image acquisition is performed based on the target acquisition parameters that match the application scene.
  • performing image recognition based on the target image data can improve the accuracy of the image recognition result.
  • an embodiment of the present application provides an image acquisition method, the method comprising: acquiring original image data of a target object through an image acquisition module; determining the corresponding image content of the original image data by detecting the image content
  • the application scene represents the scene where the target object is identified and processed; the target acquisition parameters matching the application scene are determined; the image acquisition module is controlled to detect the target object based on the target acquisition parameters Image acquisition is performed to obtain target image data.
  • the embodiment of the present application provides an image acquisition device, the device includes: a first acquisition part configured to acquire the original image data of a target object through an image acquisition module; a first determination part configured to detect the The image content in the original image data determines the application scene corresponding to the original image data, and the application scene represents the scene where the target object is recognized and processed; the second determining part is configured to determine the target matching the application scene Collection parameters; the second collection part is configured to control the image collection module to collect images of the target object based on the target collection parameters to obtain target image data.
  • an embodiment of the present application provides a terminal, the terminal includes a memory and a processor, the memory stores a computer program that can run on the processor; when the processor executes the computer program, the The image acquisition method described in the first aspect above.
  • the embodiment of the present application provides a computer-readable storage medium, on which executable instructions are stored, configured to implement the image acquisition method described in the first aspect when executed by a processor.
  • an embodiment of the present application provides a computer program product, including computer program instructions, the computer program instructions causing a computer to execute the steps of the image acquisition method described in the first aspect above.
  • Embodiments of the present application provide an image acquisition method, device, terminal, computer-readable storage medium, and computer program product.
  • the original image data of the target object is first collected by the image acquisition module.
  • the application scene corresponding to the original image data is determined by detecting the image content in the original image data.
  • the application scene represents The scene of identifying and processing the target object; next, determine the target acquisition parameters that match the application scene; control the image acquisition module to perform image acquisition on the target object based on the target acquisition parameters, and obtain the target image data.
  • the acquisition parameters of the image acquisition module are adaptively adjusted during the image acquisition process, and the adjusted target acquisition parameters match the application scene, and the image acquisition module performs image acquisition based on the target acquisition parameters that match the application scene, and then , performing image recognition based on target image data can improve the accuracy of image recognition results.
  • Fig. 1 is a flow chart of the steps of an image acquisition method provided by an embodiment of the present application
  • Fig. 2 is a flow chart of the steps of another image acquisition method provided by the embodiment of the present application.
  • FIG. 3 is a schematic diagram of depth information of a target object provided by an embodiment of the present application.
  • FIG. 4 is a flow chart of the steps of another image acquisition method provided in the embodiment of the present application.
  • FIG. 5 is a schematic structural diagram of a front end of an image processing system provided by an embodiment of the present application.
  • FIG. 6 is a flow chart of the steps of another image acquisition method provided in the embodiment of the present application.
  • FIG. 7 is a schematic structural diagram of an image acquisition device provided in an embodiment of the present application.
  • FIG. 8 is a schematic structural diagram of a terminal provided in an embodiment of the present application.
  • FIG. 9 is a schematic diagram of a chip structure provided by an embodiment of the present application.
  • the frame rate and resolution of the image acquisition module are the default low frame rate and low resolution.
  • the user actively adjusts the frame rate and resolution of the image acquisition module on the application program (Application, App) to suit different application scenarios.
  • the image acquisition is performed at a certain resolution, that is to say, in any application scenario, the frame rate and resolution are fixed during the image acquisition process.
  • the setting methods of frame rate and resolution in related technologies can meet the requirements, but for application scenarios such as gesture detection and eyeball recognition, the recognition efficiency and accuracy of shooting functions are all related to frame rate and resolution. Changes in resolution have a larger relationship.
  • the gesture detection scene in the gesture detection scene, it is necessary to recognize the gesture action, and the subsequent recognition processing module performs recognition processing on the image data collected by the image acquisition device.
  • the accuracy of the image and the embodiment of details will directly affect the accuracy of the recognition result. If the image output is directly performed with high resolution or high frame rate, it will affect the speed and efficiency of recognition processing, and also affect the accuracy of recognition results. Therefore, it is urgent to provide an image acquisition method to find a balance between the accuracy of recognition processing results and system power consumption, and improve the accuracy of recognition processing results on the basis of low power consumption.
  • gesture recognition in related technologies, gesture recognition can be based on the recognition technology of two-dimensional color image.
  • Computer graphics algorithms perform identification of content in images.
  • Two-dimensional gesture recognition technology can only recognize a few static gestures, and these actions must be set in advance.
  • Three-dimensional gesture recognition technology adds a Z-axis information, which can recognize various hand shapes, gestures and actions.
  • Three-dimensional gesture recognition includes gesture recognition with certain depth information, which requires special hardware to realize. This can usually be done with sensors and optical cameras.
  • gesture recognition includes tracking gestures and subsequent computer data processing.
  • gesture motion capture it can be realized by optical and sensor.
  • the algorithm of gesture recognition inference includes template matching technology (used in two-dimensional gesture recognition technology), through statistical sample features and deep learning neural network technology.
  • template matching technology used in two-dimensional gesture recognition technology
  • deep learning neural network technology For data processing, the amount of data will directly affect the speed and efficiency of image data processing, as well as the accuracy of results.
  • an eye pattern recognition module may be mounted on the terminal device, and the sclera recognition technology may be used.
  • the terminal device can perform accurate identification as long as it uses a camera with more than 2 million pixels. And even if the eyeballs are congested due to allergies, fatigue and other reasons, it will not affect the arrangement of blood vessels on the sclera. It is precisely because of the relative stability and non-reproducibility of the eye pattern that the safety of eye pattern recognition is guaranteed.
  • "eye pattern" is 4.5 times that of fingerprints, so its security is much higher than that of fingerprints.
  • the sclera recognition technology cannot be used to identify the eyeball.
  • Fig. 1 is the step flowchart of a kind of image collection method provided by the embodiment of the application, the image collection method comprises the following steps:
  • Step S101 collect the original image data of the target object through the image collection module.
  • the image acquisition module may include but not limited to mobile phone camera, camera (camera), optical sensor and normally open sensor (Alawys on sensor, AON sensor), wherein the sensor may represent the low power consumption image sensor.
  • the target objects in the embodiment of the present application may include, but are not limited to, animals, human bodies, human faces, human eyes, human lips, human eyeballs, and human hands.
  • the image acquisition module is an AON sensor as an example for illustration, and the AON sensor acquires the target object to obtain original image data.
  • the original image data represents the original image data that the sensor converts the captured light source signal into a digital signal, and the original image data has not been processed by subsequent compensation.
  • Step S102 by detecting the image content in the original image data, determine the application scene corresponding to the original image data.
  • the application scene represents a scene where the target object is recognized and processed.
  • the image content in the original image data can be detected by the image content detection module, so as to determine the application scene according to the image content detection result.
  • Recognition processing includes but not limited to human body recognition, human face recognition, human eye recognition, human eyeball recognition, human hand recognition, human gesture detection and preset action detection, wherein human gesture detection means non-contact gesture recognition or gesture isolation Empty identification.
  • the image content can reflect the user's shooting intention, and can also be understood as the shooting function mode used by the user.
  • the application scene is determined according to the image content detection result, which improves the accuracy of the application scene.
  • the scene mode preset by the user may also be obtained, and the application scene is determined according to the preset scene mode. That is to say, before the image acquisition module starts to capture the target object, the user can pre-select the shooting function mode on the app, such as face recognition function, gesture detection function and eyeball recognition function. That is, the scene mode is preset before shooting, and the application scene is determined according to the preset scene mode.
  • the method of directly obtaining the application scene through the scene mode preset by the user improves the efficiency of obtaining the application scene.
  • Step S103 determining target acquisition parameters that match the application scenario.
  • the target acquisition parameters are used for image acquisition of the target object.
  • the application scene corresponds to the acquisition parameters.
  • the resolution in the acquisition parameters is set to 1080P, that is, each horizontal row has 1280 pixels, and each column has 1080 pixels, the total number of pixels is 1280 ⁇ 1080, and this product is the resolution.
  • the frame rate in the acquisition parameters is set to 50FPS.
  • the number of frames per second indicates the number of frames per second of the screen, which can be understood as the number of frames of animation or video.
  • FPS is a measure of the amount of information used to save and display dynamic videos. The more frames per second, the smoother the displayed motion will be.
  • the default acquisition parameters of the image acquisition module are set to low frame rate and low resolution.
  • target acquisition parameters matching the application scenario are also determined.
  • the acquisition parameter of the image acquisition module is adjusted to the target acquisition parameter, so that the subsequent image acquisition module performs image acquisition on the target object based on the target acquisition parameter, thereby improving the accuracy of the acquisition result.
  • Step S104 controlling the image acquisition module to acquire images of the target object based on the target acquisition parameters to obtain target image data.
  • the control image acquisition module performs image acquisition on the target object based on the target acquisition parameters, and the acquisition result is closely related to the application scene.
  • the collection result is used for subsequent recognition processing of the application scene, and the collection result may be target image data, and then image recognition is performed based on the target image data, which improves the accuracy of the image recognition result.
  • the acquisition parameters of the image acquisition module are adaptively adjusted during the image acquisition process, and the adjusted target acquisition parameters match the application scene.
  • the image acquisition module performs image acquisition based on target acquisition parameters that match the application scene, and then performs image recognition based on target image data, which can improve the accuracy of image recognition results.
  • step S102 may be implemented through steps S1021 and S1022.
  • FIG. 2 is a flowchart of steps of another image acquisition method provided in the embodiment of the present application.
  • Step S1021 by detecting image content in the original image data, to obtain image feature points and/or motion parameters of image feature points.
  • the image feature points are used to represent the feature information of the feature points in the image content; the motion parameters of the image feature points are used to represent the motion information of the feature points in the image content within a preset time.
  • the image feature points may include but not limited to a preset location area, a representative location area, a preset location point and a representative location point of the target object.
  • image feature points may represent but are not limited to feature information of face contour, nose tip and ears.
  • image feature points can represent but not limited to feature information of fingers, wrists, and fingertips
  • motion parameters of image feature points can represent but not limited to motion information of fingers, wrists, and fingertips within a preset time.
  • Step S1022 according to the image feature point and/or the motion parameter of the image feature point, match with the preset application scene, and obtain the application scene corresponding to the original image data.
  • the preset application scenario may be appropriately set by those skilled in the art according to actual needs.
  • face recognition scenarios, eyeball recognition scenarios, gesture detection scenarios, and specific action detection scenarios as long as the preset application scenarios can be matched with image feature points and/or motion parameters of image feature points, determine the application scenario corresponding to the original image data That's it.
  • the preset application scenario includes preset image feature points and/or preset motion parameters of image feature points, which may be determined by analyzing image feature points and/or motion parameters of image feature points corresponding to a large amount of experimental data used.
  • the application scene corresponding to the maximum similarity of the image feature points may be determined as the application scene corresponding to the original image data by calculating the similarity between the image feature points and the preset image feature points. It is also possible to determine the application scene corresponding to the maximum value of the motion parameter similarity of the image feature point as the application scene corresponding to the original image data by calculating the similarity between the motion parameter of the image feature point and the preset motion parameter of the image feature point . It is also possible to calculate the first similarity between the image feature point and the preset image feature point, and the second similarity between the motion parameter of the image feature point and the preset image feature point motion parameter, comprehensively considering the first The first similarity and the second similarity obtain the comprehensive similarity by adding weights.
  • the application scenario corresponding to the maximum value of the comprehensive similarity is determined as the application scenario corresponding to the original image data, which is not limited in this embodiment of the present application.
  • image feature points and/or motion parameters of image feature points are obtained by detecting image content in original image data. Then, according to the image feature point and/or the motion parameter of the image feature point, it is matched with the preset application scene, thereby obtaining the application scene, and improving the accuracy of the application scene.
  • the target acquisition parameters in step S103 include at least one of the following: frame rate, resolution, focal length and image bit width information.
  • the frame rate may represent the number of pictures played in a video per second, and the higher the frame rate, the smoother the video playback.
  • the resolution represents the pixels that the display can display. The larger the resolution, the finer the image; the focal length represents the focal length of the lens, which refers to the distance from the main point of the lens optics to the focal point.
  • Image bit width information indicates the number of bits of image data that can be transmitted in one clock cycle. The larger the number of bits, the greater the amount of image data that can be transmitted instantly. It can be understood as the amount of image data that can be transmitted by memory or video memory at one time.
  • the target acquisition parameters in the embodiment of the present application include at least one of frame rate, resolution, focal length, and image bit width information.
  • one of the target acquisition parameters can be adjusted. Items, or two or more of the target acquisition parameters can be adjusted at the same time, which improves the diversity of the adjustment methods of the target acquisition parameters.
  • step S103 may be implemented through the following two examples.
  • Example 1 According to the first mapping relationship, the acquisition parameters matching the application scenario are determined to obtain the target acquisition parameters, and the first mapping relationship represents the corresponding relationship between the application scenario and the acquisition parameters.
  • the application scene there is a corresponding relationship between the application scene and the collection parameters.
  • the resolution in the collection parameters is 720P; in the eyeball recognition scene, the resolution in the collection parameters is 1080P; in the gesture detection scene , the frame rate in the acquisition parameters is 50FPS.
  • the acquisition parameters of the image acquisition module are adaptively adjusted according to the acquisition parameters of the application scene. For example, appropriately increase the resolution and appropriately reduce the frame rate, so as to adjust the acquisition parameters of the image acquisition module to a state that matches the application scene.
  • the resolution in the collection parameters is 1080P
  • the resolution of the collection parameters of the image collection module is increased to 720P.
  • the acquisition parameters of the application scene are used as the target acquisition parameters of the image acquisition module.
  • the resolution in the acquisition parameters is 720P
  • the resolution of the target acquisition parameters of the image acquisition module is set to 720P.
  • the first mapping relationship in the first example above includes the following two situations.
  • the first situation if the application scene is an application scene for recognizing the action of the target object, the target acquisition parameters include a first frame rate, and the first frame rate is higher than the second frame rate used for acquiring the original image data.
  • the second situation if the application scenario is an application scenario for identifying the target object or identifying local details of the target object, the target acquisition parameters include the first resolution, and the first resolution is higher than the second resolution used to acquire the original image data Rate.
  • the image acquisition module collects the target object based on the initial acquisition parameters to obtain the original image data.
  • the initial acquisition parameters including frame rate and resolution as an example, in order to reduce system power consumption and improve system processing capability, the default initial acquisition parameters are set to low Frame rate and low resolution.
  • the initial acquisition parameters can be adaptively adjusted according to different application scenarios. That is to say, the second frame rate in the initial acquisition parameters is a low frame rate, and the second resolution is a low resolution; One frame rate, the first frame rate is greater than the second frame rate, and the resolution may or may not be adjusted at this time, as long as the frame rate is increased, which is not limited in this embodiment of the present application.
  • the second resolution is adjusted to the first resolution, and the first resolution is greater than the second resolution.
  • the frame rate can be adjusted. It is also possible not to adjust the frame rate, but to increase the resolution, which is not limited in this embodiment of the present application.
  • the application scenario of recognizing the action of the target object in the embodiment of the present application can be understood as a detection scene that requires posture transformation, such as a gesture detection scene and an action detection scene. It is guaranteed that there are more valid image data frames per unit time for analysis and recognition by the recognition processing module.
  • the application scenario of identifying the target object or identifying the local details of the target object in the embodiment of the present application can be understood as a scenario that depends on the accuracy of image information. For example, in the face recognition scene and eyeball recognition scene, for the application scene of recognizing the target object or recognizing the local details of the target object, the priority is to increase the resolution, and the accuracy of recognition can be improved through more detailed information.
  • first and second in the embodiment of the present application are to distinguish different objects, rather than to describe a specific order, for example, the first mapping relationship, the second mapping relationship, the first frame rate, Second frame rate, first resolution, second resolution.
  • Example 2 According to the second mapping relationship, the adjustment range of the acquisition parameter matching the application scenario is determined to obtain the adjustment range of the target acquisition parameter.
  • the second mapping relationship represents the corresponding relationship between the application scenario and the adjustment range of the acquisition parameter.
  • the target acquisition parameters are determined within the adjustment range of the target acquisition parameters.
  • the adjustment range of the resolution in the collection parameter is 720P-1080P. That is to say, when the application scene is a face recognition scene or an eyeball recognition scene, if the resolution of the target acquisition parameters is set between 720P-1080P, more detailed information can be obtained, thereby improving the accuracy of recognition.
  • the adjustment range of the frame rate in the acquisition parameters is 40FPS-60FPS.
  • the application scene is a gesture detection scene and an action detection scene
  • the frame rate in the target acquisition parameters is set between 40FPS-60FPS, more valid image data frames can be obtained per unit time for recognition processing
  • the module analyzes and recognizes actions.
  • the depth information of the target object represents the distance between the target object and the camera.
  • FIG. 3 is a schematic diagram of depth information of a target object provided by an embodiment of the present application.
  • the depth information of the target object can be expressed as the focal distance between the focal plane and the position of the imaging plane.
  • the embodiment of the present application first determines the adjustment range of the target acquisition parameters, and then determines the target acquisition parameters in the adjustment range of the target acquisition parameters according to the depth information of the target object in the original image data, not only considering the application scene, but also considering the target object. Depth information improves the matching degree of target acquisition parameters and application scenarios.
  • step S104 may be implemented in the following manner.
  • the adjustment time point is determined according to the expected number of image frames required for the recognition processing of the target object in the application scenario, or the time required for the completion of the recognition processing process of the target object in the application scenario.
  • the control image acquisition module performs image acquisition on the target object based on the target acquisition parameters to obtain target image data.
  • the selection of the adjustment time point in the embodiment of the present application can be determined according to various factors, including but not limited to: the expected number of image frames required for the recognition process of the target object, and the time required to complete the recognition process of the target object in the application scene The length of time required.
  • the number of image frames required for accurate identification of details reversely determine how many time periods need to acquire the corresponding number of image frames, and then determine the corresponding time point before the time period as the adjustment time point.
  • the change speed of the action posture obtained when the acquisition parameters are adjusted reversely determine how many time periods need to detect the corresponding working posture, that is to say, the frame rate needs to be adjusted before the time period, and then before the time period
  • the corresponding time point is determined as the adjustment time point.
  • the gesture detection scene generally completes the change of the whole gesture detection scene within 0.5 seconds, that is, 500 milliseconds, which also requires the recognition processing module to complete the recognition within 500 milliseconds.
  • the frame rate of the image acquisition module is 30FPS, that is, 30 frames per second, the recognition needs to be completed within 15 frames. Therefore, for the scene of gesture detection, it is necessary to increase the frame rate of the image acquisition module when the image feature points representing the gesture appear in 3 frames. For example, if the frame rate is increased to 50FPS, then the image acquisition module collects 15 Frame time, the time used is 150 milliseconds, so that the recognition processing module can meet the requirement of completing recognition within 500 milliseconds.
  • the frame rate of the image acquisition module is low at the beginning, it takes a long time for 3 frames.
  • the number of image frames corresponding to the image feature points is set to 3 frames.
  • Increase the frame rate of the image acquisition module from 30FPS to 50FPS.
  • the acquisition parameters of the image acquisition module are adjusted through the above steps S101-S104. After the target acquisition parameters of the image acquisition module are determined, the embodiment of the present application also performs a second adjustment on the acquisition parameters according to the target image data.
  • the image acquisition method provided in the embodiment of the present application further includes step S105-step S109.
  • FIG. 4 is a flowchart of steps of another image acquisition method provided in the embodiment of the present application.
  • Step S105 detecting local information of image content in the target image data, and obtaining local image feature points and/or motion parameters of local image feature points.
  • the local information represents information about local parts or local details of the target object.
  • the target object is a human face, and the local parts of the target object include but are not limited to eyebrows, eyeballs, pupils, and sclera.
  • the target object is a human hand, and local parts of the target object include but not limited to fingertips and finger joints.
  • Step S106 according to the local image feature points and/or the motion parameters of the local image feature points, determine the target application scene corresponding to the target image data.
  • the target application scene represents a scene in which local details of the target object are recognized and processed, and may be understood as a scene in which local parts of the target object are accurately recognized.
  • step S105 and step S106 The implementation manner of determining the target application scenario through step S105 and step S106 is consistent with the implementation manner of determining the application scenario in step S1021 and step S1022 above, and will not be repeated here.
  • Step S107 if the target acquisition parameters do not match the acquisition parameters corresponding to the target application scenario, then determine the intermediate acquisition parameters that match the target application scenario.
  • step S105 and step S106 the image feature points and/or the motion parameters of the image feature points are analyzed for the target image data obtained after adjusting the acquisition parameters for the first time, and the target is further determined on the basis of the determined application scene Application scenarios. If the target acquisition parameters do not match the acquisition parameters corresponding to the target application scenario, then the acquisition parameters are adjusted for the second time, and the second adjustment method is the same as the first adjustment method, which will not be repeated here.
  • the acquisition parameters of the image acquisition module are adjusted for the first time to adapt to the face recognition scene, and the acquisition parameters of the image acquisition module are adjusted for the second time to adapt to the eyeball recognition scene.
  • Step S108 controlling the image acquisition module to acquire images of the target object based on the intermediate acquisition parameters to obtain intermediate image data.
  • Step S108 is implemented in the same manner as step S104, and will not be repeated here.
  • the embodiment of the present application may continue to perform the analysis in the above-mentioned steps S105 and S106 on the intermediate image data, so as to further determine a new application scenario. If the acquisition parameters of the image acquisition module do not match the new application scene, the acquisition parameters are adjusted again so that the acquisition parameters match the new application scene.
  • Step S109 if the target acquisition parameters match the acquisition parameters corresponding to the target application scenario, execute step S104 .
  • the image acquisition module can be controlled to acquire images of the target object based on the target acquisition parameters.
  • local image feature points and/or motion parameters of local image feature points are obtained by detecting local information of image content in target image data. Then, according to the local image feature points and/or the motion parameters of the local image feature points, the target application scene corresponding to the target image data is determined. If the target acquisition parameters do not match the acquisition parameters corresponding to the target application scene, then determine the intermediate acquisition parameters that match the target application scene, thereby controlling the image acquisition module to perform image acquisition on the target object based on the intermediate acquisition parameters to obtain intermediate image data.
  • the acquisition parameters increase or decrease along the step curve, so that the image recognition effect and the system power consumption Find a balance and realize the improvement of the accuracy of image recognition on the basis of low power consumption.
  • the image acquisition method provided in the embodiment of the present application further includes the following steps.
  • the working parameters of the recognition processing module are determined according to the target acquisition parameters.
  • the identification processing module performs identification processing on the target object in the target image data according to the working parameters.
  • the image acquisition module performs image acquisition on the target object based on the target acquisition parameters to obtain target image data.
  • the working parameters of the recognition processing module are determined according to the target acquisition parameters, and the recognition processing module performs recognition processing on the target object in the target image data according to the working parameters.
  • the image is output, and the user can see the displayed image. That is, the image acquisition method in the embodiment of the present application is completed before the image is displayed.
  • the default initial acquisition parameters are set to low frame rate and low resolution.
  • the initial working parameters of the identification processing module correspond to the initial acquisition parameters of the image acquisition module, and the initial operating parameters of the identification processing module match the low frame rate and low resolution. Therefore, after adjusting the acquisition parameters of the image acquisition module, it is necessary to determine the working parameters of the recognition processing module according to the target acquisition parameters, and synchronize the adjusted frame rate and resolution to the recognition processing module to ensure that the settings of the entire link are synchronized, so that the adjustment The subsequent data flow can be transferred and processed normally.
  • the image acquisition module is an AON sensor, and the acquisition parameters include frame rate and resolution as an example.
  • the AON sensor will not display the image to the user, but will send the acquired original image data to the recognition processing module for processing. Recognition processing. Therefore, the adjustment of the frame rate and resolution of the AON sensor depends on the frame rate and resolution set when the AON sensor is started.
  • image acquisition is performed at a fixed frame rate and resolution.
  • the image acquisition module collects images at low frame rate or low resolution, and the recognition processing module performs recognition processing on low frame rate or low resolution images, which increases the overall power consumption of the system and reduces the accuracy of image recognition processing.
  • the image acquisition module collects images at a frame rate or resolution that matches the application scene, and the recognition processing module performs recognition processing on the image that matches the application scene, and improves the image quality under the premise of low power consumption. Accuracy of recognition processing.
  • the embodiment of the present application can not only adjust the acquisition parameters of the image acquisition module, but then the identification processing module can identify and process the target object in the target image data according to the working parameters corresponding to the target acquisition parameters, thereby Improve the efficiency of image recognition processing.
  • the same technical effect can also be achieved by adjusting the processing process of the target image data.
  • adjusting the interval of reading the data frame for example, reading every 3 frames, every 2 frames, and frame by frame can reduce the pressure of image data processing , reduce system power consumption, and improve the efficiency of image recognition processing.
  • FIG. 5 is a schematic structural diagram of a front end of an image processing system provided by an embodiment of the present application.
  • the following description is made by taking the image acquisition module as an AON sensor, and the acquisition parameters include frame rate and resolution as an example.
  • the front end of the image processing system in Figure 5 includes an AON sensor, an image content detection module, and a frame rate and resolution control module.
  • the AON sensor collects the original image data of the target object according to the default low frame rate and low resolution.
  • the RAW image Represents the original image data
  • the RAW image is the original image data that the image sensor converts the captured light source signal into a digital signal.
  • the image content detection module detects the obtained RAW image, which is a rough detection, and the main purpose is to preliminarily judge whether there is an application scene that needs to be recognized.
  • the detection process can be realized by detecting the image feature points and/or the motion parameters of the image feature points.
  • Application scenarios. The image content detection module detects and analyzes the image content of the RAW image, which can be processed by a neural network processor (Neural-network Processing Units, NPU). Information is processed quickly.
  • a neural network processor Neuro-network Processing Units, NPU.
  • the image content detection module sends the image content detection result to the frame rate and resolution control module, and the image content detection result is used to determine the application scenario.
  • the frame rate and resolution control module determines the application scenario according to the image content detection result Take this as an example.
  • the frame rate and resolution control module determines the appropriate frame rate and resolution based on the preset frame rate and resolution adjustment strategy according to the image content detection result. Adjust the frame rate and resolution of the AON sensor to an appropriate frame rate and resolution, and synchronize the entire data link based on the adjusted frame rate and resolution, for example, to synchronize the working parameters of the recognition processing module.
  • the image content detection module can determine the application scene according to the image content detection result, or the frame rate and resolution control module can determine the application scene according to the image content detection result. Do limit.
  • the above frame rate and resolution adjustment strategies can be adjusted differently according to different application scenarios.
  • priority is given to increasing the frame rate to ensure that there are more valid image data frames available for analysis and recognition per unit time.
  • priority should be given to increasing the resolution to ensure that more eyeball detail information can improve the accuracy of recognition.
  • the AON sensor performs image acquisition based on the adjusted frame rate and resolution to obtain the target image data.
  • the frame rate and resolution control module is adjusted according to the preset frame rate and resolution
  • the frame rate and resolution decision can be executed according to the adjustment strategy.
  • the configuration file can also be determined according to the adjustment result (tuning result) of the AON sensor, that is, the application scene has a mapping relationship with the acquisition parameters of the AON sensor,
  • the target acquisition parameters can be determined by querying the configuration file.
  • the adjustment range of the acquisition parameters allowed by the configuration file based on the application scenario, and combine with other related parameters to perform a processing method similar to interpolation, so as to determine the target acquisition parameters. For example, for a scene of a portrait, within the adjustment range of the acquisition parameters allowed by the configuration file, combined with the depth information of the position of the portrait, the target acquisition parameters can be calculated.
  • the depth information can be understood as the distance between the target object and the lens, as shown in the above-mentioned FIG. 3 , which will not be repeated here.
  • the image content detection module at the front end of the system in the embodiment of the present application performs preliminary application scene recognition on the original image data output by the AON sensor, and obtains the judgment result of the application scene. It can be understood that the frame rate and resolution decision module can also determine the application scenario according to the image content detection result. Then the frame rate and resolution decision module adjusts the frame rate and resolution according to the judgment result of the application scene, so that the system can realize the effect of accurate recognition of the application scene under the premise of low power consumption.
  • the embodiment of the present application does not require user intervention, and the front-end image content detection module directly judges the application scene, and then the frame rate and resolution decision module adjusts the frame rate and resolution according to the judgment result of the application scene. Therefore, the embodiment of the present application is invisible to the user and can be completed automatically, which improves the interaction efficiency.
  • the embodiment of the present application provides a method for adjusting the configuration of an AON sensor based on image detection, and the adjusted acquisition parameters may include frame rate and/or resolution, for realizing Improve the accuracy of image recognition processing under the premise of low power consumption.
  • FIG. 6 is a flow chart of the steps of another image acquisition method provided in the embodiment of the present application, including step S601-step S609.
  • adjusting the frame rate and resolution is taken as an example for illustration, and it can be understood that only the frame rate or the resolution may be adjusted when adjusting the acquisition parameters.
  • Step S601 AON Sensor collects original image data, and sends the original image data to the image content detection module for image content detection processing.
  • step S601 the embodiment of the present application turns on the AON sensor, and starts the Pre-ISP (Image Signal Processing) module at the front end of the image processing system, and the Pre-ISP module represents the pre-image signal processor.
  • Pre-ISP Image Signal Processing
  • Step S602 the image content detection module performs image content detection.
  • the image content detection module detects image feature points and/or motion parameters of image feature points according to the original image data, and obtains threshold information for adjusting frame rate and resolution.
  • the image feature points and/or image feature The motion parameters of the points are set to a threshold of 1.
  • relevant adjustment strategies can be automatically activated according to the image content detection results, or the user can set the shooting scene mode in advance, for example, whether it is a gesture recognition function mode or an eyeball recognition function mode, and determine the adjustment strategy according to the scene mode.
  • Step S603 based on the obtained threshold information, adjust in combination with frame rate and resolution adjustment strategies.
  • Step S603 can be a multi-level adjustment, which can be realized through steps S6031-step S6034.
  • Step S6031 judging whether the image content detected by the image content detection module reaches the threshold 1.
  • Step S6032 if the image content detected by the image content detection module reaches the threshold 1, adjust the frame rate and resolution for the first time according to the adjustment strategy.
  • AON sensor uses the initially adjusted frame rate and resolution for image acquisition to obtain target image data, and the image content detection module detects the adjusted image content according to the target image data.
  • Step S6033 judging whether the adjusted image content detected by the image content detection module reaches threshold 2.
  • Step S6034 if the adjusted image content detected by the image content detection module reaches the threshold 2, adjust the frame rate and resolution for the second time according to the adjustment strategy.
  • step S6031-step S6034 Through the multi-level adjustment of step S6031-step S6034, the recognition process of the application scene with higher accuracy is continued.
  • the image feature points and/or the motion parameters of the image feature points in the original image data are identified and analyzed by the image content recognition module, when the change of the image feature points and/or the motion parameters of the image feature points reaches a threshold 1.
  • Increase the frame rate and resolution by one level for example, increase the frame rate from 15FPS to 30FPS, and then perform more accurate and timely recognition and judgment on the scene that needs to be recognized, so that the system can perform subsequent control operations accordingly, such as screen Operations such as lighting up or waking up the camera are not limited in this embodiment of the present application.
  • the image content (for example, face recognition, gesture detection) is recognized by the image content detection module, and when certain conditions are met (for example, a face is recognized, different gestures are recognized, a Gesture changes between frames), adjust the frame rate and resolution, output the target image data with an appropriate frame rate and resolution, and then perform image recognition based on the target image data, improving the accuracy of the image recognition results.
  • certain conditions for example, a face is recognized, different gestures are recognized, a Gesture changes between frames
  • Step S604 using the adjusted frame rate and resolution for image acquisition, and synchronizing the working parameters corresponding to the adjusted frame rate and resolution to the corresponding recognition processing module, so as to ensure that the settings of the entire link are processed synchronously.
  • Step S605 continue to preview or shoot images based on the adjusted frame rate and resolution configuration.
  • AON sensor collects target image data based on high resolution and high frame rate
  • the recognition processing module performs recognition processing based on the image data (that is, target image data) after increasing the resolution and frame rate to obtain more accurate recognition results for subsequent systems Perform corresponding operations, such as screen wake-up, camera opening, and other functions.
  • the selection of the threshold 1 and the threshold 2 above can be selected according to various factors. For example, according to the number of image frames required for accurate identification of the target object in the application scenario, and the number of image frames obtained when the acquisition parameters are adjusted The change speed of the action posture is reversely determined when the frame rate and resolution need to be adjusted, and then the corresponding threshold 1 and threshold 2 are determined, and the threshold is used as the judgment condition for judging the adjustment time point.
  • the acquisition parameters (frame rate and resolution) of the AON sensor are adjusted based on the image feature points and/or the motion parameters of the image feature points.
  • the working parameters of the recognition processing module are synchronized to ensure that the data flow of the system can be normally circulated and processed after adjustment.
  • different frame rate and resolution adjustment strategies are adopted. For example, for portrait recognition scenarios (face recognition scenarios, eyeball recognition scenarios), low frame rate and high resolution can be set.
  • gesture detection scenarios set a high frame rate to match rapid changes in gestures, and set a low resolution to reduce system power consumption.
  • adaptive adjustments may be performed based on self-feedback parameters of the image acquisition module and the image content detection module, for example, adjustments to frame rate and resolution.
  • the embodiment of the present application provides an image detection-based AON sensor configuration adjustment scheme.
  • the adjustment parameters include frame rate and resolution, which can improve the accuracy of scene recognition under the premise of low power consumption.
  • Use AON sensor to support different low resolution (such as VGA, QVGA), and different low frame rate (2/5/10/20fps) image output characteristics.
  • face recognition gesture recognition
  • the recognition processing module face recognition, gesture recognition
  • the image acquisition method provided by the embodiment of the present application it can be judged whether the image acquisition method provided by the embodiment of the present application is adopted in the following manner. Since the service will provide the corresponding dump interface, similar to the related operations of the camera service, the output frame rate and output resolution of the current corresponding AON sensor can be monitored in real time. By making changes in related scenes, for example, making some gesture recognition scenes; if there is an automatic change in frame rate or resolution when the scene that needs to be recognized occurs, it can be inferred that the product uses the image provided by the embodiment of the application collection method.
  • the accuracy of image recognition processing can also be achieved through the following examples.
  • Example 1 For the adjustment of the frame rate and resolution, not only the acquisition parameters of the AON sensor can be adjusted, but also the processing strategy for the image data stream can be adjusted.
  • the data frame of the target image data is read in the recognition processing module , the interval of reading data frames can be adjusted (3 frames at intervals, 2 frames at intervals, and frame-by-frame reading), thereby reducing the pressure on image data processing and reducing system power consumption.
  • Example 2 The adjustment of the acquisition parameters is not limited to the frame rate and resolution. It is also possible to adjust the image data volume by adjusting the image bit width information and the depth information of the target image, so as to reduce the pressure of algorithm processing and save System power consumption. Therefore, for the adjustment of related configurations, there are more choices and corresponding adjustment strategies, so that the overall recognition processing effect and power consumption of the system can be balanced, thereby reducing system power consumption and improving the accuracy of image recognition processing.
  • the embodiment of the present application also provides an image acquisition device, as shown in FIG. 7 , which is a schematic structural diagram of an image acquisition device provided in the embodiment of the present application. It includes: a first collection part 701 configured to collect the original image data of the target object through the image collection module; a first determination part 702 configured to determine the application scene corresponding to the original image data by detecting the image content in the original image data, the application scene Characterize the scene where the target object is identified and processed; the second determining part 703 is configured to determine target acquisition parameters that match the application scene; the second acquisition part 704 is configured to control the image acquisition module to perform image processing on the target object based on the target acquisition parameters Acquisition to obtain target image data.
  • a first collection part 701 configured to collect the original image data of the target object through the image collection module
  • a first determination part 702 configured to determine the application scene corresponding to the original image data by detecting the image content in the original image data, the application scene Characterize the scene where the target object is identified and processed
  • the first determining part 702 is further configured to obtain image feature points and/or motion parameters of image feature points by detecting image content in the original image data;
  • the motion parameter is matched with the preset application scene to obtain the application scene corresponding to the original image data.
  • the target acquisition parameters include at least one of the following: frame rate, resolution, focal length, and image bit width information.
  • the second determination part 703 is further configured to determine the acquisition parameters matching the application scenario according to the first mapping relationship to obtain the target acquisition parameters, and the first mapping relationship represents the correspondence between the application scenario and the acquisition parameters relation.
  • the first mapping relationship includes: if the application scenario is an application scenario for recognizing the action of the target object, the target acquisition parameters include the first frame rate, which is higher than the second frame rate used to acquire the original image data ; If the application scenario is an application scenario for identifying the target object or identifying local details of the target object, the target acquisition parameters include a first resolution, and the first resolution is higher than the second resolution used to acquire the original image data.
  • the second determination part 703 is further configured to determine the adjustment range of the acquisition parameters that match the application scenario according to the second mapping relationship, and obtain the adjustment range of the target acquisition parameters.
  • the second mapping relationship represents the application scenario and The correspondence between the adjustment ranges of the acquisition parameters; according to the depth information of the target object in the original image data, the target acquisition parameters are determined within the adjustment range of the target acquisition parameters.
  • the second acquisition part 704 is also configured to be based on the expected number of image frames required for the recognition process of the target object in the application scene, or the number of image frames required for the completion of the target object recognition process in the application scene
  • the time length determines the adjustment time point; at the adjustment time point, the control image acquisition module performs image acquisition on the target object based on the target acquisition parameters to obtain the target image data.
  • the first determination part 702 is further configured to detect the local information of the image content in the target image data, and obtain the local image feature points and/or the motion parameters of the local image feature points; according to the local image feature points and/or Or the motion parameters of local image feature points, determine the target application scene corresponding to the target image data, and the target application scene represents the scene for identifying and processing the local details of the target object; the second acquisition part 704 is also configured to If the acquisition parameters corresponding to the application scene do not match, determine the intermediate acquisition parameters that match the target application scene; the control image acquisition module performs image acquisition on the target object based on the intermediate acquisition parameters to obtain intermediate image data.
  • the image acquisition device 70 further includes an identification part 705, and the identification part 705 is configured to determine the operating parameters of the identification processing module according to the target acquisition parameters; the identification processing module determines the target object in the target image data according to the operating parameters Perform recognition processing.
  • the image acquisition device provided in the above embodiment performs image processing, it only uses the division of the above-mentioned program modules as an example for illustration. In practical applications, the above-mentioned processing allocation can be completed by different program modules as required That is, the internal structure of the device is divided into different program modules to complete all or part of the processing described above.
  • the image acquisition device and the image acquisition method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process and beneficial effects thereof are detailed in the method embodiments, and will not be repeated here. For the technical details not disclosed in the device embodiment of this application, please refer to the description of the method embodiment of this application for understanding.
  • FIG. 8 is a schematic diagram of the composition and structure of the terminal proposed in the embodiment of the present application.
  • the terminal 80 proposed in the embodiment of the present application includes a processor 801 and a memory 802. A computer program running on the processor 801.
  • the terminal 80 may further include a communication interface 803, and a bus 804 for connecting the processor 801, the memory 802, and the communication interface 803.
  • the above-mentioned processor 801 may be an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a digital signal processor (Digital Signal Processor, DSP), a digital signal processing device (Digital Signal Processing Device, DSPD ), Programmable Logic Device (ProgRAMmable Logic Device, PLD), Field Programmable Gate Array (Field ProgRAMmable Gate Array, FPGA), Central Processing Unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor at least one of .
  • ASIC Application Specific Integrated Circuit
  • DSP Digital Signal Processor
  • DSPD Digital Signal Processing Device
  • PLD Programmable Logic Device
  • Field Programmable Gate Array Field ProgRAMmable Gate Array
  • FPGA Field ProgRAMmable Gate Array
  • CPU Central Processing Unit
  • controller microcontroller, microprocessor at least one of .
  • the terminal 80 may also include a memory 802, which may be connected to the processor 801, wherein the memory 802 is configured to store executable program codes, the program codes include computer operation instructions, and the memory 802 may include a high-speed RAM memory, and may also include Non-volatile memory, eg, at least two disk memories.
  • the bus 804 is used to connect the communication interface 803 , the processor 801 and the memory 802 and communicate with each other among these devices.
  • the memory 802 is configured to store instructions and data.
  • the processor 801 when the processor 801 runs the computer program stored in the memory 802, it can execute the following instructions: collect the original image data of the target object through the image acquisition module; determine the image content by detecting the image content in the original image data.
  • the application scene corresponding to the original image data, the application scene represents the scene for identifying and processing the target object; determine the target acquisition parameters matching the application scene; control the image acquisition module to collect images of the target object based on the target acquisition parameters, and obtain the target image data .
  • the above-mentioned memory 802 may be a volatile memory (volatile memory), such as a random access memory (Random-Access Memory, RAM); or a non-volatile memory (non-volatile memory), such as a read-only memory (Read-Only Memory, ROM), flash memory (flash memory), hard disk (Hard Disk Drive, HDD) or solid-state drive (Solid-State Drive, SSD); Provide instructions and data.
  • volatile memory such as a random access memory (Random-Access Memory, RAM)
  • non-volatile memory such as a read-only memory (Read-Only Memory, ROM), flash memory (flash memory), hard disk (Hard Disk Drive, HDD) or solid-state drive (Solid-State Drive, SSD); Provide instructions and data.
  • each functional module in this embodiment may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units can be implemented in the form of hardware or in the form of software function modules.
  • the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium.
  • the technical solution of this embodiment is essentially or The part contributed by the prior art or the whole or part of the technical solution can be embodied in the form of software products, the computer software products are stored in a storage medium, and include several instructions to make a computer device (which can be a personal A computer, a server, or a network device, etc.) or a processor (processor) executes all or part of the steps of the method of this embodiment.
  • the aforementioned storage medium includes: various media that can store program codes such as U disk, mobile hard disk, read-only memory (Read Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk.
  • An embodiment of the present application provides a computer-readable storage medium on which a program is stored, and when the program is executed by a processor, the image acquisition method described in any one of the above embodiments is implemented.
  • the program instructions corresponding to an image acquisition method in this embodiment can be stored on a storage medium such as an optical disk, a hard disk, or a USB flash drive.
  • a storage medium such as an optical disk, a hard disk, or a USB flash drive.
  • the embodiments of the present application may be provided as methods, systems, or computer program products. Accordingly, the present application may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) having computer-usable program code embodied therein.
  • a computer-usable storage media including but not limited to disk storage and optical storage, etc.
  • FIG. 9 is a schematic structural diagram of a chip according to an embodiment of the present application.
  • the chip 90 shown in FIG. 9 includes a processor 901, and the processor 901 can call and run a computer program from the memory, so as to realize the image acquisition method in the embodiment of the present application.
  • the chip 90 may further include a memory 902 .
  • the processor 901 can invoke and run a computer program from the memory 902, so as to implement the image acquisition method in the embodiment of the present application.
  • the memory 902 may be an independent device independent of the processor 901 , or may be integrated in the processor 901 .
  • the chip 90 may also include an input interface 903 .
  • the processor 901 can control the input interface 903 to communicate with other devices or chips, and obtain information or data sent by other devices or chips.
  • the chip 90 may also include an output interface 904 .
  • the processor 901 can control the output interface 904 to communicate with other devices or chips, and can output information or data to other devices or chips.
  • the chip can be applied to the terminal in the embodiments of the present application, and the chip can implement the corresponding processes implemented by the terminal in the various methods of the embodiments of the present application. For the sake of brevity, details are not repeated here.
  • the chip mentioned in the embodiment of the present application may also be called a system-on-chip, a system-on-chip, a system-on-a-chip, or a system-on-a-chip.
  • the embodiment of the present application also provides a computer program product, including computer program instructions.
  • the computer program product can be applied to the terminal in the embodiments of the present application, and the computer program instructions cause the computer to execute the corresponding processes implemented by the terminal in the various methods of the embodiments of the present application.
  • the computer program instructions cause the computer to execute the corresponding processes implemented by the terminal in the various methods of the embodiments of the present application.
  • details are not repeated here. .
  • the embodiment of the present application also provides a computer program.
  • the computer program can be applied to the terminal in the embodiment of the present application.
  • the computer program When the computer program is run on the computer, the computer executes the corresponding process implemented by the terminal in each method of the embodiment of the present application. For the sake of brevity, the This will not be repeated here.
  • These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions
  • the device realizes the function specified in implementing one or more procedures of the flowchart and/or one or more blocks of the block diagram.
  • the embodiment of the application discloses an image acquisition method, device, terminal, computer readable storage medium and computer program product.
  • the original image data of the target object is collected by the image acquisition module.
  • the application scene corresponding to the original image data is determined by detecting the image content in the original image data; parameters; the control image acquisition module performs image acquisition on the target object based on the target acquisition parameters to obtain the target image data.
  • the acquisition parameters of the image acquisition module are adaptively adjusted during the image acquisition process, and the adjusted target acquisition parameters match the application scene, and the image acquisition module performs image acquisition based on the target acquisition parameters that match the application scene, and then , performing image recognition based on target image data can improve the accuracy of image recognition results.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Signal Processing (AREA)
  • Ophthalmology & Optometry (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Image Analysis (AREA)

Abstract

本申请实施例公开了一种图像采集方法、装置、终端、计算机可读存储介质及计算机程序产品。该方法通过图像采集模块采集目标对象的原始图像数据，在采集到原始图像数据之后，通过检测原始图像数据中图像内容，确定原始图像数据对应的应用场景；接下来确定与应用场景匹配的目标采集参数；控制图像采集模块基于目标采集参数，对目标对象进行图像采集，得到目标图像数据。本申请实施例在图像采集过程中对图像采集模块的采集参数进行适应性调整，调整得到的目标采集参数与应用场景相匹配，图像采集模块基于与应用场景匹配的目标采集参数进行图像采集，进而，基于目标图像数据进行图像识别，能够提高图像识别结果的准确性。

Description

图像采集方法、装置、终端、计算机可读存储介质及计算机程序产品
相关申请的交叉引用
本申请基于申请号为202111106280.0、申请日为2021年9月22日、申请名称为“图像采集方法、装置、设备和计算机可读存储介质”的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此引入本申请作为参考。
技术领域
本申请涉及计算机技术领域，尤其涉及一种图像采集方法、装置、终端、计算机可读存储介质及计算机程序产品。
背景技术
随着智能终端技术的不断发展，电子设备(如智能手机、平板电脑等)的使用越来越普及，例如，手势检测、人脸识别等功能已经广泛应用于生活中，用户不用接触电子设备即可完成人机交互，提高了交互效率。
实际应用中，电子设备通过图像采集，图像识别来识别用户的手势以及人脸。通常情况下，图像采集模块的帧率和分辨率在设置成功后，会以设置好的采集参数进行图像采集，然后识别处理模块对采集到的图像数据进行识别处理。然而，以设置好的采集参数采集到的图像，可能无法达到识别处理的需求，进而影响图像识别的准确性。
发明内容
本申请实施例提供一种图像采集方法、装置、终端、计算机可读存储介质及计算机程序产品，通过检测原始图像数据中图像内容，确定应用场景，基于与应用场景匹配的目标采集参数进行图像采集，进而，基于目标图像数据进行图像识别，能够提高图像识别结果的准确性。
本申请实施例的技术方案是这样实现的：
第一方面，本申请实施例提供一种图像采集方法，所述方法包括：通过图像采集模块采集目标对象的原始图像数据；通过检测所述原始图像数据中图像内容，确定所述原始图像数据对应的应用场景，所述应用场景表征对所述目标对象进行识别处理的场景；确定与所述应用场景匹配的目标采集参数；控制所述图像采集模块基于所述目标采集参数，对所述目标对象进行图像采集，得到目标图像数据。
第二方面，本申请实施例提供一种图像采集装置，所述装置包括：第一采集部分，配置为通过图像采集模块采集目标对象的原始图像数据；第一确定部分，配置为通过检测所述原始图像数据中图像内容，确定所述原始图像数据对应的应用场景，所述应用场景表征对所述目标对象进行识别处理的场景；第二确定部分，配置为确定与所述应用场景匹配的目标采集参数；第二采集部分，配置为控制所述图像采集模块基于所述目标采集参数，对所述目标对象进行图像采集，得到目标图像数据。
第三方面，本申请实施例提供一种终端，所述终端包括存储器和处理器，所述存储器存储有可在所述处理器上运行的计算机程序；所述处理器执行所述计算机程序时实现上述第一方面所述的图像采集方法。
第四方面，本申请实施例提供一种计算机可读存储介质，其上存储有可执行指令，配置为被处理器执行时，实现第一方面所述的上述图像采集方法。
第五方面，本申请实施例提供一种计算机程序产品，包括计算机程序指令，所述计算机程序指令使得计算机执行上述第一方面所述的图像采集方法的步骤。
本申请实施例提供了一种图像采集方法、装置、终端、计算机可读存储介质及计算机程序产品。在本申请实施例中，先通过图像采集模块采集目标对象的原始图像数据，在采集到原始图像数据之后，通过检测原始图像数据中图像内容，确定原始图像数据对应的应用场景，该应用场景表征对目标对象进行识别处理的场景；接下来确定与应用场景匹配的目标采集参数；控制图像采集模块基于目标采集参数，对目标对象进行图像采集，得到目标图像数据。本申请实施例在图像采集过程中对图像采集模块的采集参数进行适应性调整，调整得到的目标采集参数与应用场景相匹配，图像采集模块基于与应用场景匹配的目标采集参数进行图像采集，进而，基于目标图像数据进行图像识别，能够提高图像识别结果的准确性。
附图说明
图1为本申请实施例提供的一种图像采集方法的步骤流程图；
图2为本申请实施例提供的另一种图像采集方法的步骤流程图；
图3为本申请实施例提供的一种目标对象的深度信息的示意图；
图4为本申请实施例提供的又一种图像采集方法的步骤流程图；
图5为本申请实施例提供的一种图像处理系统前端的结构示意图；
图6为本申请实施例提供的再一种图像采集方法的步骤流程图；
图7为本申请实施例提供的一种图像采集装置的结构示意图；
图8为本申请实施例提供的一种终端的结构示意图；
图9为本申请实施例提供的一种芯片结构组成示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述。应当理解的是，此处所描述的一些实施例仅仅用以解释本申请的技术方案，并不用于限定本申请的技术范围。
需要说明的是，本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别不同对象，而不是用于描述特定顺序。此外，术语“包括”和“具有”以及它们任何变形，意图在于覆盖不排他的包含。例如包含了一系列步骤或单元的过程、方法、系统、产品或设备没有限定于已列出的步骤或单元，而是可选地还包括没有列出的步骤或单元，或可选地还包括对于这些过程、方法、产品或设备固有的其他步骤或单元。
为了更好地理解本申请实施例中提供的图像采集方法，在对本申请实施例的技术方案进行介绍之前，先对相关技术进行说明。
相关技术中，由于直接以高分辨率或高帧率进行图像输出，会影响系统功耗，导致系统识别处理能力变慢等问题，因此图像采集模块的帧率和分辨率是默认的低帧率和低 分辨率。在拍摄过程中，显示图像后，用户通过在应用程序(Application，App)上主动对图像采集模块的帧率和分辨率进行调整，以适用于不同的应用场景，后续会以固定的帧率和分辨率进行图像采集，也就是说，在任意应用场景下，在图像采集过程中帧率和分辨率是固定不变的。对于一般的应用场景来说，相关技术中帧率和分辨率的设置方式是可以满足需求的，但是对于手势检测和眼球识别等应用场景，拍摄功能的识别效率和准确性，均与帧率和分辨率的变化有较大的关系。
示例性的，以手势检测场景为例，手势检测场景中需要对手势动作进行识别，以及后续的识别处理模块对图像采集装置采集到图像数据进行识别处理。以眼球识别场景为例，图像的准确性和细节的体现会直接影响识别结果的准确性。若直接以高分辨率或高帧率进行图像输出，会影响识别处理的速度和效率，也会影响到识别结果的准确性。因此，亟需提供一种图像采集方法，以在识别处理结果的准确性和系统功耗之间找到平衡，在低功耗的基础上，提高识别处理结果的准确性。
示例性的，对于手势识别来说，相关技术中，手势识别可以基于二维彩色图像的识别技术，二维彩色图像是指通过普通摄像头拍出场景后，得到二维的静态图像，然后再通过计算机图形算法进行图像中内容的识别。二维手势识别技术只能识别出几个静态的手势动作，而且这些动作必须要提前进行设置好。
相较于二维手势识别，三维手势识别技术增加了一个Z轴的信息，可以识别各种手型、手势和动作。三维手势识别包括一定深度信息的手势识别，需要特别的硬件来实现。通常可以通过传感器和光学摄像头完成。
手势识别中最关键的部分包括对手势动作的跟踪以及后续的计算机数据处理。关于手势动作捕捉可以通过光学和传感器两种方式来实现。手势识别推测的算法，包括模板匹配技术(二维手势识别技术使用的)、通过统计样本特征以及深度学习神经网络技术。对于数据处理来说，数据量的大小会直接影响图像数据处理的速度和效率，以及结果的准确性。
示例性的，对于眼球追踪来说，终端设备上可以搭载眼纹识别模块，采用巩膜识别技术。利用人体巩膜上血管分布的唯一性，终端设备只要使用200万像素以上的摄像头，就可以进行精确识别。而且即使眼球因为过敏、疲劳等原因而充血，也不会影响巩膜上血管排布，正是因为眼纹这种相对稳定及不可复制性，保证了眼纹识别的安全性。并且，从信息量角度来讲，“眼纹”是指纹的4.5倍，所以其安全性也大大高于指纹。但是，在夜间没有光线的时候，就无法采用巩膜识别技术对眼球进行识别。
对于眼球识别来说，图像的准确性和细节的体现会直接影响识别的精度，而直接以高分辨率进行图像输出又会直接影响系统的功耗，造成手机发热系统变慢等问题，因此迫切需要我们在识别效果和功耗方便找到一种平衡的方案，便于在低功耗的基础上提升识别的准确度。
本申请实提供一种图像采集方法，如图1所示，图1为本申请实施例提供的一种图像采集方法的步骤流程图，图像采集方法包括以下步骤：
步骤S101、通过图像采集模块采集目标对象的原始图像数据。
本申请实施例中图像采集模块可以包括但不限于手机摄像头、相机(camera)、光学传感器和常开式传感器(Alawys on sensor，AON sensor)，其中sensor可以表示被设置用于应用场景识别的低功耗图像感应器。
本申请实施例中目标对象可以包括但不限于动物、人体、人体脸部、人体眼睛、人体唇部、人体眼球和人体手部。
示例性的，以图像采集模块是AON sensor为例进行说明，AON sensor对目标对象进行采集，得到原始图像数据。原始图像数据表征传感器将捕捉到的光源信号转化为数 字信号的原始图像数据，该原始图像数据没有经过后面补偿等处理。
步骤S102、通过检测原始图像数据中图像内容，确定原始图像数据对应的应用场景。
其中，应用场景表征对目标对象进行识别处理的场景。
本申请实施例中可以通过图像内容检测模块对原始图像数据中图像内容进行检测，从而根据图像内容检测结果确定应用场景。
识别处理包括但不限于人体识别、人体脸部识别、人体眼睛识别、人体眼球识别、人体手部识别、人体手势检测和预设动作检测，其中人体手势检测表示非接触式的手势识别或手势隔空识别。
图像内容能够反映用户的拍摄意图，也可以理解为用户所使用的拍摄功能模式，根据图像内容检测结果确定应用场景，提高了应用场景的准确性。
在一些实施例中，也可以获取用户预先设置的场景模式，根据预先设置的场景模式确定应用场景。也就是说，图像采集模块在开始对目标对象进行采集之前，用户可以在app上预先选择拍摄功能模式，例如人脸识别功能、手势检测功能和眼球识别功能。即拍摄前预先设置场景模式，根据预先设置的场景模式确定应用场景。通过用户预先设置的场景模式直接获取应用场景的方式，提高了获取应用场景的效率。
步骤S103、确定与应用场景匹配的目标采集参数。
目标采集参数用于对目标对象进行图像采集，应用场景与采集参数相对应，例如，人脸识别场景下，采集参数中分辨率设置为1080P，即每一横排有1280个像素，每一列有1080个像素，总的像素就是1280×1080个，这个乘积即为分辨率。手势检测场景下，采集参数中帧率设置为50FPS。需要说明的是，每秒传输帧数(Frames Per Second，FPS)表示画面每秒传输帧数，可以理解为动画或视频的画面数，FPS是测量用于保存、显示动态视频的信息数量，每秒帧数越多，所显示的动作就会越流畅。
示例性的，以采集参数包括帧率和分辨率为例，由于直接以高分辨率或高帧率进行图像输出，会影响系统功耗，导致智能终端发热、系统识别处理能力变慢等问题。因此，图像采集模块的默认采集参数设置为低帧率和低分辨率。但是如果一直低帧率和低分辨率进行图像采集，图像的准确性和细节体现会影响目标对象识别的准确性。因此，本申请实施例在确定应用场景之后，还确定与应用场景匹配的目标采集参数。将图像采集模块的采集参数调整为目标采集参数，使得后续图像采集模块基于目标采集参数，对目标对象进行图像采集，提高了采集结果的准确性。
步骤S104、控制图像采集模块基于目标采集参数，对目标对象进行图像采集，得到目标图像数据。
由于目标采集参数与应用场景相匹配，控制图像采集模块基于目标采集参数，对目标对象进行图像采集，采集结果与应用场景密切相关。采集结果用于应用场景的后续识别处理，采集结果可以是目标图像数据，进而，基于目标图像数据进行图像识别，提高了图像识别结果的准确性。
本申请实施例在图像采集过程中对图像采集模块的采集参数进行适应性调整，调整得到的目标采集参数与应用场景相匹配。图像采集模块基于与应用场景匹配的目标采集参数进行图像采集，进而，基于目标图像数据进行图像识别，能够提高图像识别结果的准确性。
在一些实施例中，步骤S102可以通过步骤S1021和步骤S1022实现。如图2所示，图2为本申请实施例提供的另一种图像采集方法的步骤流程图。
步骤S1021、通过检测原始图像数据中图像内容，获得图像特征点和/或图像特征点的运动参数。
本申请实施例中图像特征点用于表征图像内容中特征点的特征信息；图像特征点的运动参数用于表征在预设时间内图像内容中特征点的运动信息。其中，图像特征点可以包括但不限于目标对象的预设位置区域、代表性位置区域、预设位置点和代表性位置点。
示例性的，人脸识别场景下，图像特征点可以表征但不限于脸部轮廓、鼻尖和耳朵的特征信息。手势检测场景下，图像特征点可以表征但不限于手指、手腕、指尖的特征信息，图像特征点的运动参数可以表征但不限于预设时间内手指、手腕、指尖的运动信息。
步骤S1022、根据图像特征点和/或图像特征点的运动参数，与预设应用场景进行匹配，获得原始图像数据对应的应用场景。
在本申请实施例中，预设应用场景可以由本领域技术人员根据实际需求适当设置。例如，人脸识别场景、眼球识别场景、手势检测场景和特定动作检测场景，只要预设应用场景能够与图像特征点和/或图像特征点的运动参数进行匹配，确定原始图像数据对应的应用场景即可。预设应用场景包括预设的图像特征点和/或预设的图像特征点的运动参数，可以通过对使用的大量试验数据对应的图像特征点和/或图像特征点的运动参数的分析确定。
在进行应用场景匹配时，可以通过计算图像特征点与预设的图像特征点之间的相似度，将图像特征点相似度最大值对应的应用场景确定为原始图像数据对应的应用场景。也可以通过计算图像特征点的运动参数与预设的图像特征点的运动参数之间的相似度，将图像特征点的运动参数相似度最大值对应的应用场景确定为原始图像数据对应的应用场景。也可以通过计算图像特征点与预设的图像特征点之间的第一相似度，以及图像特征点的运动参数与预设的图像特征点的运动参数之间的第二相似度，综合考虑第一相似度和第二相似度，通过添加权重的方式获得综合相似度。将综合相似度最大值对应的应用场景确定为原始图像数据对应的应用场景，本申请实施例对此不作限制。
本申请实施例通过检测原始图像数据中图像内容，获得图像特征点和/或图像特征点的运动参数。然后根据图像特征点和/或图像特征点的运动参数，与预设应用场景进行匹配，从而获得应用场景，提高了应用场景的准确性。
在一些实施例中，步骤S103中的目标采集参数包括以下中的至少一项：帧率、分辨率、焦距和图像位宽信息。
在本申请实施例中，帧率可以表示视频每秒钟播放的图片数目，帧率越高视频播放越流畅。分辨率表征显示器所能显示的像素，分辨率越大，图像画面越细腻；焦距表征镜头焦距，是指镜头光学后主点到焦点的距离。图像位宽信息表示在一个时钟周期内所能传送图像数据的位数，位数越大则瞬间所能传输的图像数据量越大，可以理解为内存或显存一次能传输的图像数据量。
本申请实施例目标采集参数包括帧率、分辨率、焦距和图像位宽信息中的至少一项，在将图像采集模块所采用的采集参数调整为目标采集参数时，可以调整目标采集参数的一项，也可以同时调整目标采集参数中的两项及以上，提高了目标采集参数调整方式的多样性。
在一些实施例中，步骤S103可以通过以下两个示例实现。
示例一，根据第一映射关系，确定与应用场景匹配的采集参数，得到目标采集参数，第一映射关系表征应用场景与采集参数之间的对应关系。
本申请实施例中应用场景与采集参数之间存在对应关系，例如人脸识别场景下，采集参数中的分辨率为720P，眼球识别场景下，采集参数中的分辨率为1080P，手势检测场景下，采集参数中的帧率为50FPS。
在确定原始图像数据对应的应用场景后，一种可实现的方式中，根据应用场景的采 集参数适应性的调整图像采集模块的采集参数。例如，适当增加分辨率、适当减小帧率，从而将图像采集模块的采集参数调整为与应用场景相匹配的状态。示例性的，眼球识别场景下，采集参数中的分辨率为1080P，将图像采集模块的采集参数的分辨率增大到720P。另一种可实现的方式中，将应用场景的采集参数作为图像采集模块的目标采集参数。示例性的，人脸识别场景下，采集参数中的分辨率为720P，将图像采集模块的目标采集参数的分辨率设置为720P。
在一些实施例中，上述示例一中的第一映射关系包括以下两种情形。
第一种情形：若应用场景为用于识别目标对象的动作的应用场景，则目标采集参数包括第一帧率，第一帧率高于采集原始图像数据使用的第二帧率。
第二种情形：若应用场景为用于识别目标对象或识别目标对象的局部细节的应用场景，则目标采集参数包括第一分辨率，第一分辨率高于采集原始图像数据使用的第二分辨率。
图像采集模块基于初始采集参数对目标对象进行采集，得到原始图像数据，以初始采集参数包括帧率和分辨率为例，为了降低系统功耗，提高系统处理能力，默认的初始采集参数设置为低帧率和低分辨率。当根据原始图像数据中图像内容确定应用场景后，可以根据不同的应用场景对初始采集参数进行适应性的调整。也就是说，初始采集参数中第二帧率是低帧率，第二分辨率是低分辨率；若应用场景为用于识别目标对象的动作的应用场景，则将第二帧率调整为第一帧率，第一帧率大于第二帧率，此时可以调整分辨率，也可以不调整分辨率，只要将帧率提高即可，对此本申请实施例不做限制。若应用场景为用于识别目标对象或识别目标对象的局部细节的应用场景，则将第二分辨率调整为第一分辨率，第一分辨率大于第二分辨率，此时可以调整帧率，也可以不调整帧率，只有将分辨率提高即可，对此本申请实施例不做限制。
示例性的，以采集参数包括帧率和分辨率为例，帧率和分辨率的调整策略可以根据不同的场景进行差异化的调整设置。本申请实施例中识别目标对象的动作的应用场景，可以理解为需要进行姿势变换的检测场景，例如手势检测场景和动作检测场景，对于识别目标对象的动作的应用场景，优先调节帧率，以保证在单位时间内有更多的有效图像数据帧，以供识别处理模块分析和识别。本申请实施例中识别目标对象或识别目标对象的局部细节的应用场景，可以理解为依赖于图像信息准确性的场景。例如，人脸识别场景和眼球识别场景，对于识别目标对象或识别目标对象的局部细节的应用场景，优先提高分辨率，通过更多的细节信息提高识别的准确性。
需要说明的是，本申请实施例中“第一”、“第二”是为了区别不同对象，而不是用于描述特定顺序，例如，第一映射关系、第二映射关系，第一帧率、第二帧率，第一分辨率、第二分辨率。
示例二，根据第二映射关系，确定与应用场景匹配的采集参数的调整范围，得到目标采集参数的调整范围，第二映射关系表征应用场景与采集参数的调整范围之间的对应关系。根据原始图像数据中目标对象的深度信息，在目标采集参数的调整范围中确定目标采集参数。
本申请实施例中应用场景与采集参数的调整范围之间存在对应关系，示例性的，识别目标对象或识别目标对象的局部细节的应用场景，采集参数中分辨率的调整范围为720P-1080P。也就是说，当应用场景是人脸识别场景或眼球识别场景时，目标采集参数中分辨率设置在720P-1080P之间，就可以获得更多的细节信息，从而提高识别的准确性。识别目标对象的动作的应用场景，采集参数中帧率的调整范围为40FPS-60FPS。也就是说，当应用场景是手势检测场景和动作检测场景时，目标采集参数中帧率设置在40FPS-60FPS之间，就可以在单位时间内获得更多的有效图像数据帧，以供识别处理模 块对动作进行分析和识别。
目标对象的深度信息表示目标对象距离镜头之间的距离。如图3所示，图3为本申请实施例提供的一种目标对象的深度信息的示意图，图3中目标对象的深度信息可以表示为焦平面与成像面位置之间的对焦距离。
本申请实施例先确定目标采集参数的调整范围，然后根据原始图像数据中目标对象的深度信息，在目标采集参数的调整范围中确定目标采集参数，不仅考虑了应用场景，还考虑了目标对象的深度信息，提高了目标采集参数与应用场景的匹配度。
在一些实施例中，步骤S104可以通过以下方式实现。根据应用场景中对目标对象进行识别处理所需要的期望图像帧数，或者，应用场景中对目标对象的识别处理过程完成所需要的时长，确定调整时间点。在调整时间点，控制图像采集模块基于目标采集参数，对目标对象进行图像采集，得到目标图像数据。
本申请实施例中调整时间点的选取可以根据多种因素进行确定，包括但不限于：对目标对象进行识别处理所需要的期望图像帧数，和应用场景中对目标对象的识别处理过程完成所需要的时长。示例性的，根据准确识别细节所需要的图像帧数，反向确定需要在多少时间段内获取到对应的图像帧数，然后将该时间段之前对应的时间点确定为调整时间点。根据采集参数调整时得到的动作姿势的变化速度，反向确定需要在多少时间段内检测到对应的工作姿势，也就是说在该时间段之前需要对帧率进行调整，然后将该时间段之前对应的时间点确定为调整时间点。
示例性的，手势检测的场景一般在0.5秒，即500毫秒完成整个手势检测场景的变化，这也就要求识别处理模块在500毫秒内完成识别。若图像采集模块的帧率是30FPS，即1秒30帧，则需要在15帧内完成识别。因此，对于手势检测的场景，需要在表征手势的图像特征点出现在3帧的时候，就开始对提高图像采集模块的帧率，例如，将帧率提高到50FPS，那么图像采集模块采集到15帧时，所用时间是150毫秒，使得识别处理模块能够满足500毫秒内完成识别的要求。本示例中考虑到开始时图像采集模块的帧率较低，3帧就已经耗费了较久的时间，将图像特征点对应的图像帧数设置为3帧，在图像采集模块采集到3帧时，将图像采集模块的帧率从30FPS提高到50FPS。
通过上述步骤S101-步骤S104实现对图像采集模块的采集参数的调整，确定图像采集模块的目标采集参数之后，本申请实施例还根据目标图像数据对采集参数进行第二次调整。
在一些实施例中，在上述任一实施例中步骤S104之后，本申请实施例提供的图像采集方法还包括步骤S105-步骤S109。如图4所示，图4为本申请实施例提供的又一种图像采集方法的步骤流程图。
步骤S105、检测目标图像数据中图像内容的局部信息，获得局部图像特征点和/或局部图像特征点的运动参数。
本申请实施例中局部信息表征目标对象的局部部位或局部细节的信息，例如，目标对象是人脸，目标对象的局部部位包括但不限于眉毛、眼球、瞳孔和巩膜。目标对象是人体手部，目标对象的局部部位包括但不限于指尖和手指关节。
步骤S106、根据局部图像特征点和/或局部图像特征点的运动参数，确定目标图像数据对应的目标应用场景。
其中，目标应用场景表征对目标对象的局部细节进行识别处理的场景，可以理解为对目标对象的局部部位进行准确识别的场景。
通过步骤S105和步骤S106确定目标应用场景的实现方式，与上述步骤S1021和步骤S1022确定应用场景的实现方式一致，在此不再赘述。
步骤S107、若目标采集参数与目标应用场景对应的采集参数不匹配，则确定与目 标应用场景匹配的中间采集参数。
通过步骤S105和步骤S106中是对第一次调整采集参数后获得的目标图像数据，进行图像特征点和/或图像特征点的运动参数的分析，在确定的应用场景的基础上，进一步确定目标应用场景。若目标采集参数与目标应用场景对应的采集参数不匹配，则对采集参数进行第二次调整，第二次调整方法与第一次调整方法一致，在此不再赘述。示例性的，通过第一次调整图像采集模块的采集参数以适应于人脸识别场景，通过第二次调整图像采集模块的采集参数以适应于眼球识别场景。
步骤S108、控制图像采集模块基于中间采集参数，对目标对象进行图像采集，得到中间图像数据。
步骤S108与步骤S104的实现方式一致，在此不再赘述。
可以理解的是，在步骤S108获得中间图像数据之后，本申请实施例还可以继续对中间图像数据执行上述步骤S105和步骤S106中的分析，以进一步确定新的应用场景。若图像采集模块的采集参数与新的应用场景不匹配，则再次对采集参数进行调整，使得采集参数与新的应用场景相匹配。
步骤S109、若目标采集参数与目标应用场景对应的采集参数匹配，则执行步骤S104。
若目标采集参数与目标应用场景对应的采集参数匹配，则不需要对采集参数进行第二次调整，控制图像采集模块基于目标采集参数，对目标对象进行图像采集即可。
本申请实施例通过检测目标图像数据中图像内容的局部信息，获得局部图像特征点和/或局部图像特征点的运动参数。然后根据局部图像特征点和/或局部图像特征点的运动参数，确定目标图像数据对应的目标应用场景。若目标采集参数与目标应用场景对应的采集参数不匹配，则确定与目标应用场景匹配的中间采集参数，从而控制图像采集模块基于中间采集参数，对目标对象进行图像采集，得到中间图像数据。通过多级调整策略，避免采集参数一次调整跨度过大，导致功耗过大或与应用场景不匹配的问题，采集参数沿阶梯曲线增长或减小，从而在图像识别效果和系统功耗之间找到平衡，实现了在低功耗的基础上提高图像识别的准确性。
在一些实施例中，在上述任一实施例中步骤S104或步骤S109之后，本申请实施例提供的图像采集方法还包括以下步骤。根据目标采集参数确定识别处理模块的工作参数。识别处理模块根据工作参数，对目标图像数据中的目标对象进行识别处理。
本申请实施例中，首先图像采集模块基于目标采集参数，对目标对象进行图像采集，得到目标图像数据。然后根据目标采集参数确定识别处理模块的工作参数，识别处理模块根据工作参数，对目标图像数据中的目标对象进行识别处理。在识别处理模块识别处理完成后，输出图像，用户才能看到显示的图像。即本申请实施例中图像采集方法是在图像显示之前完成的。
以采集参数包括帧率和分辨率为例，为了降低系统功耗，提高系统处理能力，默认的初始采集参数设置为低帧率和低分辨率。识别处理模块的初始工作参数与图像采集模块的初始采集参数相对应，识别处理模块的初始工作参数是与低帧率和低分辨率相匹配的。因此在调整图像采集模块的采集参数之后，需要根据目标采集参数确定识别处理模块的工作参数，将调整后的帧率和分辨率同步到识别处理模块，以保证整个链路的设置同步，使得调整后的数据流能够正常流转和处理。
示例性的，以图像采集模块是AON sensor、采集参数包括帧率和分辨率为例进行说明，AON sensor不会向用户显示图像，而是将采集到的原始图像数据发送至识别处理模块，进行识别处理。因此，AON sensor的帧率和分辨率的调整，依赖于AON sensor启动时设置的帧率和分辨率。但是相关技术中以固定的帧率和分辨率进行图像采集的方式。图像采集模块在低帧率或低分辨率下采集图像，识别处理模块对低帧率或低分辨率的图 像进行识别处理，提高了系统整体功耗，降低了图像识别处理的准确性。本申请实施例中，图像采集模块在与应用场景匹配的帧率或分辨率下采集图像，识别处理模块是对与应用场景匹配的图像进行识别处理，在低功耗的前提下，提高了图像识别处理的准确性。
在一些实施例中，本申请实施例不仅可以通过对图像采集模块的采集参数进行调整，然后识别处理模块根据与目标采集参数对应的工作参数，对目标图像数据中的目标对象进行识别处理，从而提高图像识别处理的效率。还可以通过对目标图像数据的处理过程进行调整以达到同样的技术效果。示例性的，在识别处理模块读取目标图像数据的数据帧时，调整读取数据帧的间隔，例如，隔3帧、隔2帧以及逐帧读取的方式，可以降低图像数据处理的压力，减少系统功耗，提高图像识别处理的效率。
下面，将说明本申请实施例在一个实际的应用场景中的示例性应用。如图5所示，图5为本申请实施例提供的一种图像处理系统前端的结构示意图。
示例性的，以图像采集模块是AON sensor、采集参数包括帧率和分辨率，为例来进行下面的说明。
图5中图像处理系统前端包括AON sensor、图像内容检测模块以及帧率和分辨率控制模块，AON sensor根据默认的低帧率和低分辨率采集目标对象的原始图像数据，图5中以RAW图像表示原始图像数据，RAW图像就是图像感应器将捕捉到的光源信号转化为数字信号的原始图像数据。
图像内容检测模块对获得的RAW图像进行检测，该检测为概略性检测，主要目的是为了初步判断是否有需要识别的应用场景。该检测过程可以通过对图像特征点和/或图像特征点的运动参数的检测实现，当检测到预设图像特征点和/或预设的图像特征点的运动参数时，确定出现了需要识别的应用场景。图像内容检测模块对RAW图像进行图像内容检测分析，可以通过神经网络处理器(Neural-network Processing Units，NPU)进行处理，NPU为神经网络(neural-network，NN)的计算处理器，能够对输入信息进行快速处理。
图像内容检测模块将图像内容检测结果发送至帧率和分辨率控制模块，该图像内容检测结果用于确定应用场景，图5中是以帧率和分辨率控制模块根据图像内容检测结果确定应用场景为例进行说明。帧率和分辨率控制模块根据图像内容检测结果，基于预设的帧率和分辨率调整策略，确定合适的帧率和分辨率。将AON sensor的帧率和分辨率调整至合适的帧率和分辨率，并基于调整后的帧率和分辨率同步整条数据链路，例如，同步识别处理模块的工作参数。
需要说明的是，本申请实施例可以由图像内容检测模块根据图像内容检测结果确定应用场景，也可以由帧率和分辨率控制模块根据图像内容检测结果确定应用场景，对此本申请实施例不做限制。
上述帧率和分辨率调整策略，可以根据不同的应用场景进行差异化的调整设置。示例性的，对于需要进行姿势变换的检测场景(例如手势检测或动作检测)，优先提高帧率，以保证在单位时间内有更多的有效图像数据帧，可供分析识别。对于根据图像信息准确性的应用场景(例如，针对需要对眼球信息进行识别的场景)，优先提高分辨率，以保证通过更多的眼球细节信息，提高识别的准确性。
然后AON sensor基于调整后的帧率和分辨率进行图像采集，得到目标图像数据。其中，帧率和分辨率控制模块根据预设的帧率和分辨率进行调整时，可以根据调整策略执行帧率和分辨率决策。在本申请实施例中，对于图5中帧率和分辨率调整策略，也可以根据AON sensor的调整结果(tuning的结果)确定配置文件，也就是应用场景与AON sensor的采集参数具有映射关系，当需要对采集参数进行调整时，通过对配置文件的查询决定目标采集参数即可。也可以基于应用场景设置配置文件允许的采集参数的调整范 围，结合其他的相关参数进行类似插值的处理方式，从而确定目标采集参数。例如，对于人像的场景，可以在配置文件允许的采集参数的调整范围内，结合人像所在位置的深度信息，计算得到目标采集参数。其中，深度信息可以理解为目标对象与镜头之间的距离，如上述图3所示，在此不再赘述。
本申请实施例系统前端的图像内容检测模块对AON sensor输出的原始图像数据，进行初步的应用场景识别，得到应用场景的判断结果。可以理解的是，也可以由帧率和分辨率决策模块根据图像内容检测结果确定应用场景。然后帧率和分辨率决策模块根据应用场景的判断结果，对帧率和分辨率进行调整，使得系统在低功耗的前提下，实现应用场景准确识别的效果。本申请实施例不需要用户介入，直接由前端的图像内容检测模块对应用场景进行判断，进而帧率和分辨率决策模块根据应用场景的判断结果，进行帧率和分辨率的调整。因此，本申请实施例对于用户来说是不可见的，自动完成的，提高了交互效率。
结合图5提供的图像处理系统前端，本申请实施例提供了一种对基于图像检测的AON sensor进行配置调整的方法，其调整的采集参数可以包括帧率和/或分辨率，用于实现在低功耗的前提下提高图像识别处理的准确性。如图6所示，图6为本申请实施例提供的再一种图像采集方法的步骤流程图，包括步骤S601-步骤S609。图6中以调整帧率和分辨率为例来进行说明，可以理解的是，在调整采集参数时，也可以仅调整帧率或分辨率。
步骤S601、AON Sensor采集原始图像数据，将原始图像数据发送至图像内容检测模块进行图像内容检测处理。
需要说明的是，在步骤S601之前，本申请实施例打开AON sensor，并启动图像处理系统前端的Pre-ISP(Image Signal Processing)模块，Pre-ISP模块表示前置图像信号处理器。
步骤S602、图像内容检测模块进行图像内容检测。
图像内容检测模块根据原始图像数据，进行图像特征点和/或图像特征点的运动参数的检测，得到进行帧率和分辨率调整的阈值信息，图6中将图像特征点或和/或图像特征点的运动参数设置为阈值1。
步骤S602中可以根据图像内容检测结果自动启动相关调整策略，也可以由用户提前设置拍摄的场景模式，例如，是手势识别功能模式还是眼球识别功能模式，根据该场景模式确定调整策略。
步骤S603、基于得到的阈值信息，结合帧率和分辨率调整策略进行调整。
步骤S603可以是多级调整，可以通过步骤S6031-步骤S6034实现。
步骤S6031、判断图像内容检测模块检测到的图像内容是否达到阈值1。
步骤S6032、若图像内容检测模块检测到的图像内容达到阈值1，则根据调整策略对帧率和分辨率进行第一次调整。
AON sensor利用初步调整后的帧率和分辨率进行图像采集，获得目标图像数据，图像内容检测模块根据目标图像数据进行调整后的图像内容的检测。
步骤S6033、判断图像内容检测模块检测到的调整后的图像内容是否达到阈值2。
步骤S6034、若图像内容检测模块检测到的调整后的图像内容达到阈值2，则根据调整策略对对帧率和分辨率进行第二次调整。
通过步骤S6031-步骤S6034的多级调整，从而继续进行精确度更高的应用场景的识别处理。
示例性的，通过图像内容识别模块对原始图像数据中的图像特征点和/或图像特征点的运动参数，进行识别分析，当图像特征点和/或图像特征点的运动参数的变化，达到阈 值1，将帧率和分辨率提高一个级别，例如，将帧率从15FPS提升为30FPS，再对需要识别的场景进行更加准确及时的识别判断，使得系统可以据此执行后续的控制操作，例如屏幕点亮或相机唤醒等操作，本申请实施例对此不作限制。
在一些实施例中，通过图像内容检测模块对图像内容(例如，人脸识别、手势检测)进行识别，当满足一定的条件之后(例如，识别到人脸、识别到不同的手势、识别到帧与帧之间手势的变化)，对帧率和分辨率进行调整，以适当的帧率和分辨率输出目标图像数据，进而，基于目标图像数据进行图像识别，提高了图像识别结果的准确性。
步骤S604、使用调整后的帧率和分辨率进行图像采集，并将调整后的帧率和分辨率对应的工作参数，同步到相应的识别处理模块，保证整条链路的设置同步处理。
步骤S605、基于调整后的帧率和分辨率配置，继续进行图像的预览或拍摄。
在对帧率和分辨率进行调整之后，同时将图像采集模块和识别处理模块调整为同一处理模式，即，将帧率和分辨率，同步到AON sensor的输出以及后续识别处理模块的输入。AON sensor基于高分辨率和高帧率采集目标图像数据，识别处理模块基于提高分辨率和帧率之后的图像数据(即目标图像数据)进行识别处理，得到更加准确的识别结果，以供后续系统执行对应的操作，例如，屏幕唤醒、相机打开等功能。
需要说明的是，上述阈值1和阈值2的选取可以根据多种因素进行选定，示例性的，根据应用场景中，对目标对象准确识别所需要的图像帧数，以及采集参数调整时得到的动作姿势的变化速度，反向确定需要在什么时刻对帧率和分辨率进行调整，进而确定对应的阈值1和阈值2，将阈值作为判断调整时间点时所依赖的判断条件。
本申请实施例基于图像特征点和/或图像特征点的运动参数，对AON sensor的采集参数(帧率和分辨率)进行调整。同时，利用底层处理的特点，根据调整后的帧率和分辨率，对识别处理模块的工作参数进行同步，保证调整之后系统的数据流能够正常流转和处理。示例性的，针对应用场景中图像识别内容的特点，采用不同的帧率和分辨率调整策略。例如，对于人像识别场景(人脸识别场景、眼球识别场景)，可以设置低帧率和高分辨率。对于手势检测场景，设置高帧率，用于匹配手势的快速变动，设置低分辨率，用于降低系统功耗。本申请实施例可以基于图像采集模块和图像内容检测模块的自反馈参数，进行自适应调整，例如，帧率和分辨率的调整。
本申请实施例提供了一种基于图像检测的AON sensor配置调整方案，调整参数包括帧率和分辨率，可以在低功耗的前提下，提高场景识别的准确度。利用AON sensor支持不同低分辨率(例如VGA，QVGA)，以及不同低帧率(2/5/10/20fps)图像输出的特点。在图像输出的同时，利用识别处理模块(人脸识别，手势识别)对图像内容进行识别。当满足一定的条件之后，例如，识别到人脸或不同的手势(包括帧与帧之间手势的变化)，对帧率和分辨率进行调整，以较高的分辨率和较高的帧率的模式输出图像信息，从而提高识别效果的准确性。
在使用AON sensor的过程中，可以通过以下方式判断是否采用了本申请实施例提供的图像采集方法。由于service会提供对应的dump接口，类似于camera service的相关操作，可以实时对当前对应AON sensor的输出帧率和输出分辨率进行监测。通过制造相关场景的变化，例如，制造一些手势识别的场景；若对于需要识别的场景发生时，出现帧率或分辨率的自动变化，则可以推测出该产品采用了本申请实施例提供的图像采集方法。还可以通过更多的不同类型场景的相关参数变化(帧率和分辨率)，确认对应产品的调节策略，并与本申请实施例提供的图像采集过程中的控制策略进行对比，以判断产品是否采用了本申请实施例提供的图像采集方法。
在一些实施例中，还可以通过以下示例实现图像识别处理的准确性。
示例一、对于帧率和分辨率的调整，不仅可以对AON sensor的采集参数进行调整， 也可以调整对图像数据流的处理策略，示例性的，在识别处理模块读取目标图像数据的数据帧时，可以调整读取数据帧的间隔(间隔3帧、间隔2帧以及逐帧读取的方式)，从而降低图像数据处理的压力，减少系统功耗。
示例二、对于采集参数的调整不仅仅局限于帧率和分辨率，同样可以通过调整图像位宽信息和目标图像的深度信息，对图像的数据量进行调整，以减少算法处理时的压力，节省系统功耗。因此，对于相关配置的调整，还可以有更多的选择和对应的调整策略，使得系统整体识别处理效果和功耗达到平衡，从而减少系统功耗，提高图像识别处理的准确性。
基于上述实施例的图像采集方法，本申请实施例还提供一种图像采集装置，如图7所示，图7为本申请实施例提供的一种图像采集装置的结构示意图，该图像采集装置70包括：第一采集部分701，配置为通过图像采集模块采集目标对象的原始图像数据；第一确定部分702，配置为通过检测原始图像数据中图像内容，确定原始图像数据对应的应用场景，应用场景表征对目标对象进行识别处理的场景；第二确定部分703，配置为确定与应用场景匹配的目标采集参数；第二采集部分704，配置为控制图像采集模块基于目标采集参数，对目标对象进行图像采集，得到目标图像数据。
在本申请实施例中，第一确定部分702，还配置为通过检测原始图像数据中图像内容，获得图像特征点和/或图像特征点的运动参数；根据图像特征点和/或图像特征点的运动参数，与预设应用场景进行匹配，获得原始图像数据对应的应用场景。
在本申请实施例中，目标采集参数包括以下中的至少一项：帧率、分辨率、焦距和图像位宽信息。
在本申请实施例中，第二确定部分703，还配置为根据第一映射关系，确定与应用场景匹配的采集参数，得到目标采集参数，第一映射关系表征应用场景与采集参数之间的对应关系。
在本申请实施例中，第一映射关系包括：若应用场景为用于识别目标对象的动作的应用场景，则目标采集参数包括第一帧率，高于采集原始图像数据使用的第二帧率；若应用场景为用于识别目标对象或识别目标对象的局部细节的应用场景，则目标采集参数包括第一分辨率，第一分辨率高于采集原始图像数据使用的第二分辨率。
在本申请实施例中，第二确定部分703，还配置为根据第二映射关系，确定与应用场景匹配的采集参数的调整范围，得到目标采集参数的调整范围，第二映射关系表征应用场景与采集参数的调整范围之间的对应关系；根据原始图像数据中目标对象的深度信息，在目标采集参数的调整范围中确定目标采集参数。
在本申请实施例中，第二采集部分704，还配置为根据应用场景中对目标对象进行识别处理所需要的期望图像帧数，或者，应用场景中对目标对象的识别处理过程完成所需要的时长，确定调整时间点；在调整时间点，控制图像采集模块基于目标采集参数，对目标对象进行图像采集，得到目标图像数据。
在本申请实施例中，第一确定部分702，还配置为检测目标图像数据中图像内容的局部信息，获得局部图像特征点和/或局部图像特征点的运动参数；根据局部图像特征点和/或局部图像特征点的运动参数，确定目标图像数据对应的目标应用场景，目标应用场景表征对目标对象的局部细节进行识别处理的场景；第二采集部分704，还配置为若目标采集参数与目标应用场景对应的采集参数不匹配，则确定与目标应用场景匹配的中间采集参数；控制图像采集模块基于中间采集参数，对目标对象进行图像采集，得到中间图像数据。
在本申请实施例中，图像采集装置70还包括识别部分705，识别部分705，配置为根据目标采集参数确定识别处理模块的工作参数；识别处理模块根据工作参数，对目标 图像数据中的目标对象进行识别处理。
需要说明的是，上述实施例提供的图像采集装置在进行图像处理时，仅以上述各程序模块的划分进行举例说明，实际应用中，可以根据需要而将上述处理分配由不同的程序模块完成，即将装置的内部结构划分成不同的程序模块，以完成以上描述的全部或者部分处理。另外，上述实施例提供的图像采集装置与图像采集方法实施例属于同一构思，其具体实现过程及有益效果详见方法实施例，这里不再赘述。对于本装置实施例中未披露的技术细节，请参照本申请方法实施例的描述而理解。
在本申请的实施例中，图8为本申请实施例提出的终端组成结构示意图，如图8所示，本申请实施例提出的终端80包括处理器801和存储器802，存储器802存储有可在处理器801上运行的计算机程序，在本申请的一些实施例中，终端80还可以包括通信接口803，和用于连接处理器801、存储器802以及通信接口803的总线804。
在本申请的实施例中，上述处理器801可以为特定用途集成电路(Application Specific Integrated Circuit，ASIC)、数字信号处理器(Digital Signal Processor，DSP)、数字信号处理装置(Digital Signal Processing Device，DSPD)、可编程逻辑装置(ProgRAMmable Logic Device，PLD)、现场可编程门阵列(Field ProgRAMmable Gate Array，FPGA)、中央处理器(Central Processing Unit，CPU)、控制器、微控制器、微处理器中的至少一种。可以理解地，对于不同的设备，用于实现上述处理器功能的电子器件还可以为其它，本申请实施例不作具体限定。终端80还可以包括存储器802，该存储器802可以与处理器801连接，其中，存储器802配置为存储可执行程序代码，该程序代码包括计算机操作指令，存储器802可能包含高速RAM存储器，也可能还包括非易失性存储器，例如，至少两个磁盘存储器。
在本申请的实施例中，总线804用于连接通信接口803、处理器801以及存储器802以及这些器件之间的相互通信。
在本申请的实施例中，存储器802，配置为存储指令和数据。
在本申请的实施例中，上述处理器801，运行存储器802中存储的计算机程序时，可以执行以下指令：通过图像采集模块采集目标对象的原始图像数据；通过检测原始图像数据中图像内容，确定原始图像数据对应的应用场景，应用场景表征对目标对象进行识别处理的场景；确定与应用场景匹配的目标采集参数；控制图像采集模块基于目标采集参数，对目标对象进行图像采集，得到目标图像数据。
在实际应用中，上述存储器802可以是易失性存储器(volatile memory)，例如随机存取存储器(Random-Access Memory，RAM)；或者非易失性存储器(non-volatile memory)，例如只读存储器(Read-Only Memory，ROM)，快闪存储器(flash memory)，硬盘(Hard Disk Drive，HDD)或固态硬盘(Solid-State Drive，SSD)；或者上述种类的存储器的组合，并向处理器801提供指令和数据。
另外，在本实施例中的各功能模块可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。
集成的单元如果以软件功能模块的形式实现并非作为独立的产品进行销售或使用时，可以存储在一个计算机可读取存储介质中，基于这样的理解，本实施例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)或processor(处理器)执行本实施例方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、 磁碟或者光盘等各种可以存储程序代码的介质。
本申请实施例提供一种计算机可读存储介质，其上存储有程序，该程序被处理器执行时实现如上任一实施例所述的图像采集方法。
示例性的，本实施例中的一种图像采集方法对应的程序指令可以被存储在光盘，硬盘，U盘等存储介质上，当存储介质中的与一种图像采集方法对应的程序指令被一电子设备读取或被执行时，可以实现如上述任一实施例所述的图像采集方法。
本领域内的技术人员应明白，本申请的实施例可提供为方法、系统、或计算机程序产品。因此，本申请可采用硬件实施例、软件实施例、或结合软件和硬件方面的实施例的形式。而且，本申请可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器和光学存储器等)上实施的计算机程序产品的形式。
如图9所示，图9是本申请实施例的芯片的示意性结构图。图9所示的芯片90包括处理器901，处理器901可以从存储器中调用并运行计算机程序，以实现本申请实施例中的图像采集方法。
示例性的，如图9所示，芯片90还可以包括存储器902。其中，处理器901可以从存储器902中调用并运行计算机程序，以实现本申请实施例中的图像采集方法。
其中，存储器902可以是独立于处理器901的一个单独的器件，也可以集成在处理器901中。
示例性的，该芯片90还可以包括输入接口903。其中，处理器901可以控制该输入接口903与其他设备或芯片进行通信，以及获取其他设备或芯片发送的信息或数据。
示例性的，该芯片90还可以包括输出接口904。其中，处理器901可以控制该输出接口904与其他设备或芯片进行通信，以及可以向其他设备或芯片输出信息或数据。
示例性的，该芯片可应用于本申请实施例中的终端，并且该芯片可以实现本申请实施例的各个方法中由终端实现的相应流程，为了简洁，在此不再赘述。
应理解，本申请实施例提到的芯片还可以称为系统级芯片，系统芯片，芯片系统或片上系统芯片等。
本申请实施例还提供了一种计算机程序产品，包括计算机程序指令。
示例性的，该计算机程序产品可应用于本申请实施例中的终端，并且该计算机程序指令使得计算机执行本申请实施例的各个方法中由终端实现的相应流程，为了简洁，在此不再赘述。
本申请实施例还提供了一种计算机程序。
示例性的，该计算机程序可应用于本申请实施例中的终端，当该计算机程序在计算机上运行时，使得计算机执行本申请实施例的各个方法中由终端实现的相应流程，为了简洁，在此不再赘述。
本申请是参照根据本申请实施例的方法、设备(系统)、和计算机程序产品的实现流程示意图和/或方框图来描述的。应理解可由计算机程序指令实现流程示意图和/或方框图中的每一流程和/或方框、以及实现流程示意图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在实现流程示意图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在实现流程示意图一个流程或多个流程和/或方框图一 个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在实现流程示意图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
以上所述，仅为本申请的较佳实施例而已，并非用于限定本申请的保护范围。
工业实用性
本申请实施例公开了一种图像采集方法、装置、终端、计算机可读存储介质及计算机程序产品。该方法通过图像采集模块采集目标对象的原始图像数据，在采集到原始图像数据之后，通过检测原始图像数据中图像内容，确定原始图像数据对应的应用场景；接下来确定与应用场景匹配的目标采集参数；控制图像采集模块基于目标采集参数，对目标对象进行图像采集，得到目标图像数据。本申请实施例在图像采集过程中对图像采集模块的采集参数进行适应性调整，调整得到的目标采集参数与应用场景相匹配，图像采集模块基于与应用场景匹配的目标采集参数进行图像采集，进而，基于目标图像数据进行图像识别，能够提高图像识别结果的准确性。

Claims (19)

  1. 一种图像采集方法，所述方法包括：
    通过图像采集模块采集目标对象的原始图像数据；
    通过检测所述原始图像数据中图像内容，确定所述原始图像数据对应的应用场景，所述应用场景表征对所述目标对象进行识别处理的场景；
    确定与所述应用场景匹配的目标采集参数；
    控制所述图像采集模块基于所述目标采集参数，对所述目标对象进行图像采集，得到目标图像数据。
  2. 根据权利要求1所述的方法，其中，所述通过检测所述原始图像数据中图像内容，确定所述原始图像数据对应的应用场景，包括：
    通过检测所述原始图像数据中图像内容，获得图像特征点和/或所述图像特征点的运动参数；
    根据所述图像特征点和/或所述图像特征点的运动参数，与预设应用场景进行匹配，获得所述原始图像数据对应的应用场景。
  3. 根据权利要求1所述的方法，其中，所述目标采集参数包括以下中的至少一项：帧率、分辨率、焦距和图像位宽信息。
  4. 根据权利要求1-3任一项所述的方法，其中，所述确定与所述应用场景匹配的目标采集参数，包括：
    根据第一映射关系，确定与所述应用场景匹配的采集参数，得到所述目标采集参数，所述第一映射关系表征所述应用场景与采集参数之间的对应关系。
  5. 根据权利要求4所述的方法，其中，所述第一映射关系包括：
    若所述应用场景为用于识别所述目标对象的动作的应用场景，则所述目标采集参数包括第一帧率，所述第一帧率高于采集所述原始图像数据使用的第二帧率；
    若所述应用场景为用于识别所述目标对象或识别所述目标对象的局部细节的应用场景，则所述目标采集参数包括第一分辨率，所述第一分辩率高于采集所述原始图像数据使用的第二分辨率。
  6. 根据权利要求1-5任一项所述的方法，其中，所述确定与所述应用场景匹配的目标采集参数，包括：
    根据第二映射关系，确定与所述应用场景匹配的采集参数的调整范围，得到所述目标采集参数的调整范围，所述第二映射关系表征所述应用场景与采集参数的调整范围之间的对应关系；
    根据所述原始图像数据中所述目标对象的深度信息，在所述目标采集参数的调整范围中确定所述目标采集参数。
  7. 根据权利要求1-5任一项所述的方法，其中，控制所述图像采集模块基于所述目标采集参数，对所述目标对象进行图像采集，得到目标图像数据，包括：
    根据所述应用场景中对所述目标对象进行识别处理所需要的期望图像帧数，或者，所述应用场景中对所述目标对象的识别处理过程完成所需要的时长，确定调整时间点；
    在所述调整时间点，控制所述图像采集模块基于所述目标采集参数，对所述目标对象进行图像采集，得到目标图像数据。
  8. 根据权利要求1-7任一项所述的方法，其中，所述方法还包括：
    检测所述目标图像数据中图像内容的局部信息，获得局部图像特征点和/或局部图像特征点的运动参数；
    根据所述局部图像特征点和/或所述局部图像特征点的运动参数，确定所述目标图像数据对应的目标应用场景，所述目标应用场景表征对所述目标对象的局部细节进行识别处理的场景；
    若所述目标采集参数与所述目标应用场景对应的采集参数不匹配，则确定与所述目标应用场景匹配的中间采集参数；
    控制所述图像采集模块基于所述中间采集参数，对所述目标对象进行图像采集，得到中间图像数据。
  9. 根据权利要求1-7任一项所述的方法，其中，所述方法还包括：
    根据所述目标采集参数确定识别处理模块的工作参数；
    所述识别处理模块用于根据所述工作参数，对所述目标图像数据中的所述目标对象进行识别处理。
  10. 一种图像采集装置，所述装置包括：
    第一采集部分，配置为通过图像采集模块采集目标对象的原始图像数据；
    第一确定部分，配置为通过检测所述原始图像数据中图像内容，确定所述原始图像数据对应的应用场景，所述应用场景表征对所述目标对象进行识别处理的场景；
    第二确定部分，配置为确定与所述应用场景匹配的目标采集参数；
    第二采集部分，配置为控制所述图像采集模块基于所述目标采集参数，对所述目标对象进行图像采集，得到目标图像数据。
  11. 根据权利要求10所述的装置，其中，
    所述第一确定部分，还配置为通过检测所述原始图像数据中图像内容，获得图像特征点和/或所述图像特征点的运动参数；根据所述图像特征点和/或所述图像特征点的运动参数，与预设应用场景进行匹配，获得所述原始图像数据对应的应用场景。
  12. 根据权利要求10所述的装置，其中，
    所述第二确定部分，还配置为根据第一映射关系，确定与所述应用场景匹配的采集参数，得到所述目标采集参数，所述第一映射关系表征所述应用场景与采集参数之间的对应关系。
  13. 根据权利要求10所述的装置，其中，
    所述第二确定部分，还配置为根据第二映射关系，确定与所述应用场景匹配的采集参数的调整范围，得到所述目标采集参数的调整范围，所述第二映射关系表征所述应用场景与采集参数的调整范围之间的对应关系；根据所述原始图像数据中所述目标对象的深度信息，在所述目标采集参数的调整范围中确定所述目标采集参数。
  14. 根据权利要求10-13任一项所述的装置，其中，
    所述第二采集部分，还配置为根据所述应用场景中对所述目标对象进行识别处理所需要的期望图像帧数，或者，所述应用场景中对所述目标对象的识别处理过程完成所需要的时长，确定调整时间点；在所述调整时间点，控制所述图像采集模块基于所述目标采集参数，对所述目标对象进行图像采集，得到目标图像数据。
  15. 根据权利要求10-13任一项所述的装置，其中，
    所述第一确定部分，还配置为检测所述目标图像数据中图像内容的局部信息，获得局部图像特征点和/或局部图像特征点的运动参数；根据所述局部图像特征点和/或所述局部图像特征点的运动参数，确定所述目标图像数据对应的目标应用场景，所述目标应用场景表征对所述目标对象的局部细节进行识别处理的场景；
    所述第二采集部分，还配置为若所述目标采集参数与所述目标应用场景对应的采集参数不匹配，则确定与所述目标应用场景匹配的中间采集参数；控制所述图像采集模块基于所述中间采集参数，对所述目标对象进行图像采集，得到中间图像数据。
  16. 根据权利要求10-13任一项所述的装置，其中，所述装置还包括识别部分；
    所述识别部分，配置为根据所述目标采集参数确定识别处理模块的工作参数；所述识别处理模块用于根据所述工作参数，对所述目标图像数据中的所述目标对象进行识别处理。
  17. 一种终端，所述终端包括存储器和处理器；
    所述存储器存储有可在所述处理器上运行的计算机程序；
    所述处理器执行所述计算机程序时实现权利要求1-9任一项所述的方法。
  18. 一种计算机可读存储介质，其上存储有可执行指令，配置为被处理器执行时实现权利要求1-9任一项所述的方法。
  19. 一种计算机程序产品，包括计算机程序指令，所述计算机程序指令使得计算机执行如权利要求1-9任一项所述的方法。
PCT/CN2022/112739 2021-09-22 2022-08-16 图像采集方法、装置、终端、计算机可读存储介质及计算机程序产品 WO2023045626A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111106280.0A CN115862081A (zh) 2021-09-22 2021-09-22 图像采集方法、装置、设备和计算机可读存储介质
CN202111106280.0 2021-09-22

Publications (1)

Publication Number Publication Date
WO2023045626A1 true WO2023045626A1 (zh) 2023-03-30

Family

ID=85652117

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/112739 WO2023045626A1 (zh) 2021-09-22 2022-08-16 图像采集方法、装置、终端、计算机可读存储介质及计算机程序产品

Country Status (2)

Country Link
CN (1) CN115862081A (zh)
WO (1) WO2023045626A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116456057B (zh) * 2023-04-26 2023-11-14 河南铭视科技股份有限公司 一种基于物联网的视频处理系统及方法
CN116320737B (zh) * 2023-05-10 2023-08-18 珠海星云智联科技有限公司 用于千兆以太网标准工业相机的控制方法、设备及介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111050150A (zh) * 2019-12-24 2020-04-21 成都极米科技股份有限公司 焦距调节方法、装置、投影设备及存储介质
CN111144439A (zh) * 2019-11-28 2020-05-12 浪潮金融信息技术有限公司 一种物体识别中选择最优图像匹配算法的方法
CN111383224A (zh) * 2020-03-19 2020-07-07 Oppo广东移动通信有限公司 图像处理方法、装置、存储介质及电子设备
CN113259597A (zh) * 2021-07-16 2021-08-13 上海豪承信息技术有限公司 图像处理方法、装置、设备、介质及程序产品

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111144439A (zh) * 2019-11-28 2020-05-12 浪潮金融信息技术有限公司 一种物体识别中选择最优图像匹配算法的方法
CN111050150A (zh) * 2019-12-24 2020-04-21 成都极米科技股份有限公司 焦距调节方法、装置、投影设备及存储介质
CN111383224A (zh) * 2020-03-19 2020-07-07 Oppo广东移动通信有限公司 图像处理方法、装置、存储介质及电子设备
CN113259597A (zh) * 2021-07-16 2021-08-13 上海豪承信息技术有限公司 图像处理方法、装置、设备、介质及程序产品

Also Published As

Publication number Publication date
CN115862081A (zh) 2023-03-28

Similar Documents

Publication Publication Date Title
RU2762142C1 (ru) Способ и устройство для определения ключевой точки лица, компьютерное устройство и носитель данных
WO2023045626A1 (zh) 图像采集方法、装置、终端、计算机可读存储介质及计算机程序产品
TWI751161B (zh) 終端設備、智慧型手機、基於臉部識別的認證方法和系統
WO2019128508A1 (zh) 图像处理方法、装置、存储介质及电子设备
US11715231B2 (en) Head pose estimation from local eye region
WO2019120029A1 (zh) 智能调节屏幕亮度的方法、装置、存储介质及移动终端
US9147107B2 (en) Facial expression recognition apparatus, image sensing apparatus, facial expression recognition method, and computer-readable storage medium
US9288388B2 (en) Method and portable terminal for correcting gaze direction of user in image
WO2017161867A1 (zh) 一种调节屏幕亮度的方法、装置及智能终端
US8243159B2 (en) Image capturing device, image processing device, image analysis method for the image capturing device and the image processing device, and program for facial attribute detection
CN112771612B (zh) 拍摄图像的方法和装置
US10380420B2 (en) Electronic device and screen adjusting method based on condition of user's eye
CN104182721A (zh) 提升人脸识别率的图像处理系统及图像处理方法
CN113903317B (zh) 电子设备的屏幕亮度调节方法、装置以及电子设备
WO2023044233A1 (en) Region of interest capture for electronic devices
CN111880640A (zh) 屏幕控制方法、装置、电子设备及存储介质
CN108093170B (zh) 用户拍照方法、装置及设备
JP2007013480A (ja) 監視システム、監視方法、及び監視プログラム
CN108259769B (zh) 图像处理方法、装置、存储介质及电子设备
CN108564537B (zh) 图像处理的方法、装置、电子设备及介质
JP6373446B2 (ja) ビデオフレームを選択するプログラム、システム、装置、及び方法
US9323981B2 (en) Face component extraction apparatus, face component extraction method and recording medium in which program for face component extraction method is stored
CN108495038B (zh) 图像处理方法、装置、存储介质及电子设备
TW202011154A (zh) 目標物資訊的預載顯示方法及裝置
CN110121030B (zh) 一种保存拍摄图像的方法以及电子设备

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22871684

Country of ref document: EP

Kind code of ref document: A1