WO2022148142A1

WO2022148142A1 - Image processing method and apparatus

Info

Publication number: WO2022148142A1
Application number: PCT/CN2021/131470
Authority: WO
Inventors: 孙旭彤; 池清华; 张仁宇
Original assignee: 华为技术有限公司
Priority date: 2021-01-05
Filing date: 2021-11-18
Publication date: 2022-07-14
Also published as: CN114723928A

Abstract

An image processing method and apparatus, which can be applied to the field of intelligent driving of smart automobiles, self-driving or autonomous driving. The image processing method comprises: acquiring at least one initial image (S210); and according to the at least one initial image and a first mapping relationship, obtaining a first pre-processed image after being subjected to pre-processing, wherein there is a first mapping relationship between the at least one initial image and a pixel position of the first pre-processed image, and the first mapping relationship corresponds to a plurality of pre-processing manners (S220). Pre-processing is performed on an initial image by means of a first mapping relationship, so that there is a first mapping relationship between a first pre-processed image obtained by means of pre-processing and a pixel position of the initial image, avoiding the generation and storage of intermediate data, and reducing the occupancy of storage resources.

Description

Image processing method and device

This application claims the priority of the Chinese patent application with the application number 202110009291.0 and the application name "Image Processing Method and Apparatus" filed with the China Patent Office on January 5, 2021, the entire contents of which are incorporated into this application by reference.

technical field

The present application relates to the field of image processing, and can be specifically applied to intelligent driving, automatic driving, or unmanned driving, etc., and more specifically, to an image processing method and apparatus.

Background technique

During image analysis, the quality of the image directly affects the accuracy of the design and effect of the recognition algorithm. Therefore, preprocessing is required before image analysis (feature extraction, segmentation, matching, and recognition, etc.). The main purpose of image preprocessing is to eliminate irrelevant information in images, restore useful real information, enhance the detectability of relevant information, and simplify data to the greatest extent, thereby improving the reliability of feature extraction, image segmentation, matching and recognition.

When preprocessing the initial image collected by the camera, perform translation, transposition, mirroring, rotation, affine transformation, inverse perspective transformation (IPM), lens distortion correction (LDC), resolution in turn. Various preprocessing operations in rate adjustment, etc. In the process of preprocessing the initial image, a large number of intermediate calculation results are generated, occupying the storage resources of memory or other memory.

SUMMARY OF THE INVENTION

The present application provides an image processing method and apparatus, which can reduce resource occupation of image preprocessing.

In a first aspect, an image processing method is provided, comprising: acquiring at least one initial image; obtaining a preprocessed first preprocessed image according to the at least one initial image and a first mapping relationship, the at least one initial image and A first mapping relationship exists between pixel positions of the first preprocessed image, and the first mapping relationship corresponds to multiple preprocessing modes.

By using the first mapping relationship to preprocess the initial image, there is a first mapping relationship between the first preprocessed image obtained by preprocessing and the pixel positions of the initial image, which avoids the generation and storage of intermediate data, and reduces the need for Occupation of storage resources.

With reference to the first aspect, in some possible implementation manners, the obtaining a preprocessed first preprocessed image according to the at least one initial image and the first mapping relationship includes: according to the first mapping relationship, to A first region image of a first ROI in the at least one region of interest ROI in the at least one initial image is preprocessed to obtain the first preprocessed image.

By preprocessing the image located in the first ROI, the amount of data that needs to be preprocessed is reduced, and the time required for preprocessing is reduced.

With reference to the first aspect, in some possible implementation manners, the number of the at least one ROI is multiple, the first ROI corresponds to the first mapping relationship, and the method further includes: using a method corresponding to the second ROI Preprocessing is performed on the second region image located in the second ROI in the at least one initial image, where the second ROI is an ROI other than the first ROI in the at least one ROI.

If the subsequent image processing puts forward various requirements for image preprocessing, that is, the preprocessing results of the initial image need to be obtained by different preprocessing methods, the number of ROIs can be multiple, and each ROI can correspond to a kind of preprocessing processing needs. Using the mapping relationship corresponding to each ROI for preprocessing provides a more flexible image processing method.

It should be understood that each ROI may correspond to a mapping relationship. Each ROI can be preprocessed by using the mapping relationship between the original image and the pixel positions in the preprocessed image. When each ROI needs to be preprocessed by multiple preprocessing methods, the images located in each ROI are preprocessed by mapping, which avoids the storage of the intermediate data generated by each preprocessing method and reduces the consumption of storage resources.

Each ROI can include only one continuous area, or multiple continuous areas separated from each other.

With reference to the first aspect, in some possible implementations, the at least one initial image is from a vehicle-mounted camera device, and the position of each of the ROIs is acquired according to the driving state of the vehicle.

The position of the ROI is adjusted according to the driving state of the vehicle, so as to avoid that the preprocessing result of the image of the region where the initial image is located in the ROI does not meet the requirements of subsequent image processing due to the change of the driving state of the vehicle.

With reference to the first aspect, in some possible implementations, the first mapping relationship includes a plurality of sub-mapping relationships, and each of the sub-mapping relationships corresponds to a sub-region in the first ROI. The first mapping relationship, performing preprocessing on the first region image of the first ROI in the at least one region of interest ROI in the at least one initial image includes: according to the plurality of sub-mapping relationships, performing preprocessing on the at least one initial image in the first ROI. The images located in each of said sub-regions are preprocessed in parallel.

The first mapping relationship includes a plurality of sub-mapping relationships, and the initial images in each region corresponding to the sub-mapping relationships are preprocessed in parallel by using the sub-mapping relationships. Therefore, while parallel processing is performed to reduce the time of image preprocessing, the division of the initial image is avoided, so that the parallel preprocessing of the initial image is more convenient.

With reference to the first aspect, in some possible implementations, the first mapping relationship includes a plurality of sub-mapping relationships, each of the sub-mapping relationships corresponds to a sub-region, and the first mapping relationship is based on the at least one initial image and the first The mapping relationship to obtain a preprocessed first preprocessed image includes: performing parallel preprocessing on images located in each of the subregions in the at least one initial image according to the plurality of submapping relationships.

With reference to the first aspect, in some possible implementation manners, the multiple preprocessing manners include at least one of affine transformation, inverse perspective transformation IPM, lens distortion correction LDC, and geometric transformation. It should be understood that various preprocessing manners may also include other manners.

With reference to the first aspect, in some possible implementations, the at least one initial image is from a vehicle-mounted camera device, and the method further includes: acquiring demand information, where the demand information is used to indicate the multiple preprocessing methods; The requirement information generates the first mapping relationship.

According to the requirement information, a first mapping relationship is generated. On the one hand, the determination of the first mapping relationship is made more flexible; on the other hand, there is no need to store a large number of mapping relationships, and only the required first mapping relationship can be generated when needed, which reduces the occupation of storage resources.

In a second aspect, an image processing apparatus is provided, including an acquisition module and a processing module, where the acquisition module is configured to acquire at least one initial image; the processing module is configured to, according to the at least one initial image and the first mapping relationship, A preprocessed first preprocessed image is obtained, and a first mapping relationship exists between the at least one initial image and the pixel positions of the first preprocessed image, and the first mapping relationship corresponds to multiple preprocessing modes.

With reference to the second aspect, in some possible implementations, the processing module is specifically configured to, according to the first mapping relationship, perform the first mapping of the first ROI located in the first ROI in the at least one region of interest ROI in the at least one initial image. The region image is preprocessed to obtain the first preprocessed image.

With reference to the second aspect, in some possible implementations, the number of the at least one ROI is multiple, the first ROI corresponds to the first mapping relationship, and the processing module is further configured to: The mapping relationship corresponding to the ROI preprocesses the second region image located in the second ROI in the at least one initial image, and the second ROI is the ROI other than the first ROI in the at least one ROI . .

With reference to the second aspect, in some possible implementations, the at least one initial image is from a vehicle-mounted camera device, and the position of the ROI is obtained according to the driving state of the vehicle.

With reference to the second aspect, in some possible implementations, the first mapping relationship includes multiple sub-mapping relationships, each of the sub-mapping relationships corresponds to a sub-region in the first ROI, and the processing module specifically is configured to, according to the plurality of sub-mapping relationships, perform parallel preprocessing on images located in each of the sub-regions in the at least one initial image.

With reference to the second aspect, in some possible implementations, the first mapping relationship includes multiple sub-mapping relationships, each of the sub-mapping relationships corresponds to a sub-region, and the processing module is specifically configured to, according to the multiple sub-mapping relationships There are sub-mapping relationships, and the images located in each of the sub-regions in the at least one initial image are preprocessed in parallel.

With reference to the second aspect, in some possible implementation manners, the multiple preprocessing manners include at least one of affine transformation, inverse perspective transformation IPM, lens distortion correction LDC, and geometric transformation.

With reference to the second aspect, in some possible implementations, the obtaining module is further configured to obtain requirement information, where the requirement information is used to indicate the multiple preprocessing methods; the processing module is further configured to, according to the The requirement information is generated, and the first mapping relationship is generated.

In a third aspect, an image processing apparatus is provided, comprising at least one memory and at least one processor, wherein the at least one memory is used to store a program, and the at least one processor is used to run the program to implement the first aspect. method.

It should be understood that a program may also be referred to as program code, computer instructions, computer programs, program instructions, or the like.

In a fourth aspect, a chip is provided, comprising at least one processor and an interface circuit, the interface circuit is configured to provide program instructions or data for the at least one processor, and the at least one processor is configured to execute the program instructions, to implement the method described in the first aspect.

Optionally, as an implementation manner, the chip system may further include a memory, in which a program is stored, the processor is configured to execute the program stored in the memory, and when the program is executed, the The processor is configured to perform the method in the first aspect.

In a fifth aspect, a computer-readable storage medium is provided, where the computer-readable medium stores a program code for execution by a device, and when the program code is executed by the device, the method described in the first aspect is implemented.

In a sixth aspect, a computer program product is provided, the computer program product comprising a computer program, when the computer program product is executed by a computer, the computer performs the method of the aforementioned first aspect.

It should be understood that, in this application, the method of the first aspect may specifically refer to the method in the first aspect and any one of the various implementation manners of the first aspect.

In a seventh aspect, a terminal is provided, including the image processing apparatus described in the second aspect or the third aspect.

Further, the terminal may be an intelligent transportation device (vehicle or drone), a smart home device, an intelligent manufacturing device, or a robot, and the like. The intelligent transportation device may be, for example, an automated guided vehicle (AGV), or an unmanned transportation vehicle.

Description of drawings

FIG. 1 is a schematic structural diagram of an image processing system.

FIG. 2 is a schematic flowchart of an image processing method.

FIG. 3 is a schematic flowchart of an image processing method provided by an embodiment of the present application.

FIG. 4 is a schematic structural diagram of an image processing system provided by an embodiment of the present application.

FIG. 5 is a schematic diagram of an image processing method provided by an embodiment of the present application.

FIG. 6 is a schematic diagram of an image storage format.

FIG. 7 is a mapping relationship table provided by an embodiment of the present application.

FIG. 8 is a schematic structural diagram of an image processing apparatus provided by an embodiment of the present application.

FIG. 9 is a schematic structural diagram of another image processing apparatus provided by an embodiment of the present application.

Detailed ways

The technical solutions in the present application will be described below with reference to the accompanying drawings.

Intelligent driving technology relies on the cooperation of computer vision, radar, monitoring devices and global positioning systems, etc., so that motor vehicles can realize automatic driving or assisted driving without the need for human active operation or the need for partial human operation intervention. Intelligently driven vehicles use various computing systems to help transport passengers or cargo from one location to another. Some intelligently driven vehicles may require some initial or continuous input from an operator, such as a pilot, driver, or passenger. Intelligent driving vehicles permit the operator to switch from a manual mode of operation to an autonomous driving mode or a mode in between. Since automatic driving technology does not require humans to drive motor vehicles, it can theoretically effectively avoid human driving errors, reduce the occurrence of traffic accidents, and improve the efficiency of highway transportation. Therefore, more and more attention has been paid to intelligent driving technologies such as autonomous driving.

The mode between the manual mode operation mode and the automatic driving mode may be referred to as an assisted driving mode. In the assisted driving mode, the driving of the motor vehicle does not completely depend on the active operation of humans. The realization of autonomous driving and assisted driving depends on the development of intelligent driving technology.

Intelligent driving technology includes perception, decision-making, control and other stages. In the perception stage, information about the surrounding environment can be received, and the environment in which the cognition is located can be understood according to the surrounding environment information. In the decision-making stage, the information output in the perception stage can be used to predict the behavior of traffic participants, so as to make behavioral decisions for the self-vehicle. In the control stage, the lateral acceleration and longitudinal acceleration of the vehicle can be calculated according to the output of the decision-making stage to control the driving of the self-vehicle.

The surrounding environment information may include images captured by the vehicle-mounted camera device. In the perception stage, the images collected by the on-board camera can be processed.

In image analysis such as feature extraction, segmentation, matching and recognition, the quality of the initial image directly affects the design of the analysis algorithm and the accuracy of the analysis results. Therefore, the image needs to be preprocessed before image analysis. The main purpose of image preprocessing is to eliminate irrelevant information such as noise in the image, enhance the detectability of effective information, and simplify the data to the greatest extent, thereby improving the reliability of feature extraction, image segmentation, matching and recognition.

FIG. 1 is a schematic structural diagram of an image processing system.

The image processing system 100 includes a preprocessing module 110 and a processing module 120 . The functions of the preprocessing module 110 and the processing module 120 may be implemented by at least one processor.

The preprocessing module 110 is used for preprocessing the initial image.

The initial image may be an image captured by at least one camera. The initial image as shown in FIG. 1 may be collected by a forward-looking camera installed on the vehicle.

The preprocessing module 110 may perform translation, transposition, mirroring, rotation, affine transformation, inverse perspective transformation (IPM), lens distortion correction (LDC), resolution adjustment, etc. on the initial image. one or more operations.

Affine transformation is used to linearly transform the coordinates of each point in the image, and it can also translate the linearly transformed coordinates by a certain distance. Through affine transformation, the image can be rotated by any angle around any center.

In the field of intelligent driving such as automatic driving and assisted driving, the detection of lane lines is very important. In the image captured by the front-view camera, due to the existence of perspective effect, objects such as lane lines that are originally parallel tend to intersect in the image. The inverse perspective transform can eliminate this perspective effect. Through IPM, the projection of the object in the initial image along the vertical downward direction (ie, the orthographic projection) can be obtained.

Lenses can introduce distortion due to manufacturing accuracy and assembly process deviations, resulting in distortion of the original image. The distortion of the lens is divided into two categories: radial distortion and tangential distortion. Radial distortion is the distortion distributed along the radial direction of the lens, which is caused by the fact that the light rays are more curved at the center of the principle lens than near the center. Tangential distortion is caused by the lens itself being not parallel to the camera sensor plane (imaging plane) or the image plane, which is mostly caused by the installation deviation of the lens being pasted on the lens module. Image distortion caused by the lens can be reduced or even eliminated by LDC.

The preprocessed image shown in Figure 1 is obtained by LDC.

The processing module 120 may further process the preprocessed image.

For example, in the field of autonomous driving, the preprocessed image will be sent to the processing module 120 at the back end. The processing module 120 recognizes the preprocessed image, and determines people, vehicles, lane lines, traffic signs and the like in the preprocessed image.

In general, different neural network models can be used for the recognition of different types of targets. The processing result shown in FIG. 1 will be described by taking the recognition of a vehicle as an example.

When the initial image needs to undergo multiple preprocessing methods to obtain the required preprocessed image, the preprocessing module 110 sequentially uses multiple preprocessing methods to preprocess the initial image, and needs to perform preprocessing in each preprocessing method. After the processing is completed, the data obtained by the preprocessing method is stored.

Figure 2 is a schematic flow chart of an image processing method.

Many vehicles are equipped with in-vehicle surround view systems, that is, through a series of image preprocessing through at least four cameras installed around the body, an inverse perspective mapping (IPM) map (ie, bird's eye view, top view) is obtained. ) to assist the driver in scenarios such as parking.

The initial image can be collected using a fisheye camera. The azimuth field of view of the fisheye camera can reach 360°, and the elevation field of view can reach 90°, which can realize a wide range of monitoring without dead angle.

Specifically, the four-way fisheye cameras installed on the vehicle body collect initial images, and the preprocessing module 110 can respectively perform lens distortion correction and inverse perspective transformation on the initial images collected by each fisheye camera, and then the multiple initial images Image stitching is performed on the result after the perspective transformation of , so as to obtain a ring view, which is the preprocessed image.

Each of the various preprocessing methods such as lens distortion correction and inverse perspective transformation of the image can be realized by matrix operations or mapping according to the mapping relationship between the pixels in the image before and after processing the preprocessing method. . The preprocessing module 110 performs multiple preprocessing modes in sequence, and generates a preprocessing result corresponding to the preprocessing mode after completing each preprocessing mode.

Taking the vehicle driving on the right side of the road as an example, when the vehicle needs to park sideways, the parking route of the vehicle can be determined according to the environmental conditions on the right side of the vehicle. Image detection can be performed on the right region in the ring view obtained after preprocessing.

According to the preprocessing process, multiple preprocessing methods are sequentially performed on the initial image. Due to the limited performance of the processor, the image preprocessing process is slow. In addition, except for the last preprocessing method in the preprocessing process, after each processing corresponding to a preprocessing method, some intermediate calculation results will be generated, occupying the storage resources of memory or other memory.

In order to solve the above problem, an embodiment of the present application provides an image preprocessing method, which can reduce resource occupation of image preprocessing.

At S210, at least one initial image is acquired.

At least one initial image sent by other devices can be received. The at least one initial image can also be read in memory.

Each initial image may be acquired by a camera. After each camera device captures an initial image, the initial image is transmitted and stored in the memory.

It should be noted that, one or more of the at least one initial image may also be obtained by preliminarily processing the images collected by the camera device. For example, at least one preprocessing method may be performed on the image acquired by the camera to obtain at least one initial image. This application does not limit the acquisition method of the initial image, which is subject to subsequent preprocessing according to the mapping relationship.

At S220, a preprocessed first preprocessed image is obtained according to the at least one initial image and the first mapping relationship, and a first mapping exists between the at least one initial image and the pixel positions of the first preprocessed image relationship, the first mapping relationship corresponds to multiple preprocessing modes.

The multiple preprocessing manners corresponding to the first mapping relationship may include one or more of affine transformation, IPM, LDC, and geometric transformation. It should be understood that various preprocessing manners may also include other manners. The geometric transformation includes at least one of translation, transposition, mirroring, rotation, resolution adjustment, and the like.

That is to say, by using the first mapping relationship to process the initial image, the result obtained after processing through the various preprocessing methods can be realized.

Through S210 to S220, the first mapping relationship is used to realize the preprocessing of the at least one initial image, and there is no need to perform operations for each preprocessing mode respectively, which avoids the storage of intermediate data generated by each preprocessing mode, and reduces the need for storage. Occupation of resources.

Requirement information can be obtained, and the requirement information is used to indicate the multiple preprocessing methods. The first mapping relationship may be generated according to the requirement information. The requirement information can be used to indicate the requirement of the multiple preprocessing methods for subsequent image processing.

When the requirement of the subsequent image processing on the preprocessing method changes, the apparatus for performing the subsequent image processing may generate the requirement information, and transmit the requirement information to the image processing apparatus for performing S210 to S220.

Wherein, the preprocessing may be performed on the entire area of each initial image, or only a part of the area may be preprocessed.

Further, it is also possible to preprocess the first region image of the first ROI in at least one region of interest (region of interest, ROI) in the at least one initial image according to the first mapping relationship, so as to obtain the first region of interest. A preprocessed image.

By preprocessing the image located in the first ROI, the amount of data that needs to be preprocessed is reduced, and the time required for preprocessing is reduced. It should be understood that the first ROI is a partial area in the original image.

For the initial image used by the vehicle-mounted camera device, there are differences in the location areas of people, vehicles, lane lines, and traffic signs in the initial image. When the vehicle needs to detect different types of targets, only the images in the ROI corresponding to each type of target in the initial image can be preprocessed.

Optionally, multiple ROIs may correspond to the same preprocessing method. Alternatively, multiple ROIs may have a one-to-one correspondence with multiple preprocessing methods. Alternatively, there may be at least two ROIs in the multiple ROIs corresponding to different preprocessing methods. For each ROI, a preprocessing method corresponding to the ROI can be used to preprocess the image of the region in the ROI.

If the subsequent image processing only needs to process a partial area in the preprocessing result of the initial image, the ROI in the initial image corresponding to the partial area may be determined. Only preprocess the image of the region where the initial image is located in the ROI, which reduces the data that needs to be preprocessed, saves processing resources, and reduces processing time.

If the subsequent image processing puts forward various requirements for image preprocessing, that is, the preprocessing results of the initial image need to be obtained by different preprocessing methods, the number of ROIs can be multiple, and each ROI can correspond to a kind of preprocessing processing needs.

According to the corresponding relationship between the multiple ROIs and the multiple mapping relationships, the mapping relationship corresponding to each ROI can be used to preprocess the image of the region where the at least one initial image is located in the ROI. The first ROI corresponds to the first mapping relationship, and the first region image of the first ROI is preprocessed according to the first mapping relationship.

That is, the image of the second region located in the second ROI in the at least one initial image may be preprocessed by using a mapping relationship corresponding to any one or more second ROIs, or each second ROI, The second ROI is an ROI other than the first ROI in the at least one ROI.

It should be understood that, for each second ROI, using the second mapping relationship corresponding to the second ROI to preprocess at least one second region image where the initial image is located in the second ROI, a second preprocessed image can be obtained. The second mapping relationship exists between the pixel positions of the at least one initial image and the second preprocessed image.

It should be understood that each ROI may correspond to a mapping relationship. Each ROI can be preprocessed by using the mapping relationship between the original image and the pixel positions in the preprocessed image. When each ROI needs to be preprocessed by multiple preprocessing methods, the images located in each ROI are preprocessed by mapping, which avoids the storage of intermediate data generated by each preprocessing method and reduces the consumption of storage resources.

It should be understood that each ROI may include only one continuous area, or may be separated from each other by multiple continuous areas. When the same preprocessing method is used for multiple ROIs, there may be no overlap between the multiple ROIs, which avoids repeated processing of the same region and improves the image processing efficiency. When the multiple ROIs adopt different preprocessing methods, the multiple ROIs may or may not overlap.

The preprocessing of the region images in multiple ROIs can be performed in a serial or parallel manner. Preprocessing the region images in multiple ROIs in parallel can reduce the time required for image preprocessing.

The initial image may come from an onboard camera. It can be captured by a vehicle-mounted camera. That is to say, steps S210 to S220 may be used to preprocess the initial image collected by the vehicle-mounted camera device. Further processing of the preprocessed image obtained in S220 may be used to realize functions such as image recognition and prediction of automatic driving/assisted driving.

The at least one ROI may be determined according to the driving state of the vehicle, and the vehicle-mounted camera device used for initial image acquisition is set on the vehicle. Compared with the situation where the vehicle is driving horizontally, when the vehicle is uphill, downhill or bumpy, the angle at which the vehicle camera captures the initial image changes, and the area that needs to be detected by various types of targets changes.

Different vehicle driving states may correspond to different positions of the ROI.

Driving state information can be acquired, and the driving state information is used to indicate the driving state information of the vehicle. The position of the first ROI may be determined according to the driving state information. For example, different driving state information may correspond to different positions of the first ROI.

It should be understood that the first mapping relationship can be used to map each pixel in the original image. When preprocessing the first region image located in the first ROI in the at least one initial image, only each pixel in the first region image is mapped.

Alternatively, the first mapping relationship may only be used to map each pixel in the first ROI. When the driving state of the vehicle changes, a new first mapping relationship may be generated according to the demand information.

For at least one initial image, preprocessing can be performed in a serial or parallel manner.

Performing the preprocessing in a serial manner can be understood as processing each pixel in the at least one initial image in sequence. For each pixel to be processed in the at least one initial image, the first mapping relationship may be used to sequentially determine the position of the pixel to be processed in the first preprocessed image, thereby generating a first preset image. The pixel to be processed may be a pixel in at least one ROI.

Performing preprocessing by means of parallel processing can be understood as processing multiple pixels in at least one initial image at the same time. The positions of the plurality of pixels to be processed in the at least one initial image in the first preprocessed image can be determined at the same time by using the first mapping relationship.

By preprocessing at least one initial image in parallel, the time for image preprocessing can be reduced and the processing rate can be improved. In order to implement a parallel preprocessing process, at least one initial image may be divided. Each initial image is stored in memory in some format. Dividing at least one initial image into a plurality of image blocks can be understood as storing each image block in this format. That is, when at least one initial image is divided, the at least one initial image needs to be copied, and the processing process is relatively complicated.

Alternatively, the first mapping relationship may include multiple sub-mapping relationships, and each sub-mapping relationship corresponds to a sub-region. That is, each sub-mapping relationship is used for the mapping relationship of pixel positions in a sub-region.

The initial images in the multiple sub-regions may be preprocessed in parallel according to the multiple sub-mapping relationships. Therefore, while parallel processing is performed to reduce the time of image preprocessing, the division of at least one initial image is avoided, so that the parallel preprocessing of the initial image is more convenient.

It should be understood that preprocessing can also be performed in a parallel manner for the region images in which at least one initial image is located in each ROI. For example, for the first ROI, the first mapping relationship may include a plurality of sub-mapping relationships, and each sub-mapping relationship corresponds to a sub-region in the first ROI. Thus, images in which at least one initial image is located in multiple sub-regions in the first ROI can be preprocessed in parallel according to the multiple sub-mapping relationships.

The image processing methods provided in the embodiments of the present application may be applied in the field of automatic driving or assisted driving. Specifically, reference may be made to the descriptions of FIG. 4 to FIG. 6 . The initial image may be captured by an onboard camera.

The image processing methods provided in the embodiments of the present application can also be applied to scenarios such as security monitoring, VR equipment, and projection equipment that require image processing through various preprocessing methods.

In a security monitoring scenario, the initial images collected by multiple cameras for collecting monitoring images need to be preprocessed. The cameras installed in supermarkets are mainly used for centralized monitoring of supermarket entrances, cashiers and other locations. The initial images collected by all security cameras can be uniformly preprocessed. The preprocessing for LDC and IPM is performed in parallel for multiple ROI regions in the initial images collected by multiple cameras, and then the preprocessed images of the multiple ROI regions are stitched and displayed for viewing by relevant personnel.

By preprocessing multiple ROI regions in the initial images collected by multiple cameras in parallel, the processing efficiency can be improved. Compared with the way that multiple processors preprocess the initial image collected by one camera, one processor preprocesses multiple ROI regions uniformly, which can avoid the preprocessing caused by the different processing capabilities of different processors. The moments corresponding to the images of different regions in the image after stitching the initial image are not exactly the same.

For VR equipment and projection equipment, it is generally necessary to perform various preprocessing methods such as resolution adjustment and radiation change on the acquired initial image. Using S210 to S220 to preprocess the initial image can avoid the storage of the intermediate data generated by each preprocessing method, and reduce the occupation of storage resources.

The image processing system 300 can be applied to the fields of automatic driving, assisted driving, and the like. Image processing system 300 may be located in a vehicle.

The image processing system 300 includes a first processing module 310 and a second processing module 320 .

The first processing module 310 receives the initial image collected by the camera.

The camera device used to capture the initial image may be a vehicle-mounted camera. FIG. 4 takes the initial image collected by the front-view camera installed on the vehicle as an example for description. The forward-looking camera performs image acquisition in real time to obtain the initial image with distortion.

The positions of the multiple ROIs can be determined according to the driving state of the vehicle.

After acquiring the initial image, the first processing module 310 may determine a region image located in each of the multiple ROIs in the initial image. For the region image in each ROI, it is used for preprocessing using the mapping relationship of the pixel positions corresponding to the ROI, so that the position of each pixel in the ROI in the preprocessing image as the preprocessing result can be determined. Also, the preprocessing of the region images in multiple ROIs can be performed in parallel.

After the first processing module 310 processes the initial image, a preprocessed image corresponding to each ROI can be obtained.

The preprocessing of the first processing module 310 may implement one or more functions of affine transformation, inverse perspective transformation, LDC, resolution adjustment, etc. on the image.

As shown in FIG. 4 , after the first processing module 310 preprocesses the initial image collected by the front-view camera, three preprocessed images can be obtained, which are image 1 to image 3 respectively.

There are differences in the location areas of objects such as people, vehicles, lane lines, and traffic signs in the initial image. For example, for the initial image collected by the front-view camera (such as the initial image shown in Figure 4), the traffic sign is generally located in the upper area of the initial image, the vehicle is generally located in the middle area of the preprocessed image, and the lane line is generally located in The lower region of the preprocessed image.

The ROI corresponding to image 1 is the area where objects such as people and vehicles generally appear in the initial image; the ROI corresponding to image 2 is the area where objects such as lane lines generally appear in the initial image; the ROI corresponding to image 3 is the initial image Medium traffic signs identify areas where such objects typically occur.

When the second processing module 320 processes different types of objects, the required image precision is different. For example, for the image of the area where the targets such as people and vehicles are located, a higher resolution is required to achieve more accurate vehicle driving planning; for the image of the area where the target such as the lane line is located, the lower resolution image is used. Lane detection can be achieved.

Each ROI may correspond to different requirements of the second processing module 320 for the image.

Image 1 implements LDC and resolution enhancement, that is to say, the preprocessing method that can be used to achieve LDC and resolution enhancement using the mapping relationship corresponding to image 1 is LDC and resolution enhancement. The preprocessing methods that can be used to implement the mapping relationship corresponding to image 2 are LDC and resolution reduction. The preprocessing method that can be implemented by using the mapping relationship corresponding to image 3 is LDC.

The second processing module 320 is used for further processing the preprocessed image. The second processing module 320 may use a neural network or a traditional computer vision method to perform functions such as target detection, target classification, ranging, etc. on the preprocessed image.

The first processing module 310 may be located in an image signal processor (image signal processor, ISP). The ISP may be a processor in an image acquisition device such as a camera.

The second processing module 320 may use one or more processors from a central processing unit (CPU), a digital signal processor (DSP), a graphics processing unit (GPU), and the like .

The mapping relationship may be stored in the memory in the form of a mapping relationship table. The first processing module 310 may use the mapping relationship table when processing the initial image. Alternatively, the mapping relationship can also be in the form of a mapping function or any other corresponding relationship that can represent pixel positions.

The mapping relationship corresponding to each ROI can be determined according to the mapping relationship of each preprocessing mode required for subsequent image processing of the ROI, or it can also be determined according to the parameters of the matrix operation corresponding to each preprocessing mode .

When the subsequent image processing of a certain ROI requires multiple preprocessing methods, use the mapping relationship between the initial image and the pixel position in the preprocessed image to process the initial image to generate a preprocessed image without generating Intermediate results, reducing the amount of data generated, saving storage resources and processing resources.

In the case of using matrix operations or other operations to implement each operation mode in turn, in order to avoid interference to the initial image during the preprocessing process of one combination, thereby affecting the preprocessing corresponding to other combinations, each combination can be corresponding to The original image is copied, but the copy of the original image occupies more resources.

When preprocessing is performed in a mapping manner, there is no need to copy the initial image, which reduces the occupation of storage resources.

The pixel position mapping relationship can store the coordinates of each pixel of the initial image in the preprocessed image. The first processing module 310 can determine the preprocessed image by using the mapping relationship table and the bilinear interpolation equal difference algorithm. Specifically, please refer to the description of FIG. 5 .

It should be understood that after the camera captures the initial image, the initial image may be stored in the memory corresponding to the first processing module 310 or in other memory. The preprocessed image generated by the first processing module 310 may be stored in the memory corresponding to the first processing module 310, the memory corresponding to the second processing module 320, or other storages.

In the process of image preprocessing, each pixel in the preprocessed image can be understood as coming from the pixels in the initial image before preprocessing, that is, there is a correspondence between the pixel positions in the initial image and the preprocessed image.

Taking the camera distortion correction as an example, each pixel on the picture after distortion correction is calculated by the pixel on the distortion map captured by the camera through the internal parameters of the camera and the distortion model. The specific relationship can be expressed as:

(x',y')=F(x,y)

Among them, (x, y) is used to represent the coordinates of a pixel in the initial image, (x', y') can be understood as the pixel coordinates in the preprocessed image, and the function F is used to represent the mapping relationship. That is, the pixel (x',y') in the preprocessed image is the same color as the pixel (x,y) in the original image. When x, y are integers, x' and/or y' may not be integers. After determining the pixel coordinates of each pixel in the initial image in the preprocessed image, the color corresponding to the pixel represented by each integer coordinate in the preprocessed image can be determined through a bilinear interpolation and equal difference algorithm.

The corresponding relationship between the pixels in the initial image and the preprocessed image can be expressed as a mapping relationship table from the initial image to the preprocessed image. Processor resources such as ISP, GPU, CPU, and DSP can use the mapping relationship F to generate preprocessed images. The mapping relationship F may be a mapping relationship table, that is, the preprocessed image may be obtained by looking up the table.

An initial image contains a large number of pixels (such as an image with a resolution of 1920 × 1080, including 2073600 pixels), and the way of querying the mapping table for each pixel in the initial image to generate a preprocessed image still requires a relatively large amount of space. long processing time.

As shown in Figure 5, multiple regions of the initial graph can be processed in parallel. FIG. 5 illustrates by taking an example of increasing the resolution of the image (which can also be understood as the enlargement of the image).

The mapping relationship table may include four sub-mapping relationships, each of which corresponds to one of the four regions A1 to A4 in the initial image shown in FIG. 5 . Therefore, when the preprocessed image is generated, the regions A1 to A4 can be mapped in parallel by using each sub-mapping relationship in parallel, and the images of the regions B1 to B4 in the preprocessed image can be generated respectively.

For more complex image preprocessing operations, the preprocessing method can also be performed by using multiple sub-mapping relationships in the mapping relationship table to perform mapping in parallel when looking up the mapping relationship table. In the process of generating the preprocessed image, if the pixel coordinates in the preprocessed image indicated by the mapping relationship table are not integers, the preprocessed image may be generated by means of image interpolation.

There are various formats for storing images in memory, and Figure 6 illustrates them in YUV format. Each pixel in the image consists of three types of data: Y, U, and V. Among them, Y represents the brightness (luminance or luma), that is, the grayscale value; "U" and "V" represent the chrominance (chrominance or chroma), which is used to describe the color and saturation of the image, which is used to indicate the pixel. s color.

For an image stored in the memory in the YUV format, the Y data of each pixel in the image in the memory is arranged in the order of the pixels. After the Y data of all pixels in the image, the U data and V data of each pixel are adjacent and are arranged in the memory in the order of the pixels. That is, after the Y data of all pixels of the image in memory, the U data and V data of the image are interleaved.

If the initial image in YUV format is divided into image blocks to realize parallel preprocessing of the initial image, each image block needs to be stored as data in YUV format, and then each image block is preprocessed.

When the preprocessing method with multiple sub-mappings in parallel is adopted, according to the YUV format, the storage location in the memory of the image data whose initial image is located in the region corresponding to each sub-mapping relationship can be determined in parallel, thereby realizing parallel image preprocessing.

As shown in FIG. 7 , a mapping relationship table used to realize the preprocessing of improving the overall resolution of the initial image shown in FIG. 5 is shown.

Using the serial image preprocessing method can be understood as querying the mapping table according to the order of each pixel in the initial image to determine the position of each pixel in the initial pixel in the preprocessing image. After that, according to the order of the pixels in the preprocessed image, the color of each pixel is calculated by means of the difference value.

By adopting the parallel image preprocessing method, the mapping relationship table can be queried for the image pixels of the initial image located in the region corresponding to each sub-mapping relationship at the same time.

For the preprocessing shown in Figure 5 for improving the overall resolution of the initial image, four sub-mapping relationships are used to determine the corresponding pixel coordinates of the initial image in the image corresponding to each sub-mapping relationship in the preprocessed image. coordinate. The four sub-mapping relationships correspond to the areas A1 to A4, respectively. After that, using the Y data and U, V data of each area, the color at the position where each pixel coordinate is an integer is calculated for each area by means of difference, so that the areas B1 to B4 in the preprocessed image can be generated in parallel Image.

By dividing the mapping relationship of pixel positions into multiple sub-maps, a reasonable area division of the initial image can be realized. By using the preprocessing method of parallel processing, the time required for image preprocessing can be saved, and the complexity of image preprocessing can be reduced. Computational resources to improve the efficiency of image preprocessing.

By using the image processing method provided by the embodiment of the present application, the image processing process shown in FIG. 2 can be optimized.

When the vehicle needs to park sideways, the ROI area in the initial image captured by the four-way fisheye camera can be determined, and the ROI area corresponds to the image used to represent the environmental situation on the right side of the vehicle in the ring view. Look up the mapping table for the images in the ROI area to generate the preprocessed right environment image. The mapping relationship table is used to represent the corresponding relationship between each pixel in the initial image captured by the four-way fisheye camera and each pixel in the ring view.

On the one hand, only the image of the ROI area in the initial image is preprocessed, which reduces the amount of data that needs to be preprocessed and improves the processing efficiency. On the other hand, the preprocessed image is obtained through the mapping table, without the need to perform various operations such as lens distortion correction, inverse perspective transformation, etc., which improves the processing efficiency, avoids the generation and storage of intermediate data, and reduces the processing resources and resources. Occupation of storage resources.

The image processing method of the embodiment of the present application is described above with reference to FIG. 1 to FIG. 7 , and the image processing apparatus of the embodiment of the present application is described below with reference to FIG. 8 to FIG. 9 . It should be understood that the description of the image processing apparatus and the description of the image processing method correspond to each other, and therefore, for the parts not described in detail, reference may be made to the foregoing description of the image processing method.

The image processing apparatus 200 includes an acquisition module 2010 and a processing module 2020 .

The acquiring module 2010 is used for acquiring at least one initial image.

The processing module 2020 is configured to obtain a pre-processed first pre-processed image according to the at least one initial image and the first mapping relationship, and there is a difference between the pixel positions of the at least one initial image and the first pre-processed image A first mapping relationship, where the first mapping relationship corresponds to multiple preprocessing modes.

Optionally, the processing module 2020 is specifically configured to, according to the first mapping relationship, preprocess the first region image of the first ROI in the at least one region of interest ROI in the at least one initial image, so as to obtain the The first preprocessed image.

Optionally, the number of the at least one ROI is multiple, and the first ROI corresponds to the first mapping relationship.

The processing module 2020 is further configured to preprocess the image of the second region located in the second ROI in the at least one initial image by using a mapping relationship corresponding to any one or more or each of the second ROIs, where The second ROI is an ROI other than the first ROI in the at least one ROI.

Optionally, the at least one initial image is from a vehicle-mounted camera, and the position of each of the ROIs is obtained according to the driving state of the vehicle.

Optionally, the first mapping relationship includes a plurality of sub-mapping relationships, and each of the sub-mapping relationships corresponds to a sub-region in the first ROI.

The processing module 2020 is specifically configured to, according to the multiple sub-mapping relationships, perform parallel preprocessing on the images located in each of the sub-regions in the at least one initial image.

Optionally, the first mapping relationship includes a plurality of sub-mapping relationships, and each of the sub-mapping relationships corresponds to a sub-region.

Optionally, the multiple preprocessing manners include at least one of affine transformation, inverse perspective transformation IPM, lens distortion correction LDC, and geometric transformation.

Optionally, the obtaining module 2010 is further configured to obtain demand information, where the demand information is used to indicate the multiple preprocessing methods;

The processing module 2020 is further configured to generate the first mapping relationship according to the requirement information.

FIG. 9 is a schematic structural diagram of an image processing apparatus provided by an embodiment of the present application.

The image processing apparatus 3000 includes at least one memory 3010 and at least one processor 3020, the at least one memory 3010 is used for storing a program, and the at least one processor 3020 is used for running the program to implement the aforementioned method.

Specifically, the processor 3020 is configured to acquire at least one initial image.

The processor 3020 is further configured to obtain a preprocessed first preprocessed image according to the at least one initial image and the first mapping relationship, and the pixel positions between the at least one initial image and the first preprocessed image are There is a first mapping relationship, and the first mapping relationship corresponds to multiple preprocessing modes.

Optionally, the processor 3020 is specifically configured to, according to the first mapping relationship, preprocess the first region image of the first ROI in the at least one region of interest ROI in the at least one initial image, so as to obtain the The first preprocessed image.

The processor 3020 is further configured to preprocess the image of the second region located in the second ROI in the at least one initial image by using a mapping relationship corresponding to any one or more or each of the second ROIs, where The second ROI is an ROI other than the first ROI in the at least one ROI.

Optionally, the at least one initial image is from a vehicle-mounted camera device, the position of each of the ROIs is obtained according to the driving state of the vehicle, and the vehicle-mounted camera device is provided on the vehicle.

The processor 3020 is specifically configured to, according to the multiple sub-mapping relationships, perform parallel preprocessing on images located in each of the sub-regions in the at least one initial image.

The processor 3020 is specifically configured to, according to the multiple sub-mapping relationships, perform parallel preprocessing on images located in each of the sub-regions in the at least one initial image

Optionally, the processor 3020 is further configured to acquire demand information, where the demand information is used to indicate the multiple preprocessing methods

The processor 3020 is further configured to generate the first mapping relationship according to the requirement information.

It should be understood that the processor in the embodiment of the present application may be a central processing unit (central processing unit, CPU), and the processor may also be other general-purpose processors, digital signal processors (digital signal processors, DSP), application-specific integrated circuits (application specific integrated circuit, ASIC), off-the-shelf programmable gate array (field programmable gate array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

It should also be understood that the memory in the embodiments of the present application may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically programmable Erase programmable read-only memory (electrically EPROM, EEPROM) or flash memory. Volatile memory may be random access memory (RAM), which acts as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (DRAM) Access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous connection dynamic random access memory Fetch memory (synchlink DRAM, SLDRAM) and direct memory bus random access memory (direct rambus RAM, DR RAM).

The descriptions of the processes corresponding to the above figures have their own emphasis, and for parts that are not described in detail in a certain process, please refer to the relevant descriptions of other processes.

Embodiments of the present application further provide a computer-readable storage medium, characterized in that, the computer-readable storage medium has program instructions, and when the program instructions are directly or indirectly executed, the foregoing method can be implemented.

Embodiments of the present application also provide a computer program product containing instructions, which, when run on a computing device, cause the computing device to execute the foregoing method, or cause the computing device to implement the functions of the foregoing apparatus.

An embodiment of the present application further provides a chip system, characterized in that, the chip system includes at least one processor, and when a program instruction is executed in the at least one processor, the foregoing method can be implemented.

An embodiment of the present application further provides a terminal, including the image processing apparatus described above.

The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented in software, the above-described embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer may be a general purpose computer, special purpose computer, computer network, or other programmable device. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be downloaded from a website site, computer, server, or data center Transmission to another website site, computer, server or data center by wire (eg, infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, a data center, or the like that contains one or more sets of available media. The usable media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, DVDs), or semiconductor media. The semiconductor medium may be a solid state drive.

It should be understood that the term "and/or" in this document is only an association relationship to describe associated objects, indicating that there can be three kinds of relationships, for example, A and/or B, which can mean that A exists alone, and A and B exist at the same time , there are three cases of B alone, where A and B can be singular or plural. In addition, the character "/" in this document generally indicates that the related objects before and after are an "or" relationship, but may also indicate an "and/or" relationship, which can be understood with reference to the context.

In this application, "at least one" means one or more, and "plurality" means two or more. "At least one item(s) below" or similar expressions thereof refer to any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one (a) of a, b or c may represent: a, b, c, a-b, a-c, b-c or a-b-c, wherein a, b, and c may be single or multiple.

It should be understood that, in various embodiments of the present application, the size of the sequence numbers of the above-mentioned processes does not mean the sequence of execution, and the execution sequence of each process should be determined by its functions and internal logic, and should not be dealt with in the embodiments of the present application. implementation constitutes any limitation.

Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of this application.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described systems, devices and units may refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components shown as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as independent products, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence, or the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, removable hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk and other media that can store program codes .

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited to this. should be covered within the scope of protection of this application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

An image processing method, characterized in that the method comprises:

Get at least one initial image;

According to the at least one initial image and the first mapping relationship, a preprocessed first preprocessed image is obtained, and there is a first mapping relationship between the pixel positions of the at least one initial image and the first preprocessed image, so The first mapping relationship described above corresponds to multiple preprocessing methods.
The method of claim 1, wherein:

The obtaining a preprocessed first preprocessed image according to the at least one initial image and the first mapping relationship includes: according to the first mapping relationship, performing a ROI on at least one region of interest in the at least one initial image The first region image of the first ROI is preprocessed to obtain the first preprocessed image.
The method according to claim 2, wherein the number of the at least one ROI is multiple, and the first ROI corresponds to the first mapping relationship,

The method also includes:

The second region image located in the second ROI in the at least one initial image is preprocessed by using the mapping relationship corresponding to the second ROI, where the second ROI is the first ROI in the at least one ROI outside ROI.
The method according to claim 2 or 3, wherein the at least one initial image is from a vehicle-mounted camera device, the position of the ROI is obtained according to the driving state of the vehicle, and the vehicle-mounted camera device is provided on the vehicle superior.
The method according to any one of claims 2-4, wherein the first mapping relationship includes a plurality of sub-mapping relationships, and each of the sub-mapping relationships corresponds to a sub-region in the first ROI ,

The preprocessing of the first region image located in the first ROI in the at least one region of interest ROI in the at least one initial image according to the first mapping relationship includes:

According to the plurality of sub-mapping relationships, the images located in each of the sub-regions in the at least one initial image are preprocessed in parallel.
The method according to claim 1, wherein the first mapping relationship includes a plurality of sub-mapping relationships, and each of the sub-mapping relationships corresponds to a sub-region,

The obtaining the preprocessed first preprocessed image according to the at least one initial image and the first mapping relationship includes: according to the plurality of sub-mapping relationships, for each of the sub-maps in the at least one initial image. The images of the regions are preprocessed in parallel.
The method according to any one of claims 1-6, wherein the multiple preprocessing methods include at least one of affine transformation, inverse perspective transformation IPM, lens distortion correction LDC, and geometric transformation.
The method according to any one of claims 1-7, wherein the method further comprises:

acquiring demand information, where the demand information is used to indicate the multiple preprocessing methods;

The first mapping relationship is generated according to the requirement information.
An image processing device, comprising: an acquisition module and a processing module;

The obtaining module is used to obtain at least one initial image;

The processing module is configured to, according to the at least one initial image and the first mapping relationship, obtain a preprocessed first preprocessed image, and the pixel positions between the at least one initial image and the first preprocessed image are There is a first mapping relationship, and the first mapping relationship corresponds to multiple preprocessing modes.
The device of claim 9, wherein:

The processing module is specifically configured to, according to the first mapping relationship, preprocess the first region image of the first ROI in the at least one region of interest ROI in the at least one initial image, so as to obtain the first preliminary image. Process images.
The device according to claim 10, wherein the number of the at least one ROI is multiple, and the first ROI corresponds to the first mapping relationship,

The processing module is further configured to use a mapping relationship corresponding to a second ROI to preprocess an image of a second region located in the second ROI in the at least one initial image, where the second ROI is the at least one of the at least one initial image. An ROI other than the first ROI in one ROI.
The device according to claim 10 or 11, wherein the at least one initial image is from a vehicle-mounted camera device, and the position of the ROI is obtained according to the driving state of the vehicle.
The apparatus according to any one of claims 10-12, wherein the first mapping relationship includes a plurality of sub-mapping relationships, and each sub-mapping relationship corresponds to a sub-region in the first ROI ,

The processing module is specifically configured to, according to the multiple sub-mapping relationships, perform parallel preprocessing on images located in each of the sub-regions in the at least one initial image.
The device according to claim 9, wherein the first mapping relationship includes a plurality of sub-mapping relationships, and each of the sub-mapping relationships corresponds to a sub-region,

The processing module is specifically configured to, according to the multiple sub-mapping relationships, perform parallel preprocessing on images located in each of the sub-regions in the at least one initial image.
The apparatus according to any one of claims 9-14, wherein the multiple preprocessing methods include at least one of affine transformation, inverse perspective transformation IPM, lens distortion correction LDC, and geometric transformation.
The device according to any one of claims 9-15, characterized in that,

The obtaining module is further configured to obtain demand information, where the demand information is used to indicate the multiple preprocessing methods;

The processing module is further configured to generate the first mapping relationship according to the requirement information.
An image processing device, characterized in that it comprises at least one memory and at least one processor, the at least one memory is used for storing a program, and the at least one processor is used for running the program, so as to realize the claims 1-8 The method of any one.
A chip, characterized by comprising at least one processor and an interface circuit, wherein the interface circuit is configured to provide program instructions or data for the at least one processor, and the at least one processor is configured to execute the program instructions to Implementing the method of any of claims 1-8.
A computer-readable storage medium, characterized in that the computer-readable medium stores program codes for device execution, and when the program codes are executed by the device, the implementation of any one of claims 1-8 Methods.