WO2022140896A1

WO2022140896A1 - Multi-field-of-view image acquisition method and apparatus, and computer device and storage medium

Info

Publication number: WO2022140896A1
Application number: PCT/CN2020/140031
Authority: WO
Inventors: 杨超
Original assignee: 深圳元戎启行科技有限公司
Priority date: 2020-12-28
Filing date: 2020-12-28
Publication date: 2022-07-07
Also published as: CN116097659A

Abstract

A multi-field-of-view image acquisition method, comprising: during autonomous driving, acquiring a driving environment according to an original field of view of a camera device to obtain a driving environment image; inputting the driving environment image into a first image processing path and a second image processing path; capturing at least one target field-of-view image from the driving environment image by means of the first image processing path according to a target field of view; down-sampling the driving environment image by means of the second image processing path to obtain a down-sampled driving environment image of the original field of view; and using the extracted target field-of-view image and the down-sampled driving environment image as a multi-field-of-view image.

Description

Image acquisition method, device, computer equipment and storage medium with multiple viewing angles

technical field

The present application relates to an image generation method, apparatus, computer equipment and storage medium.

Background technique

With the development of computer technology, autonomous driving technology has emerged. Automatic driving technology needs to obtain environmental information through camera equipment, so the camera equipment is required to collect images through multiple different field of view angles, so that the collected images cover a large field of view range and can clearly distinguish distant objects.

However, the inventor realized that by using a plurality of image sensors with different field of view, although images with different field of view can also be obtained, the number of image sensors is large, which requires extra space of the camera equipment.

SUMMARY OF THE INVENTION

According to various embodiments disclosed in the present application, an image acquisition method, apparatus, computer device, and storage medium with multiple viewing angles are provided.

An image acquisition method with multiple viewing angles, executed by a camera device, includes:

During the automatic driving process, the driving environment is collected according to the original field of view of the camera device to obtain a driving environment image;

inputting the driving environment image into the first image processing path and the second image processing path respectively;

through the first image processing path, intercepting at least one target field of view image in the driving environment image according to the target field of view;

Through the second image processing path, down-sampling the driving environment image to obtain the down-sampled driving environment image of the original field of view;

The extracted target field of view image and the down-sampled driving environment image are used as a multi-field of view image.

An image acquisition device with multiple viewing angles, comprising:

an image acquisition module, used for acquiring the driving environment image according to the original field of view of the camera device during the automatic driving process;

an input module for inputting the driving environment image into the first image processing path and the second image processing path respectively;

an interception module, configured to intercept at least one target field of view image in the driving environment image according to the target field of view through the first image processing path;

A downsampling module, configured to downsample the driving environment image through the second image processing path to obtain a downsampled driving environment image of the original field of view;

The multi-view-angle image acquisition module is configured to use the extracted target-view-angle image and the down-sampled driving environment image as a multi-view-angle image.

A computer device includes a memory and one or more processors, where computer-readable instructions are stored in the memory, and when the computer-readable instructions are executed by the processor, implement the steps of the image acquisition method with multiple viewing angles.

One or more non-volatile storage media storing computer-readable instructions, when the computer-readable instructions are executed by one or more processors, the one or more processors implement the image acquisition method with multiple viewing angles A step of.

The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below. Other features and advantages of the present application will be apparent from the description, drawings, and claims.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without any creative effort.

FIG. 1 is an application scenario diagram of an image acquisition method with multiple viewing angles according to one or more embodiments.

FIG. 2 is a schematic flowchart of an image acquisition method with multiple viewing angles according to one or more embodiments.

3a is a schematic diagram of a horizontal field of view in accordance with one or more embodiments.

3b is a schematic diagram of a vertical field of view in accordance with one or more embodiments.

FIG. 4 is a schematic flowchart of a method for capturing a driving environment image according to one or more embodiments.

FIG. 5 is a schematic diagram of moving an image capture area in accordance with one or more embodiments.

FIG. 6 is a schematic flowchart of a method for capturing an image of a second target field of view according to one or more embodiments.

FIG. 7a is a schematic diagram of a target image cropping area according to one or more embodiments.

FIG. 7b is a schematic diagram of a target image clipping region in another embodiment.

FIG. 8 is a schematic flowchart of image processing of a driving environment image according to one or more embodiments.

FIG. 9 is a block diagram of an image acquisition apparatus with multiple viewing angles in accordance with one or more embodiments.

FIG. 10 is a block diagram of an image acquisition device with multiple viewing angles in another embodiment.

11 is a block diagram of a computer device in accordance with one or more embodiments.

Detailed ways

In order to make the technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

The image acquisition method with multiple viewing angles provided by the present application can be applied to the application environment shown in FIG. 1 . During the automatic driving process, the camera device 120 mounted on the automatic driving tool 110 collects the driving environment according to the original field of view, obtains the driving environment image, and inputs the driving environment image into the first image processing path and the second image processing path respectively. The camera device 120 intercepts at least one target field of view image in the driving environment image according to the target field of view through the first image processing path, and downsamples the driving environment image through the second image processing path to obtain the original field of view image. Downsampled driving environment images. The imaging apparatus 120 takes the extracted target view angle image and the down-sampled driving environment image as a multi-view angle image. The imaging device 120 may be, but is not limited to, various cameras, video cameras, in-vehicle cameras, and the like.

In one of the embodiments, as shown in Figure 2, a method for acquiring images with multiple viewing angles is provided, and the method is applied to the imaging device in Figure 1 as an example to illustrate, including the following steps:

Step 202: During the automatic driving process, the driving environment is collected according to the original field of view of the camera device to obtain a driving environment image.

Automatic driving is a driving method in which a motor vehicle controls the vehicle through a preset control program without any human operation, so that the motor vehicle can drive safely on the road. The camera device is a device that collects light signals in the driving environment through image sensors and converts the collected light signals into electrical signals. The image sensor in the camera device has a specific field of view, and the larger the field of view, the larger the range of the captured driving environment.

Taking the focus of the camera device as the vertex, two straight lines are formed from the vertex to the boundary of the driving environment that can be captured by the camera device, and the field of view is the angle between the two straight lines. The field of view determines the field of view of the camera device. The larger the field of view, the larger the range of the driving environment that the camera device can capture, and the more objects appear in the driving environment image. The original field of view is the default field of view of the image sensor in the camera device when it leaves the factory, or the field of view that is set by the user before capturing images.

Step 204, the imaging device inputs the driving environment image into the first image processing path and the second image processing path respectively.

The first image processing path and the second image processing path are hardware paths for processing driving environment images. For example, the first image processing path and the second image processing path may be two different hardware processing paths, eg, two different hardware chips. Alternatively, different processing paths formed by time-division multiplexing of one hardware processing path may also be performed, and the driving environment image may be processed according to different image processing methods in different time periods.

The imaging device inputs the driving environment image into the first image processing path and the second image processing path respectively, so as to process the driving environment image respectively through the hardware circuit.

Step 206 , the imaging device intercepts at least one target field of view image according to the target field of view in the driving environment image through the first image processing path.

Clipping is an operation process of extracting part of an image from an image according to the image clipping area. When capturing an image, the imaging device does not change the resolution of the image, nor does it change the pixel value of the pixels in the image.

The target field of view is the field of view set by the computer equipment to narrow the range of the field of view. The target field of view image is the image captured by the camera device from the driving environment image according to the target field of view, which is equivalent to the image obtained by the camera device by collecting the driving environment according to the target field of view.

After the camera equipment obtains the driving environment image, since the driving environment image is collected according to the original field of view, the collected field of view has a larger range. The driving environment image is intercepted, and at least one target field of view image is intercepted. Since the target field of view image captured by the camera device is a part of the driving environment image, and the amount of data is smaller than that of the driving environment image, the amount of calculation is small when performing image recognition on the target field of view image.

Step 208 , the imaging device down-samples the driving environment image through the second image processing channel to obtain the down-sampled driving environment image of the original field of view.

Downsampling is the process of reducing the number of sample points in an image. For an N×M image, if the downsampling coefficient is k, from each row and each column of pixels in the driving environment image, one pixel is taken every k pixels to form a downsampling driving environment image. The process of image downsampling does not change the field of view of the original image, but reduces the resolution of the image, thereby reducing the amount of data in the image.

In one embodiment, the camera device outputs the down-sampled driving environment image to the image recognition unit, so that the image recognition unit recognizes objects in the driving environment image, so as to control the automatic driving vehicle according to the recognized objects. For example, if a zebra crossing is recognized from a driving environment image, the vehicle that controls the autonomous driving suspends or slows down.

Step 210 , the imaging device uses the extracted target field of view image and the down-sampled driving environment image as a multi-field of view image.

The field of view of the target field of view image is smaller than the original field of view, and the field of view of the down-sampled driving environment image is the same as the original field of view. The camera device obtains multiple images with different field of view from only one image of the driving environment collected with the original field of view.

In the above image acquisition method with multiple viewing angles, the camera device first collects the driving environment according to the original viewing angle, and obtains the driving environment image with the largest viewing range. Then, the driving environment image is intercepted through the first image processing path to obtain the target field of view image whose field of view is smaller than the original field of view; the driving environment image is downsampled through the second image processing path, and the original field of view is obtained. Corner downsampled images of the driving environment. Finally, the camera device only needs one image sensor, and can obtain multiple images with different field of view through the driving environment image collected by the image sensor with the original field of view. The cost of acquiring multiple images with different fields of view is reduced, and the size of the camera equipment is also reduced since only one image sensor is used.

In one embodiment, the target angle of view includes a target horizontal angle of view and a target vertical angle of view; the original angle of view includes an original horizontal angle of view and an original vertical angle of view; in the driving environment image, according to the target field of view The angle interception of at least one target field of view angle image includes: calculating the length of the image interception according to the length calculation formula; the length calculation formula is:

Among them, l is the image interception length, _l ′ is the length of the driving environment image, α1 is the original horizontal field of view, and β1 is the target horizontal field of view _; the image interception width is calculated according to the width calculation formula; the width calculation formula is:

Wherein, w is the image interception width, w′ is the width of the driving environment image, _α2 is the original vertical field of view, and _β2 is the target vertical field of view; at least one image interception area is determined according to the image interception length and the image interception width; In the driving environment image, the target field of view image is captured according to the determined image capturing area.

As shown in Fig. 3a, the horizontal field of view is an angle between two straight lines determined according to the boundary of the driving environment that can be captured by the camera device in the horizontal direction and the focus of the camera device. As shown in Fig. 3b, the vertical field of view is the angle between two straight lines determined according to the boundary of the driving environment that can be captured by the camera device in the vertical direction and the focus of the camera device.

In one of the embodiments, the imaging device determines a rectangular area according to the length and width of image capture, and then captures the image of the target field of view according to the rectangular area in the driving environment image.

In one of the embodiments, as shown in FIG. 4 , in the driving environment image, the camera device intercepts the target field of view image according to the determined image interception area, including the following steps:

S402 , the imaging device determines a starting point for capturing in the driving environment image.

S404, the imaging device intercepts the target field of view image in the driving environment image according to the image interception area at the interception starting point.

S406 , the imaging device gradually moves the image capturing area at the capturing starting point according to the preset moving step, and captures the target field of view image in the driving environment image according to the moved image capturing area after each movement.

The interception starting point is a pixel point selected by the camera device from the driving environment image. The imaging device may take the interception starting point as the center of the image interception area, and intercept the target field of view image in the driving environment image according to the image interception area. Alternatively, the imaging device may take the interception starting point as the corner of the image interception area, and intercept the target field of view image in the driving environment image according to the image interception area.

The preset moving step is the step that the camera device moves to the interception starting point each time. The step-by-step movement of the image capture area by the camera device means that the camera device translates the image capture area according to a preset moving step size. As shown in FIG. 5 , the imaging device first takes the interception starting point 1 as the lower left corner of the image interception area 1, and intercepts the target field of view image from the driving environment image according to the image interception area 1. Then, the imaging device moves the interception start point to interception start point 2, and takes interception start point 2 as the lower left corner of the image interception area 2, and intercepts the target field of view image from the driving environment image according to the image interception area 2. Then, the imaging device moves the interception starting point to the intercepting start point 3, and takes the interception starting point 3 as the lower left corner of the image interception area 3, and intercepts the target field of view image from the driving environment image according to the image interception area 3.

The imaging device translates the image capture area according to a preset moving step, and captures a plurality of target field-of-view images according to the translated image capture area.

In one of the embodiments, as shown in FIG. 6 , in the driving environment image, the camera device intercepts the target field of view image according to the determined image interception area, including the following steps:

S602, the imaging device performs scaling processing on the image interception area to obtain at least one target image interception area.

S604, in the driving environment image, the imaging device captures the first target field of view image according to the image capture area, and captures the second target field of view image according to the target image capture area.

The imaging device performing zoom processing on the image clipping area refers to enlarging or reducing the size of the image clipping area. The imaging device can zoom in and out of the image capturing area with the center point of the image capturing area as the center. Alternatively, the imaging device may also select one of the points from the image clipping area, and then use the selected point as the center to zoom the image clipping area.

As shown in Fig. 7a, the camera device enlarges the image interception area twice to obtain target image interception area 1 and target image interception area 2 respectively, and then performs the driving environment image according to target image interception area 1 and target image interception area 2 respectively. Intercept to obtain two images of the second target field of view with different sizes.

In one of the embodiments, as shown in FIG. 7b , the camera device performs scaling processing on the image capture area to obtain a target image capture area 1 and a target image capture area 2 . Then, the camera device selects pixels at different positions in the driving environment image as center point 1 and center point 2 . The imaging device takes the center point 1 as the center of the target image capture area 1, and captures a second target field of view image from the driving environment image. And, taking the center point 2 as the center of the target image clipping area 2, a second target field of view image is clipped from the driving environment image.

In one of the embodiments, the imaging device performs statistics on the grayscale values of the pixels in the first target field of view image to obtain a first grayscale histogram, and performs a calculation on the grayscale values of the pixels in the second target field of view image. Statistics are performed to obtain a second grayscale histogram; the gain of the first target field of view image is adjusted according to the first grayscale histogram, and the gain of the second target field of view image is adjusted according to the second grayscale histogram.

A grayscale value is a numerical value representing the color of a black and white image between white and black, usually 0 for black and 255 for white. The grayscale histogram is a statistical map formed by counting the grayscale values of all pixels in a digital image to obtain the frequency of occurrence of each grayscale value. The grayscale histogram represents the number of pixels with a certain grayscale value in the image, and reflects the frequency of each grayscale value in the image.

If there are more pixels with smaller gray values in the first grayscale histogram or the second grayscale histogram, it means that the brightness of the first target field of view image and the second target field of view image is relatively small, and the number of imaging devices is increased. The gain of the first target field of view image and the second target field of view image brightens the first target field of view image and the second target field of view image. On the contrary, if there are more pixels with larger gray values in the first grayscale histogram or the second grayscale histogram, it means that the brightness of the first target field of view image and the second target field of view image is relatively high, then The imaging device reduces the gain of the first target field of view image and the second target field of view image, and dims the first target field of view image and the second target field of view image.

In one of the embodiments, the imaging device calculates the first average brightness of the first target field of view image and the second average brightness of the second target field of view image; the first average brightness and the first target field of view image are The brightness value of the pixel is used as the calculation parameter in the first mapping calculation formula to perform mapping calculation to obtain the adjusted brightness value; the first mapping calculation formula is:

Among them, L ₁ (x, y) represents the adjusted brightness value of the pixel in the first target field of view image,

represents the first average brightness, L _{w1(x, y)} represents the brightness value of the pixel in the first target field of view image, a is a constant; the second average brightness and the second target field of view The brightness value of the pixel in the image, Perform the mapping calculation as the calculation parameter in the second mapping calculation formula to obtain the adjusted brightness value; the second mapping calculation formula is:

Among them, L ₂ (x, y) represents the adjusted brightness value of the pixel in the second target field of view image,

represents the second average brightness, and L _{w2(x, y)} represents the brightness value of the pixel in the second target field of view image.

The first average brightness is the average value of the brightness of the first target field of view images, and the second average brightness is the average value of the brightness of the second target field of view images. For images in YUV format, the average brightness of the image is obtained by calculating the average value of the Y channel. For an image in RGB format, first, according to the formula Y=0.299R+0.587G+0.114B, the Y brightness value of the pixel is calculated by the R, G, and B values of the pixel, and then the average brightness of the image is calculated.

The imaging device performs a mapping calculation on the brightness values of the pixels in the first target field of view image and the second target field of view image through the first mapping calculation formula and the second mapping calculation formula, so as to map the first target field of view image and the second target field of view image. The brightness of the pixels in the image of the second target field of view is adjusted to make the brightness distribution more uniform and the image clearer.

In one of the embodiments, the camera device receives the field of view angle selection instruction, and extracts the target field of view angle from the field of view angle selection instruction; or acquires a configuration file, and configures the field of view angle extracted from the configuration file as the target field of view.

The viewing angle selection instruction may include one or more target viewing angles. For example, the viewing angle selection instruction may include three target viewing angles of 30 degrees, 60 degrees, and 90 degrees. The field angle selection instruction can be an instruction triggered by a touch screen, or an instruction input through an external hardware device. For example, commands entered through a mouse, keyboard, etc.

A configuration file is a file that configures parameters for a computer program. The configuration file may be a file that has been configured on the camera device before it leaves the factory, or may be a file temporarily created when the camera device is used.

In one of the embodiments, as shown in FIG. 8 , the imaging device acquires optical signals through a camera lens, and then converts the optical signals into electrical signals through an image sensor to generate a driving environment image. The camera device then inputs the driving environment image into channel 1 and channel 2, respectively. In channel 1, the camera equipment intercepts the driving environment image to obtain at least one target field of view image with a different field of view from the original field of view, and then performs ISP (Image Signal Processing) image on the intercepted target field of view image. deal with. In channel 2, the camera device performs image downsampling on the driving environment image to obtain a downsampled driving environment image, and then performs ISP image processing on the downsampled driving environment image. ISP image processing includes adjusting the gain, white balance, and tone mapping of the image. Path 1 and Path 2 may be different hardware paths, or may be different processing paths formed by very multiplexing the same hardware circuit. The driving environment image output by the image sensor may go through serializer, network, MIPI (Mobile Industry Processor Interface), LVDS (Low Voltage Differential Signaling, Low-voltage differential signal interface), etc. The image sensor can also be integrated with an image interception program, and the image sensor intercepts the driving environment image to obtain the target field of view image.

It should be understood that although the steps in the flowcharts of FIGS. 2, 4 and 6 are sequentially displayed according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, the execution of these steps is not strictly limited to the order, and these steps may be performed in other orders. Moreover, at least a part of the steps in FIGS. 2, 4 and 6 may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but may be executed at different times. These sub-steps are not necessarily completed at the same time. Alternatively, the order of execution of the stages is not necessarily sequential, but may be performed alternately or alternately with other steps or sub-steps of other steps or at least a portion of a stage.

In one of the embodiments, as shown in FIG. 9 , an image acquisition device with multiple viewing angles is provided, including: an acquisition module 902 , an input module 904 , an image interception module 906 , a downsampling module 908 and an acquisition module 910 , in:

The acquisition module 902 is used for acquiring the driving environment image according to the original field of view of the camera device during the automatic driving process;

an input module 904, configured to input the driving environment image into the first image processing path and the second image processing path respectively;

The image interception module 906 is used for intercepting at least one target angle of view image in the driving environment image according to the target angle of view through the first image processing path;

A downsampling module 908, configured to downsample the driving environment image through the second image processing path to obtain the downsampled driving environment image of the original field of view;

The acquisition module 910 is configured to use the extracted target field of view image and the down-sampled driving environment image as a multi-field of view image.

In one embodiment, the target angle of view includes a target horizontal angle of view and a target vertical angle of view; the original angle of view includes an original horizontal angle of view and an original vertical angle of view; the image interception module 906 is further configured to:

Calculate the image interception length according to the length calculation formula; the length calculation formula is:

Among them, l is the image interception length, l' is the length of the driving environment image, α ₁ is the original horizontal field of view, and β ₁ is the target horizontal field of view;

Calculate the image interception width according to the width calculation formula; the width calculation formula is:

Among them, w is the image interception width, w' is the width of the driving environment image, α ₂ is the original vertical field of view, and β ₂ is the target vertical field of view;

Determine the image clipping area according to the image clipping length and image clipping width;

In the driving environment image, the target field of view image is captured according to the determined image capturing area.

In one embodiment, the image capturing module 906 is further configured to:

Determine the interception starting point in the driving environment image;

At the interception starting point, the target field of view image in the driving environment image is intercepted according to the image interception area; The interception area intercepts the target field of view image in the driving environment image.

In one embodiment, the image capturing module 906 is further configured to:

scaling the image interception area to obtain at least one target image interception area;

In the driving environment image, the first target view angle image is captured according to the image capturing area, and the second target view angle image is captured according to the target image capturing area.

In one of the embodiments, as shown in Figure 10, the device further includes:

The statistics module 912 is used to perform statistics on the grayscale values of the pixels in the first target field of view image to obtain a first grayscale histogram, and perform statistics on the grayscale values of the pixels in the second target field of view image to obtain the second grayscale histogram;

The adjustment module 914 is configured to adjust the gain of the first target field of view image according to the first grayscale histogram, and adjust the gain of the second target field of view image according to the second grayscale histogram.

In one embodiment, the apparatus further includes:

A calculation module 916, configured to calculate the first average brightness of the first target field of view image and the second average brightness of the second target field of view image;

The calculation module 916 is further configured to use the first average brightness and the brightness value of the pixel in the first target field of view image as the calculation parameter in the first mapping formula to perform mapping calculation to obtain the adjusted brightness value;

The first mapping formula is:

represents the first average brightness, L _{w1(x, y)} represents the brightness value of the pixel in the first target field of view image, and a is a constant;

The calculation module 916 is also used for the second average brightness and the brightness value of the pixel in the second target field of view image, as the calculation parameter in the second mapping formula, to carry out the mapping calculation to obtain the adjusted brightness value;

The second mapping formula is:

In one embodiment, the apparatus further includes:

The receiving module 918 is configured to receive the field of view angle selection instruction, and extract the target field of view angle from the field of view angle selection instruction; or, obtain a configuration file, and configure the field of view angle extracted from the configuration file as the target field of view angle .

For the specific definition of the image acquisition device with multiple viewing angles, please refer to the definition of the image acquisition method with multiple viewing angles above, which will not be repeated here. Each module in the above-mentioned multi-field-of-view image acquisition device may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

In one of the embodiments, a computer device is provided, the computer device may be a camera device, and an internal structure diagram of which may be shown in FIG. 11 . The computer equipment includes a processor, memory, a communication interface, a display screen, and an input device connected by a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium, an internal memory. The non-volatile storage medium stores an operating system and computer-readable instructions. The internal memory provides an environment for the execution of the operating system and computer-readable instructions in the non-volatile storage medium. The communication interface of the computer device is used for wired or wireless communication with an external terminal, and the wireless communication can be realized by WIFI, operator network, NFC (Near Field Communication) or other technologies. The computer-readable instructions, when executed by a processor, implement a method for acquiring images with multiple viewing angles. The display screen of the computer equipment may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment may be a touch layer covered on the display screen, or a button, a trackball or a touchpad set on the shell of the computer equipment , or an external keyboard, trackpad, or mouse.

Those skilled in the art can understand that the structure shown in FIG. 11 is only a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer equipment to which the solution of the present application is applied. Include more or fewer components than shown in the figures, or combine certain components, or have a different arrangement of components.

A computer device comprising a memory and one or more processors, the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processors, causes the one or more processors to perform the following steps: during an autonomous driving process , collect the driving environment according to the original field of view of the camera device, and obtain the driving environment image; input the driving environment image into the first image processing path and the second image processing path respectively; through the first image processing path, in the driving environment image according to the target The field of view angle intercepts at least one target field of view image; through the second image processing path, the driving environment image is down-sampled to obtain a down-sampled driving environment image of the original field of view; the extracted target field of view image and down-sampled Driving environment images as multi-view images.

In one embodiment, the target angle of view includes a target horizontal angle of view and a target vertical angle of view; the original angle of view includes an original horizontal angle of view and an original vertical angle of view; when the processor executes the computer-readable instructions, the Implement the following steps: Calculate the length of image interception according to the length calculation formula; the length calculation formula is:

Among them, w is the image interception width, w′ is the width of the driving environment image, _α2 is the original vertical field of view, and _β2 is the target vertical field of view; the image interception area is determined according to the image interception length and image interception width; In the environment image, the target field of view image is intercepted according to the determined image interception area.

In one of the embodiments, when the processor executes the computer-readable instructions, it further implements the following steps: determining a starting point for interception in the driving environment image; intercepting a target field of view image in the driving environment image according to the image interception area at the interception starting point; and , at the interception starting point, the image interception area is gradually moved according to the preset moving step, and after each movement, the target field of view image in the driving environment image is intercepted according to the moved image interception area.

In one embodiment, when the processor executes the computer-readable instructions, the following steps are further implemented: performing zoom processing on the image interception area to obtain at least one target image interception area; in the driving environment image, intercepting the first target view area according to the image interception area A field angle image, and a second target field angle image is captured according to the target image capture area.

In one of the embodiments, when the processor executes the computer-readable instructions, it further implements the following steps: performing statistics on the grayscale values of pixels in the image of the first target field of view to obtain a first grayscale histogram; The gray value of the pixels in the field of view image is counted to obtain a second gray histogram; the gain of the first target field of view image is adjusted according to the first gray histogram, and the second gray histogram is adjusted according to the second gray histogram. The gain of the target field of view image.

In one embodiment, when the processor executes the computer-readable instructions, it further implements the following steps: calculating the first average brightness of the first target field of view image and the second average brightness of the second target field of view image; The average brightness and the brightness value of the pixel in the image of the first target field of view are used as the calculation parameters in the first mapping calculation formula to perform mapping calculation to obtain the adjusted brightness value; the first mapping calculation formula is:

In one of the embodiments, when the processor executes the computer-readable instructions, it further implements the following steps: receiving a viewing angle selection instruction, and extracting the target viewing angle from the viewing angle selection instruction; The field of view extracted from the configuration file is configured as the target field of view.

One or more non-volatile computer-readable storage media storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the following steps: during the autonomous driving process , collect the driving environment according to the original field of view of the camera device to obtain the driving environment image; input the driving environment image into the first image processing path and the second image processing path respectively; through the first image processing path, in the driving environment image according to At least one target field of view image is intercepted from the target field of view; the driving environment image is downsampled through the second image processing path to obtain a downsampled driving environment image of the original field of view; the extracted target field of view image and the downsampled driving environment image are obtained; Sample the driving environment image as a multi-view image.

In one embodiment, the target angle of view includes a target horizontal angle of view and a target vertical angle of view; the original angle of view includes an original horizontal angle of view and an original vertical angle of view; when the computer-readable instructions are executed by the processor The following steps are also implemented: calculating the length of image interception according to the length calculation formula; the length calculation formula is:

In one embodiment, when the computer-readable instructions are executed by the processor, the following steps are further implemented: determining an interception starting point in the driving environment image; intercepting a target field of view image in the driving environment image according to the image interception area at the intercepting starting point; And, the image capturing area is gradually moved according to the preset moving step at the capturing starting point, and after each movement, the target field of view image in the driving environment image is captured according to the moved image capturing area.

In one embodiment, when the computer-readable instructions are executed by the processor, the following steps are further implemented: performing zoom processing on the image interception area to obtain at least one target image interception area; in the driving environment image, intercepting the first target according to the image interception area A field of view image, and a second target field of view image is captured according to the target image capture area.

In one of the embodiments, the computer-readable instructions, when executed by the processor, further implement the following steps: collecting statistics on the grayscale values of pixels in the first target field of view image to obtain a first grayscale histogram; The gray value of the pixel in the target field of view image is counted to obtain a second grayscale histogram; the gain of the first target field of view image is adjusted according to the first grayscale histogram, and the second grayscale histogram is adjusted according to the second grayscale histogram. The gain of the second target field of view image.

In one embodiment, the computer-readable instructions, when executed by the processor, further implement the following steps: calculating a first average brightness of the first target field of view image and a second average brightness of the second target field of view image; An average brightness and the brightness value of the pixel in the first target field of view image are used as the calculation parameters in the first mapping calculation formula to perform mapping calculation to obtain the adjusted brightness value; the first mapping calculation formula is:

In one of the embodiments, the computer-readable instructions further implement the following steps when executed by the processor: receiving a viewing angle selection instruction, and extracting a target viewing angle from the viewing angle selection instruction; or, acquiring a configuration file, and converting The field of view extracted from the configuration file is configured as the target field of view.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a non-volatile computer. In the readable storage medium, the computer-readable instructions, when executed, may include the processes of the foregoing method embodiments. Wherein, any reference to memory, storage, database or other medium used in the various embodiments provided in this application may include non-volatile and/or volatile memory. Nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Road (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

The technical features of the above embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features It is considered to be the range described in this specification.

The above-mentioned embodiments only represent several embodiments of the present application, and the descriptions thereof are specific and detailed, but should not be construed as a limitation on the scope of the invention patent. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which all belong to the protection scope of the present application. Therefore, the scope of protection of the patent of the present application shall be subject to the appended claims.

Claims

An image acquisition method with multiple viewing angles, performed by a camera device, characterized in that the method comprises:

During the automatic driving process, the driving environment is collected according to the original field of view of the camera device to obtain a driving environment image;

inputting the driving environment image into the first image processing path and the second image processing path respectively;

through the first image processing path, intercepting at least one target field of view image in the driving environment image according to the target field of view;

Through the second image processing path, down-sampling the driving environment image to obtain the down-sampled driving environment image of the original field of view;

The extracted target field of view image and the down-sampled driving environment image are used as a multi-field of view image.
The method according to claim 1, wherein the target angle of view includes a target horizontal angle of view and a target vertical angle of view; the original angle of view includes an original horizontal angle of view and an original vertical angle of view ; Said intercepting at least one target field of view image according to the target field of view in the driving environment image includes:

Calculate the image interception length according to the length calculation formula; the length calculation formula is:
Wherein, l is the length of the image interception, l′ is the length of the driving environment image, α 1 is the original horizontal field of view, and β 1 is the target horizontal field of view;

Calculate the image interception width according to the width calculation formula; the width calculation formula is:
Wherein, w is the image interception width, w' is the width of the driving environment image, α 2 is the original vertical field of view, and β 2 is the target vertical field of view;

Determine an image clipping area according to the image clipping length and the image clipping width;

In the driving environment image, the target field of view image is captured according to the determined image capturing area.
The method according to claim 2, wherein, in the driving environment image, capturing the target field of view image according to the determined image capturing area comprises:

determining an interception starting point in the driving environment image;

intercepting the target field of view image in the driving environment image at the interception starting point according to the image intercepting area; After each movement, the target field of view image in the driving environment image is captured according to the moved image capturing area.
The method according to claim 2, wherein, in the driving environment image, capturing the target field of view image according to the determined image capturing area comprises:

scaling the image capture area to obtain at least one target image capture area;

In the driving environment image, the first target view angle image is captured according to the image capturing region, and the second target view angle image is captured according to the target image capturing region.
The method according to claim 4, wherein the method further comprises:

Statistics are performed on the grayscale values of the pixels in the first target field of view image to obtain a first grayscale histogram, and statistics are performed on the grayscale values of the pixels in the second target field of view image to obtain a second grayscale value. grayscale histogram;

The gain of the first target field of view image is adjusted according to the first grayscale histogram, and the gain of the second target field of view image is adjusted according to the second grayscale histogram.
The method according to claim 4, wherein the method further comprises:

calculating the first average brightness of the first target field of view image and the second average brightness of the second target field of view image;

The first average brightness and the brightness value of the pixel in the first target field of view image are used as the calculation parameters in the first mapping calculation formula to perform mapping calculation to obtain the adjusted brightness value;

The first mapping formula is:
Wherein, L 1 (x, y) represents the adjusted brightness value of the pixel in the first target field of view image,
represents the first average brightness, L w1(x, y) represents the brightness value of the pixel in the first target field of view image, and α is a constant;

The second average brightness and the brightness value of the pixel in the second target field of view image are used as the calculation parameters in the second mapping formula to perform mapping calculation to obtain the adjusted brightness value;

The second mapping formula is:
Wherein, L 2 (x, y) represents the adjusted brightness value of the pixel in the second target field of view image,
represents the second average brightness, and L w2(x, y) represents the brightness value of the pixel in the second target field of view image.
The method according to claim 1, wherein the method further comprises:

Receive a viewing angle selection instruction, and extract the target viewing angle from the viewing angle selection instruction; or, obtain a configuration file, and configure the viewing angle extracted from the configuration file as the target viewing angle field angle.
An image acquisition device with multiple viewing angles, characterized in that the device comprises:

an acquisition module, used for acquiring the driving environment image according to the original field of view of the camera device during the automatic driving process;

an input module for inputting the driving environment image into the first image processing path and the second image processing path respectively;

an image interception module, configured to intercept at least one target angle of view image in the driving environment image according to the target angle of view through the first image processing path;

A downsampling module, configured to downsample the driving environment image through the second image processing path to obtain a downsampled driving environment image of the original field of view;

The acquisition module is configured to use the extracted target field of view image and the down-sampled driving environment image as a multi-field of view image.
The device according to claim 8, wherein the target angle of view includes a target horizontal angle of view and a target vertical angle of view; the original angle of view includes an original horizontal angle of view and an original vertical angle of view ; The image interception module is also used for:

Calculate the image interception length according to the length calculation formula; the length calculation formula is:
Wherein, l is the length of the image interception, l′ is the length of the driving environment image, α 1 is the original horizontal field of view, and β 1 is the target horizontal field of view;

Calculate the image interception width according to the width calculation formula; the width calculation formula is:
Wherein, w is the image interception width, w' is the width of the driving environment image, α 2 is the original vertical field of view, and β 2 is the target vertical field of view;

Determine an image clipping area according to the image clipping length and the image clipping width;

In the driving environment image, the target field of view image is captured according to the determined image capturing area.
The device according to claim 9, wherein the image interception module is further configured to:

determining an interception starting point in the driving environment image;

intercepting a target field of view image in the driving environment image at the interception starting point according to the image intercepting area; After each movement, the target field of view image in the driving environment image is captured according to the moved image capturing area.
The device according to claim 9, wherein the image interception module is further configured to:

scaling the image capture area to obtain at least one target image capture area;

In the driving environment image, the first target view angle image is captured according to the image capturing region, and the second target view angle image is captured according to the target image capturing region.
The apparatus of claim 11, wherein the apparatus further comprises:

A statistics module is used to perform statistics on the grayscale values of pixels in the first target field of view image to obtain a first grayscale histogram, and to perform statistics on the grayscale values of pixels in the second target field of view image. Statistics to obtain the second grayscale histogram;

An adjustment module, configured to adjust the gain of the first target field of view image according to the first grayscale histogram, and adjust the gain of the second target field of view image according to the second grayscale histogram.
The apparatus of claim 11, wherein the apparatus further comprises:

a calculation module for calculating the first average brightness of the first target field of view image and the second average brightness of the second target field of view image;

The calculation module is further configured to use the first average brightness and the brightness value of the pixel in the first target field of view image as the calculation parameter in the first mapping formula to perform mapping calculation to obtain the adjusted brightness value. ;

The first mapping formula is:
Wherein, L 1 (x, y) represents the adjusted brightness value of the pixel in the first target field of view image,
represents the first average brightness, L w1(x, y) represents the brightness value of the pixel in the first target field of view image, and α is a constant;

The calculation module is further configured to use the second average brightness and the brightness value of the pixel in the second target field of view image as the calculation parameter in the second mapping calculation formula to perform mapping calculation to obtain the adjusted brightness value. ;

The second mapping formula is:
Wherein, L 2 (x, y) represents the adjusted brightness value of the pixel in the second target field of view image,
represents the second average brightness, and L w2(x, y) represents the brightness value of the pixel in the second target field of view image.
The apparatus according to claim 8, wherein the apparatus further comprises:

A receiving module, configured to receive a field of view angle selection instruction, and extract the target field of view angle from the field of view angle selection instruction; or obtain a configuration file, and configure the field of view angle extracted from the configuration file is the target field of view.
A computer device comprising a memory and one or more processors, the memory having computer-readable instructions stored therein, the computer-readable instructions, when executed by the one or more processors, cause the one or more processors Each processor performs the following steps:

During the automatic driving process, the driving environment is collected according to the original field of view of the camera device to obtain a driving environment image;

inputting the driving environment image into the first image processing path and the second image processing path respectively;

through the first image processing path, intercepting at least one target field of view image in the driving environment image according to the target field of view;

Through the second image processing path, down-sampling the driving environment image to obtain the down-sampled driving environment image of the original field of view;

The extracted target field of view image and the down-sampled driving environment image are used as a multi-field of view image.
The computer device according to claim 15, wherein the target angle of view includes a target horizontal angle of view and a target vertical angle of view; the original angle of view includes an original horizontal angle of view and an original vertical angle of view angle; when the processor executes the computer-readable instructions, the processor further performs the following steps:

Calculate the image interception length according to the length calculation formula; the length calculation formula is:
Wherein, l is the length of the image interception, l′ is the length of the driving environment image, α 1 is the original horizontal field of view, and β 1 is the target horizontal field of view;

Calculate the image interception width according to the width calculation formula; the width calculation formula is:
Wherein, w is the image interception width, w' is the width of the driving environment image, α 2 is the original vertical field of view, and β 2 is the target vertical field of view;

Determine an image clipping area according to the image clipping length and the image clipping width;

In the driving environment image, the target field of view image is captured according to the determined image capturing area.
The computer device of claim 16, wherein the processor further performs the following steps when executing the computer-readable instructions:

determining an interception starting point in the driving environment image;

intercepting a target field of view image in the driving environment image at the interception starting point according to the image intercepting area; After each movement, the target field of view image in the driving environment image is captured according to the moved image capturing area.
One or more non-volatile computer-readable storage media storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the following steps:

During the automatic driving process, the driving environment is collected according to the original field of view of the camera device to obtain a driving environment image;

inputting the driving environment image into the first image processing path and the second image processing path respectively;

through the first image processing path, intercepting at least one target field of view image in the driving environment image according to the target field of view;

Through the second image processing path, down-sampling the driving environment image to obtain the down-sampled driving environment image of the original field of view;

The extracted target field of view image and the down-sampled driving environment image are used as a multi-field of view image.
The storage medium according to claim 18, wherein the target angle of view includes a target horizontal angle of view and a target vertical angle of view; the original angle of view includes an original horizontal angle of view and an original vertical angle of view angle; the computer-readable instructions, when executed by the processor, further perform the following steps:

Calculate the image interception length according to the length calculation formula; the length calculation formula is:
Wherein, l is the length of the image interception, l′ is the length of the driving environment image, α 1 is the original horizontal field of view, and β 1 is the target horizontal field of view;

Calculate the image interception width according to the width calculation formula; the width calculation formula is:
Wherein, w is the image interception width, w' is the width of the driving environment image, α 2 is the original vertical field of view, and β 2 is the target vertical field of view;

Determine an image clipping area according to the image clipping length and the image clipping width;

In the driving environment image, the target field of view image is captured according to the determined image capturing area.
The storage medium of claim 19, wherein the computer-readable instructions further perform the following steps when executed by the processor:

determining an interception starting point in the driving environment image;

intercepting a target field of view image in the driving environment image at the interception starting point according to the image intercepting area; After each movement, the target field of view image in the driving environment image is captured according to the moved image capturing area.