JP2022181996A - Image processing apparatus and vehicle - Google Patents

Abstract

To provide an image processing apparatus or the like, configured to increase the speed of estimation processing speed of an object position.SOLUTION: An image processing apparatus includes: an area setting unit which sets one or more image areas in a captured image; an extraction unit which extracts two-dimensional layout feature quantities included in the image areas of the captured image; and a position estimation unit which performs correlation operation using convolutional operation on the feature quantities, in image areas of the captured image obtained in each of frame periods adjacent along a time axis, to estimate a position of an object to be tracked. The position estimation unit uses Fourier transform on the feature quantities in the convolution operation, and executes the Fourier transform and the convolution operation on the two-dimensional layout feature quantities per dimension.SELECTED DRAWING: Figure 6

Description

The present disclosure relates to an image processing device that estimates the position of an object based on a captured image, and a vehicle equipped with such an image processing device.

Captured images obtained by an imaging device include images of various objects. For example, Patent Literature 1 discloses an image processing device that estimates the position of an object based on such captured images.

JP 2008-123141 A

By the way, in such an image processing apparatus, there is a demand for speeding up the process of estimating the position of an object. It is desirable to provide an image processing device capable of speeding up object position estimation processing, and a vehicle equipped with such an image processing device.

An image processing apparatus according to an embodiment of the present disclosure includes a region setting unit that sets one or more image regions in a captured image, and an extraction unit that extracts a two-dimensional arrangement feature amount included in the image region of the captured image. Position estimation for estimating the position of the tracked object by performing a correlation operation using a convolution operation on the feature quantity between the image areas of the captured images obtained in the preceding and following frame periods along the time axis. and When performing the convolution operation, the position estimation unit uses the Fourier transform of the feature amount, and performs the Fourier transform and the convolution operation on the two-dimensionally arranged feature amount for each one dimension.

A vehicle according to an embodiment of the present disclosure is a vehicle that performs vehicle control using the image processing device according to the embodiment of the present disclosure and the estimated position of a tracked object obtained from the position estimation unit. and a control unit.

1 is a block diagram showing a schematic configuration example of a vehicle according to an embodiment of the present disclosure; FIG. FIG. 2 is a top view schematically showing an external configuration example of the vehicle shown in FIG. 1; FIG. 2 is a schematic diagram showing an example of a left image and a right image generated by the stereo camera shown in FIG. 1; 2 is a schematic diagram showing an example of an image area set by an area setting unit shown in FIG. 1; FIG. FIG. 4 is a schematic diagram for explaining an overview of object tracking using correlation calculation; FIG. 10 is a schematic diagram showing a processing example of correlation calculation according to the embodiment; 7 is a flow chart showing an example of processing including correlation calculation according to the embodiment; FIG. 10 is a schematic diagram showing a comparison of correlation calculation processing examples according to a comparative example and an example.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. The description will be given in the following order.
1. Embodiment (example of setting an image area using distance information or machine learning)
2. Modification

<1. Embodiment>
[Constitution]
FIG. 1 is a block diagram showing a schematic configuration example of a vehicle (vehicle 10) according to an embodiment of the present disclosure. FIG. 2 is a schematic top view showing an example of the exterior configuration of the vehicle 10 shown in FIG.

The vehicle 10 includes a stereo camera 11, an image processing device 12, and a vehicle control section 13, as shown in FIG. It should be noted that FIG. 1 omits illustration of a driving force source (engine, motor, etc.) of the vehicle 10 . The vehicle 10 is, for example, an electric vehicle such as a hybrid vehicle (HEV) or an electric vehicle (EV), or a gasoline vehicle.

(A. Stereo camera 11)
The stereo camera 11 is a camera that generates a pair of images (a left image PL and a right image PR) having parallax with each other by capturing an image in front of the vehicle 10, as shown in FIG. 2, for example. This stereo camera 11 has a left camera 11L and a right camera 11R, as shown in FIGS.

Left camera 11L and right camera 11R each include, for example, a lens and an image sensor. For example, as shown in FIG. 2, the left camera 11L and the right camera 11R are arranged in the vicinity of the upper portion of the windshield 19 of the vehicle 10 with a predetermined distance therebetween along the width direction of the vehicle 10 . These left camera 11L and right camera 11R are designed to perform imaging operations in synchronization with each other. Specifically, as shown in FIG. 1, the left camera 11L generates a left image PL and the right camera 11R generates a right image PR. The left image PL contains multiple pixel values, and the right image PR contains multiple pixel values. These left image PL and right image PR constitute a stereo image PIC, as shown in FIG.

FIG. 3 shows an example of such a stereo image PIC. Specifically, FIG. 3A shows an example of the left image PL, and FIG. 3B shows an example of the right image PR. Note that x and y shown in FIG. 3 represent the x-axis and y-axis, respectively. In this example, another vehicle (preceding vehicle 90) is running in front of the vehicle 10 on the road on which the vehicle 10 is running. The left camera 11L captures the preceding vehicle 90 to generate the left image PL, and the right camera 11R captures the preceding vehicle 90 to generate the right image PR.

The stereo camera 11 is adapted to generate a stereo image PIC including such left image PL and right image PR. Also, the stereo camera 11 is adapted to generate a series of stereo images PIC by performing imaging operations at a predetermined frame rate (eg, 60 [fps]).

(B. Image processing device 12)
The image processing device 12 is a device that performs various image processing (such as tracking an object in front of the vehicle 10) based on the stereo image PIC supplied from the stereo camera 11. FIG. The image processing device 12 has an image memory 121, a distance information generating section 122, an area setting section 123, a feature quantity extracting section 124, and a position estimating section 125, as shown in FIG.

Such an image processing device 12 includes, for example, one or more processors (CPU: Central Processing Unit) that executes a program, and one or more memories communicably connected to these processors. be done. Further, such a memory includes, for example, a RAM (Random Access Memory) that temporarily stores processing data, a ROM (Read Only Memory) that stores programs, and the like.

Note that the feature amount extraction unit 124 described above corresponds to a specific example of the “extraction unit” in the present disclosure.

(Image memory 121)
The image memory 121, as shown in FIG. 1, is a memory that temporarily stores the left image PL and the right image PR included in the stereo image PIC. Further, the image memory 21 sequentially supplies at least one of the left image PL and the right image PR thus stored to the distance information generation unit 122 and the feature amount extraction unit 124 as the captured image P. (See Figure 1).

(Distance information generator 122)
The distance information generation unit 122 performs predetermined image processing including stereo matching processing and filtering processing based on the captured image P (here, the left image PL and the right image PR) read from the image memory 121. to generate the distance information Iz (see FIG. 1). Specifically, the distance information generator 122 generates a distance image including a plurality of pixel values based on the left image PL and right image PR. Each of the multiple pixel values is a parallax value in this example. In other words, each of the plurality of pixel values corresponds to the distance to the point corresponding to each pixel in the three-dimensional real space. However, the present invention is not limited to this example. For example, each of the plurality of pixel values may be a distance value indicating the distance to a point corresponding to each pixel in a three-dimensional real space. In this manner, the distance information generator 122 generates distance information Iz, which is information indicating the distance to the point corresponding to each pixel.

(Region setting unit 123)
The area setting section 123 sets one or more image areas R in the captured image P based on the distance information Iz supplied from the distance information generating section 122 . Specifically, based on the distance information Iz, the region setting unit 123 identifies a plurality of pixels that are positioned close to each other in the captured image P and have substantially the same parallax value, and selects a rectangle that includes the plurality of pixels. The area is set as an image area R. That is, when there is an object in the captured image P, the pixels in the area corresponding to the object are positioned close to each other and have substantially the same parallax value. Therefore, by setting the image area R in this manner, the area setting unit 123 sets the image area R so as to surround the object.

FIG. 4 schematically shows an example of the image region R set by the region setting section 123. As shown in FIG. In the example shown in FIG. 4, an image region R is set for each of the two vehicles in the captured image P (here, one of the left image PL and the right image PR). In this example, the area setting unit 123 sets the image area R on the vehicle, but is not limited to this example, and may set the image area R on a person, a guardrail, a wall, etc., for example. .

Information about one or a plurality of image regions R set by the region setting unit 123 in this manner is supplied to the feature quantity extraction unit 124 as shown in FIG. .

Note that in the example shown in FIG. 1, the region setting unit 123 sets the image region R using the distance information Iz, but is not limited to this example. That is, the region setting unit 123 uses a learned model such as a DNN (Deep Neural Network), for example, to identify an object in the captured image P, and outputs the coordinates of the identified object to create a rectangular region. may be set. That is, the region setting unit 123 may set the image region R using machine learning.

(Feature quantity extraction unit 124)
The feature amount extraction unit 124 extracts the feature amount F included in one or more image regions R in the captured image P (here, one of the left image PL and the right image PR) ( See Figure 1). The feature amount F, which will be described in detail later (FIG. 6, etc.), is composed of pixel values of a plurality of pixels arranged in a matrix (two-dimensional arrangement). In addition, as such a feature amount F, for example, an RGB (Red, Green, Blue) feature amount, a HOB (Histograms of Oriented Gradients) feature amount, or the like can be cited.

The feature quantity extraction unit 124 extracts such a feature quantity F using, for example, a DNN trained model. In that case, for example, each neural network in the feature amount extraction unit 124 has a plurality of convolution layers and a plurality of pooling layers.

The feature quantity F extracted by the feature quantity extraction unit 124 in this manner is supplied to the position estimation unit 125 (see FIG. 1).

(Position estimation unit 125)
The position estimating unit 125 performs a correlation operation using a convolution operation on the feature amount F between the image regions R of the captured image P respectively obtained in the preceding and succeeding frame periods along the time axis, thereby performing a tracking operation. It estimates the position of the target object. That is, the position estimation unit 125 performs object tracking by estimating the position of the tracked object using the feature amount F for each captured image P in each frame period.

Further, such object tracking, for example, KCF (Kernelized Correlation Filter) and template matching, such as Siamese Network, such as CNN (Convolutional Neural Network: convolutional neural network) using an object tracking method, so as to be carried out It's becoming

FIG. 5 schematically shows an overview of object tracking using such correlation calculation. Note that t shown in FIG. 5 represents the time axis, and the same applies to the following.

For example, as shown in FIGS. 5A and 5B, in object tracking using correlation calculation, captured images P0, P1, . . , at Pn, how the position of the tracked object (position coordinate x0) has moved is estimated (refer to the amount of movement Δx). Specifically, during such object tracking, the position where the correlation value among the captured images P0, P1, . and is updated from time to time.

As an example, when tracking an object using the KCF described above, various calculations are performed as follows. That is, first, machine learning of the coefficient w is performed so as to minimize the value represented by the following equation (1). If the value of this coefficient w can be optimized, it becomes an equation that outputs the position coordinates of the tracked object in the input image. When optimizing the coefficient w in this manner, for example, a kernel trick, which will be described later, is used.

FIG. 6 schematically shows an example of correlation calculation processing according to the present embodiment. Specifically, the feature amount F extracted in the image area R of the captured image Pn obtained in the current frame period (see FIG. 6A) and the immediately following frame period (one frame later: FIG. 6 ( B) shows an example of correlation calculation processing using the feature amount Fn+1 acquired in the image region R of the captured image Pn+1 obtained in (see B)).

In each of the above-described feature amounts Fn and Fn+1, 16 pixels PX (vertical direction corresponding to the y-axis direction: 4 pixels x horizontal direction corresponding to the x-axis direction: 4 pixels) are arranged in a matrix. (two-dimensional arrangement) (see FIGS. 6A and 6B). FIG. 6 also shows a correlation map Mc obtained from correlation calculation by convolution using Fourier transform, which will be described below.

In the present embodiment, the position estimating unit 125 uses the Fourier transform of the feature amount F when performing the convolution operation on the feature amount F described above. Further, the position estimating unit 125 is configured to perform correlation calculation using such Fourier transform, for example, using the KCF described above. Furthermore, when using this KCF, for example, as shown in FIG. It is designed to speed up processing

Here, when the kernel trick described above is used in such a KCF, the function F(x) for Fourier transform is defined by the following equation (2). Further, when the convolution operation described above is expressed as (f*g), it is defined by the following equation (3).

Then, the convolution operation between the objects subjected to the Fourier transform is represented by the following equation (4). That is, as can be seen from the last expression in this expression (4), the convolution operation between the Fourier-transformed values is equal to the product of the Fourier-transformed values. Specifically, in the example of FIG. 6, when performing a convolution operation on the feature amounts Fn and Fn+1 transformed into the frequency space using the Fourier transform, the product of these feature amounts Fn and Fn+1 (matrices) is taken. By doing so, the calculation is performed. By using the KCF in this way, the amount of calculation is reduced, and high speed processing is realized.

Here, in the present embodiment, the position estimating unit 125 performs such Fourier transform and convolution operation on the two-dimensionally arranged feature quantity F one dimension at a time (for example, (See dashed arrows shown in feature quantities Fn and Fn+1 in FIGS. 6A and 6B). A detailed processing example in such a position estimation unit will be described later (FIGS. 7 and 8).

(C. Vehicle control unit 13)
The vehicle control unit 13 uses the position estimation result (estimated position Pe of the tracked object) supplied from the position estimation unit 125 to perform various vehicle controls in the vehicle 10 (see FIG. 1). Specifically, the vehicle control unit 13 performs, for example, travel control of the vehicle 10 and operation control of various members in the vehicle 10 based on the information of the estimated position Pe.

Similar to the image processing device 12, such a vehicle control unit 13 includes, for example, one or more processors (CPUs) that execute programs, one or more memories communicatively connected to these processors, Consists of Further, like the image processing apparatus 12, such a memory is also composed of, for example, a RAM for temporarily storing processing data, a ROM for storing programs, and the like.

[Operation and action/effect]
Next, the operation, functions and effects of this embodiment will be described in detail.

(A. Position estimation processing, etc. of the present embodiment)
First, with reference to FIGS. 7 and 8 in addition to FIGS. 1 to 6, an example of processing (position estimation processing, etc.) including the aforementioned correlation calculation according to the present embodiment will be described. FIG. 7 is a flowchart showing an example of processing including such a correlation calculation. Further, FIG. 8 shows a comparative example (FIG. 8A) and an example of the correlation calculation according to the example of the present embodiment (FIG. 8B) in comparison (marked by the dashed line CF ), which is a schematic representation.

In the processing example shown in FIG. 7, first, the stereo camera 11 captures an image in front of the vehicle 10 to generate a stereo image PIC (left image PL and right image PR) (step S100). Specifically, for example, a stereo image PIC (left image PL and right image PR) as shown in FIG. 3 is generated.

Next, the image memory 21 in the image processing device 12 temporarily stores the stereo image PIC (the left image PL and the right image PR) thus generated as the captured image P (step S101). Subsequently, the area setting unit 123 determines whether or not the image area R in the captured image P has been set (step S102). Here, if it is determined that such an image area R has been set (step S102: Y), the process proceeds to step S104, which will be described later.

On the other hand, if it is determined that such an image region R has not been set (step S102: N), then the region setting unit 123 uses the above-described method (distance information Iz, machine learning, etc.). to set one or a plurality of image regions R in the captured image P (step S103). Then, in step S104, the feature amount extraction unit 124 extracts the above-described feature amount F (two-dimensional arrangement) included in the image region R of the captured image P. FIG.

Subsequently, the position estimating unit 125 calculates the position of the tracked object by performing a correlation operation using a convolution operation on the feature amount F between the image regions R of the captured images Pn and Pn+1 (see FIG. 6). Estimate (step S105). Specifically, the position estimating unit 125 performs the above-described Fourier transform and convolution operation on the two-dimensionally arranged feature quantity F one dimension at a time, thereby estimating the position of such a tracked object. I do.

More specifically, in the example of FIG. 8B corresponding to the example of FIG. 6, the position estimation unit 125 performs position estimation as follows. That is, the position estimating unit 125 first executes the above-described one-dimensional processing of Fourier transform and convolution along the x-axis direction (row direction) of the feature amount Fn (see the dashed arrow). Then, the position estimation unit 125 adds the results of such one-dimensional processing along the x-axis direction to a plurality of sets (four sets in this example) along the y-axis direction (column direction). , performs a Fourier transform and a convolution operation on the two-dimensionally arranged feature amount Fn.

Here, the x-axis direction (row direction) described above corresponds to a specific example of the "first direction" in the present disclosure. Also, the y-axis direction (column direction) described above corresponds to a specific example of the “second direction” in the present disclosure.

If it is determined in step S105 that it is difficult to estimate the position of the object to be tracked, such position estimation ends, and the series of processing examples shown in FIG. 7 also ends.

Subsequently, the vehicle control unit 13 uses the position estimation result obtained in step S105 (estimated position Pe of the tracked object described above) to perform various vehicle controls in the vehicle 10 (running of the vehicle 10 described above). control, operation control of various members, etc.) is performed (step S106).

This completes the series of processing examples shown in FIG.

(B. Action and effect)
In this manner, in the present embodiment, the convolution operation is performed on the extracted feature amount F (two-dimensional arrangement) between the image regions R of the captured image P obtained respectively in the preceding and succeeding frame periods along the time axis. The following processing is performed when estimating the position of the tracked object by performing the correlation calculation using. That is, when performing a convolution operation on the feature amount F, a Fourier transform is used on the feature amount F, and these Fourier transform and convolution operation are each performed on the feature amount F arranged in two dimensions one dimension at a time. be. As a result, in the present embodiment, for example, the amount of arithmetic processing (computational amount) is reduced compared to the case where the Fourier transform and the convolution operation are each executed two-dimensionally (corresponding to the case of the comparative example below).

Specifically, in the comparative example shown in FIG. 8A, the amount of calculation is as follows, considering the case of using the discrete Fourier transform (DFT). That is, first, the order of the amount of calculation in the case of one-dimensional DFT for pixels PX in a two-dimensional arrangement of (N×N) is O(N ² ). Therefore, in this comparative example, as described above, the Fourier transform is performed in two dimensions (along each of the x-axis direction (row direction) and y-axis direction (column direction): see dashed arrows). Therefore, the order of the amount of calculation for two-dimensional DFT is O(N ² ×N ² )=O(N ⁴ ). That is, in the example of FIG. 8A, since the pixels PX are two-dimensionally arranged (4×4), the order of the amount of calculation for the two-dimensional DFT is O(4 ² ×4 ² )=O (4 ⁴ ).

On the other hand, in the embodiment shown in FIG. 8B, the amount of calculation is as follows when considering the case of using DFT in the same manner. That is, in this embodiment, the results of the one-dimensional DFT (see the dashed arrow) executed along the x-axis are added together in a plurality of sets (four sets) along the y-axis direction. is of the order of O(N ² ×N)=O(N ³ )=O(4 ³ ). That is, in this embodiment, the amount of calculation for DFT on the feature amount Fn is reduced while maintaining the calculation accuracy as compared with the above-described comparative example. Note that such a reduction in the amount of calculation is not limited to the above-described DFT, and the same applies to other types of Fourier transform such as FFT (Fast Fourier Transform).

As described above, in the present embodiment, it is possible to speed up the object position estimation process (reduce the processing time).

In addition, in the present embodiment, the results of one-dimensional processing along the x-axis direction are added together in a plurality of sets along the y-axis direction to perform a Fourier transform and a convolution operation on the feature quantity F in a two-dimensional arrangement. is executed, so it will be as follows. That is, by using such a specific calculation technique, it is possible to actually reduce the amount of calculation as described above.

Furthermore, in the present embodiment, when estimating the position of the object to be tracked, the correlation calculation is performed using the KCF described above, and the feature quantity F representing the color space is transformed into the frequency space by Fourier transform. As a result, it is possible to further increase the speed (further shorten the processing time).

In addition, in the present embodiment, the above-described distance information Iz or machine learning is used to set the image region R in the captured image P, so that such an image region R can be easily set. In this respect as well, it is possible to achieve further speeding up (further shortening of processing time).

<2. Variation>
Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to these embodiments, and various modifications are possible.

For example, the configuration (type, shape, arrangement, number, etc.) of each member in the vehicle 10 and the image processing device 12 is not limited to that described in the above embodiment. That is, the configuration of each of these members may be of other types, shapes, arrangements, numbers, and the like. Further, the values, ranges, magnitude relationships, etc. of the various parameters described in the above embodiments are not limited to those described in the above embodiments, and may be other values, ranges, magnitude relationships, and the like.

Specifically, for example, in the above-described embodiment, the stereo camera 11 is configured to capture an image in front of the vehicle 10. However, the present invention is not limited to such a configuration. It may be configured to image the side or the rear. Further, in the above-described embodiment, an example in which the stereo camera 11 is used has been described, but the present invention is not limited to this example. may

Further, for example, in the above-described embodiment, various processes performed in the vehicle 10 and the image processing device 12 have been described with specific examples, but the present invention is not limited to these specific examples. That is, other methods may be used to perform these various processes. Specifically, for example, the one-dimensional processing of the Fourier transform and convolution operation described above is not limited to the method described in the above embodiment (one-dimensional processing along the x-axis direction). It may be one-dimensional processing along the line or one-dimensional processing along other directions (such as oblique directions). Also, in the above embodiment, an example of a processing method using KCF has been described, but the present invention is not limited to this example, and other processing methods may be used.

Furthermore, the series of processes described in the above embodiment may be performed by hardware (circuit) or by software (program). When it is performed by software, the software consists of a program group for executing each function by a computer. Each program, for example, may be installed in the computer in advance and used, or may be installed in the computer from a network or a recording medium and used.

Further, in the above-described embodiment, an example in which the image processing device 12 is provided in a vehicle has been described, but the present invention is not limited to this example. Alternatively, it may be provided in a device other than the mobile body.

Furthermore, the various examples described so far may be applied in any combination.

Note that the effects described in this specification are merely examples and are not limited, and other effects may be provided.

In addition, the present disclosure can also be configured as follows.
(1)
an area setting unit that sets one or more image areas in a captured image;
an extraction unit that extracts a two-dimensional arrangement feature amount included in the image area of the captured image;
estimating the position of the tracked object by performing a correlation operation using a convolution operation on the feature amount between the image regions of the captured image obtained respectively in the preceding and following frame periods along the time axis; a position estimator and
The position estimation unit
When performing the convolution operation, using a Fourier transform for the feature amount,
An image processing device that executes the Fourier transform and the convolution operation on the feature amount arranged in the two-dimensional arrangement one dimension at a time.
(2)
The position estimation unit
Along the first direction in the feature quantity, performing one-dimensional processing of the Fourier transform and the convolution operation, respectively,
By adding together the results of the one-dimensional processing along the first direction in a plurality of sets along a second direction different from the first direction,
The image processing device according to (1) above, wherein the Fourier transform and the convolution operation are performed on the feature amount in the two-dimensional arrangement.
(3)
The position estimation unit
While performing the correlation calculation using KCF (Kernelized Correlation Filter),
The image processing apparatus according to (1) or (2) above, wherein the Fourier transform is used to transform the feature quantity representing a color space into a frequency space.
(4)
The region setting unit sets the image region using distance information generated based on the left image and the right image as the captured images obtained from a stereo camera, or machine learning. The image processing device according to any one of (3).
(5)
an image processing device according to any one of (1) to (4) above;
and a vehicle control unit that performs vehicle control using the estimated position of the tracked object supplied from the position estimation unit.
(6)
one or more processors; and one or more memories communicatively coupled to the one or more processors;
The one or more processors are
setting one or more image areas in a captured image;
Extracting a two-dimensional arrangement feature amount included in the image area of the captured image;
estimating the position of the tracked object by performing a correlation operation using a convolution operation on the feature amount between the image regions of the captured image obtained respectively in the preceding and following frame periods along the time axis; While doing things and
When performing the convolution operation, using a Fourier transform for the feature amount,
An image processing device that executes the Fourier transform and the convolution operation on the feature amount arranged in the two-dimensional arrangement one dimension at a time.

Reference Signs List 10 Vehicle 11 Stereo camera 11L Left camera 11R Right camera 12 Image processing device 121 Image memory 122 Distance information generation unit 123 Area setting unit 124 Feature amount extraction unit 125 Position estimation unit 13 Vehicle control unit 19 Windshield 90 Leading vehicle PL Left image PR Right image PIC Stereo image P, P0 to Pn, Pn+1 Captured image R Image area, Iz: distance information, F, Fn, Fn+1: feature amount, Mc: correlation map, Pe: estimated position, x0: position coordinate, PX: pixel, t: time axis, t0 to tn: timing.

Claims (5)

an area setting unit that sets one or more image areas in a captured image;
an extraction unit that extracts a two-dimensional arrangement feature amount included in the image area of the captured image;
estimating the position of the tracked object by performing a correlation operation using a convolution operation on the feature amount between the image regions of the captured image obtained respectively in the preceding and following frame periods along the time axis; a position estimator and
The position estimation unit
When performing the convolution operation, using a Fourier transform for the feature amount,
An image processing device that executes the Fourier transform and the convolution operation on the feature amount arranged in the two-dimensional arrangement one dimension at a time. The position estimation unit
Along the first direction in the feature quantity, performing one-dimensional processing of the Fourier transform and the convolution operation, respectively,
By adding together the results of the one-dimensional processing along the first direction in a plurality of sets along a second direction different from the first direction,
The image processing apparatus according to claim 1, wherein the Fourier transform and the convolution operation are performed on the feature amount of the two-dimensional arrangement. The position estimation unit
While performing the correlation calculation using KCF (Kernelized Correlation Filter),
3. The image processing apparatus according to claim 1, wherein the Fourier transform is used to transform the feature amount representing a color space into a frequency space. The area setting unit sets the image area using distance information generated based on the left image and the right image as the captured images obtained from a stereo camera, or machine learning. Item 4. The image processing device according to any one of Item 3. an image processing apparatus according to any one of claims 1 to 4;
and a vehicle control unit that performs vehicle control using the estimated position of the tracked object supplied from the position estimation unit.

Images