JP2020190412A - Vehicle state estimating system, vehicle state estimating method, and vehicle state estimating program - Google Patents

Abstract

To provide a vehicle state estimating system, a vehicle state estimating method and a vehicle state estimating program which, when a vehicle is automatically driven in a parking lot, properly estimate the initial bearing and/or the initial position of the vehicle at the starting origin point of automatic driving.SOLUTION: In a vehicle state estimating system 100, an image recognition unit 25 of a vehicle control device 20 recognizes a belt-like ground feature out of the ground features included in surrounding information detected while being parked when it is determined by an operating state determination unit 27 that the vehicle is parked, and an estimation unit 23 estimates the bearing of the vehicle being parked, on the basis of the recognized bearing that is the extension bearing of the belt-like ground feature based on the recognition result and the individual bearing information of the belt-like ground features included in ground feature information.SELECTED DRAWING: Figure 2

Description

The present invention relates to a vehicle state estimation system that estimates the direction of a vehicle that automatically travels toward a set destination before starting, a vehicle state estimation method, and a vehicle state estimation program.

In recent years, a technique for realizing automatic valley parking in which vehicles are automatically loaded and unloaded in a parking lot has been studied. At the time of entry and exit, the vehicle is configured to go to the destination by automatically traveling.

JP-A-2015-41348

In order to realize the above-mentioned automatic valley parking, it is important to accurately grasp the initial position and initial direction of the vehicle at the starting point of automatic driving when the vehicle enters and exits the garage.

In view of the above situation, when the vehicle is automatically driven in the parking lot, it is required to realize a technique capable of appropriately estimating at least one of the initial direction and the initial position of the vehicle at the starting point of the automatic driving. There is.

The characteristic configuration of the vehicle condition estimation system in view of the above is
It is a vehicle condition estimation system that estimates the direction of a vehicle that automatically travels toward a set destination before starting in a parking lot.
An operation state determination unit that determines the operation state of the vehicle,
A feature information acquisition unit that acquires feature information including the location information of features provided in the parking lot,
Peripheral information acquisition unit that acquires peripheral information of the vehicle detected by the peripheral detection sensor mounted on the vehicle, and
A recognition unit that performs recognition processing of the feature included in the surrounding information,
An estimation unit for estimating the orientation of the vehicle in the parking lot is provided.
The feature information includes individual orientation information indicating the extending orientation of each of the strip-shaped features provided in the parking lot.
The operating state determination unit determines whether or not the vehicle is stopped in the parking lot, and determines whether or not the vehicle is stopped.
When the operating state determination unit determines that the vehicle is stopped, the recognition unit recognizes the strip-shaped feature among the features included in the peripheral information detected while the vehicle is stopped. And
The estimation unit is the stopped vehicle based on the recognition direction which is the extension direction of the strip-shaped feature based on the recognition result and the individual direction information of the strip-shaped feature included in the feature information. It is at the point of estimating the direction of.

Further, in view of the above, the technical features of the vehicle state estimation system can also be applied to a vehicle state estimation method and a vehicle state estimation program, and therefore, the present invention also covers such methods and programs. can do.

In that case, the characteristic configuration of the vehicle state estimation method is
It is a vehicle condition estimation method that estimates the direction of a vehicle that automatically travels toward a set destination before starting in a parking lot.
The operation state determination step for determining the operation state of the vehicle and
A feature information acquisition step for acquiring feature information including the position information of the feature provided in the parking lot, and
A peripheral information acquisition step for acquiring peripheral information of the vehicle detected by a peripheral detection sensor mounted on the vehicle, and
A recognition step for performing recognition processing of the feature included in the surrounding information, and
It comprises an estimation step of estimating the orientation of the vehicle in the parking lot.
The feature information includes individual orientation information indicating the extending orientation of each of the strip-shaped features provided in the parking lot.
In the operation state determination step, it is determined whether or not the vehicle is stopped in the parking lot.
In the recognition step, when it is determined by the operation state determination step that the vehicle is stopped, the strip-shaped feature is recognized among the features included in the peripheral information detected while the vehicle is stopped. And
In the estimation step, the stopped vehicle is based on the recognition direction which is the extension direction of the strip-shaped feature based on the recognition result and the individual direction information of the strip-shaped feature included in the feature information. It is at the point of estimating the direction of.

In that case, the feature configuration of the vehicle condition estimation program is
It is a vehicle condition estimation program that estimates the direction of a vehicle that automatically travels toward a set destination before starting in the parking lot.
The operation state determination function for determining the operation state of the vehicle and
A feature information acquisition function that acquires feature information including the location information of features provided in the parking lot, and
Peripheral information acquisition function that acquires peripheral information of the vehicle detected by the peripheral detection sensor mounted on the vehicle, and
A recognition function that performs recognition processing of the feature included in the surrounding information,
A computer is realized with an estimation function for estimating the orientation of the vehicle in the parking lot.
The feature information includes individual orientation information indicating the extending orientation of each of the strip-shaped features provided in the parking lot.
The operating state determination function determines whether or not the vehicle is stopped in the parking lot, and determines whether or not the vehicle is stopped.
In the recognition function, when the vehicle is determined to be stopped by the operation state determination function, the strip-shaped feature is recognized among the features included in the peripheral information detected while the vehicle is stopped. And
In the estimation function, the vehicle is stopped based on the recognition direction which is the extension direction of the strip-shaped feature based on the recognition result and the individual direction information of the strip-shaped feature included in the feature information. It is at the point of estimating the direction of.

According to these configurations, when the vehicle is stopped, the orientation of the vehicle is based on the recognition orientation of the belt-shaped feature based on the recognition result of the belt-shaped feature and the individual orientation information included in the feature information. Can be estimated appropriately. Therefore, according to this configuration, it is possible to appropriately estimate the direction before the vehicle starts when the vehicle is automatically driven in the parking lot, in other words, the initial direction of the vehicle at the starting point of the automatic driving.

In addition, other feature configurations of the vehicle condition estimation system are
It is a vehicle condition estimation system that estimates the position of a vehicle that automatically travels toward a set destination before starting in a parking lot.
An operation state determination unit that determines the operation state of the vehicle,
A feature information acquisition unit that acquires feature information including the location information of features provided in the parking lot,
Peripheral information acquisition unit that acquires peripheral information of the vehicle detected by the peripheral detection sensor mounted on the vehicle, and
A recognition unit that performs recognition processing of the feature included in the surrounding information,
An estimation unit for estimating the position of the vehicle in the parking lot is provided.
The operating state determination unit determines whether or not the vehicle is stopped in the parking lot, and determines whether or not the vehicle is stopped.
The recognition unit is installed in the parking lot among the features included in the peripheral information detected while the vehicle is stopped when the operation state determination unit determines that the vehicle is stopped. Recognize the identification signs provided for each of the multiple parking spaces
The estimation unit estimates the position of the stopped vehicle based on the recognition position which is the position of the identification mark based on the recognition result and the position information of the identification mark included in the feature information. It is in.

In that case, the characteristic configuration of the vehicle state estimation method is
It is a vehicle condition estimation method that estimates the position of a vehicle that automatically travels toward a set destination before starting in a parking lot.
The operation state determination step for determining the operation state of the vehicle and
A feature information acquisition step for acquiring feature information including the position information of the feature provided in the parking lot, and
A peripheral information acquisition step for acquiring peripheral information of the vehicle detected by a peripheral detection sensor mounted on the vehicle, and
A recognition step for performing recognition processing of the feature included in the surrounding information, and
It comprises an estimation step of estimating the position of the vehicle in the parking lot.
In the operation state determination step, it is determined whether or not the vehicle is stopped in the parking lot.
In the recognition step, when it is determined that the vehicle is stopped in the operation state determination step, it is installed in the parking lot among the features included in the peripheral information detected while the vehicle is stopped. Recognize the identification signs provided for each of the multiple parking spaces
In the estimation step, a point of estimating the position of the stopped vehicle based on the recognition position which is the position of the identification mark based on the recognition result and the position information of the identification mark included in the feature information. It is in.

In that case, the feature configuration of the vehicle condition estimation program is
It is a vehicle condition estimation program that estimates the position of a vehicle that automatically travels toward a set destination before starting in a parking lot.
The operation state function for determining the operation state of the vehicle and
A feature information acquisition function that acquires feature information including the location information of features provided in the parking lot, and
Peripheral information acquisition function that acquires peripheral information of the vehicle detected by the peripheral detection sensor mounted on the vehicle, and
A recognition function that performs recognition processing of the feature included in the surrounding information,
A computer is realized with an estimation function for estimating the position of the vehicle in the parking lot.
The operating state determination function determines whether or not the vehicle is stopped in the parking lot, and determines whether or not the vehicle is stopped.
In the recognition function, when the vehicle is determined to be stopped by the operation state determination function, it is installed in the parking lot among the features included in the peripheral information detected while the vehicle is stopped. Recognize the identification signs provided for each of the multiple parking spaces
The estimation function estimates the position of the stopped vehicle based on the recognition position which is the position of the identification mark based on the recognition result and the position information of the identification mark included in the feature information. It is in.

According to these configurations, when the vehicle is stopped, the recognition position of the identification mark based on the recognition result of the identification mark provided corresponding to each of the plurality of parking spaces installed in the parking lot, and the recognition position of the identification mark. The position of the vehicle can be appropriately estimated based on the position information of the identification mark included in the feature information. Therefore, according to these configurations, it is possible to appropriately estimate the position before the vehicle starts when the vehicle is automatically driven in the parking lot, in other words, the initial position of the vehicle which is the starting point of the automatic driving.

Further features and advantages of the techniques according to the present disclosure will be further clarified by the following illustration of exemplary and non-limiting embodiments described with reference to the drawings.

Diagram showing an example of a parking lot to which a vehicle condition estimation system is applied Block diagram showing the system configuration of the vehicle state estimation system A plan view showing a state in which the vehicle is stopped at the initial position which is the starting point of automatic driving. Explanatory diagram of the process of estimating the position of a stopped vehicle Explanatory diagram of the process of estimating the direction of a stopped vehicle Flow chart showing the control procedure when estimating the position and orientation of the vehicle Top view showing the presence of other parked vehicles An explanatory diagram of a process of estimating the direction of a stopped vehicle in the second embodiment.

[First Embodiment]
The vehicle state estimation system (vehicle state estimation method, vehicle state estimation program) according to the first embodiment will be described with reference to the drawings. As shown in FIG. 1, the vehicle state estimation system 100 is applied to an automatic valley parking system that automatically enters and exits the vehicle 2 in the parking lot 1.

[Overview of automatic valley parking system]
First, the outline of the automatic valley parking system will be briefly explained.

FIG. 1 shows an example of a parking lot 1 to which an automatic valley parking system is applied. In the parking lot 1, the parking space P3, which is partitioned by the parking frame line L (corresponding to a strip-shaped feature) and serves as a parking place for the vehicle 2, and the occupant 9 disembarks from the vehicle 2 to be stored in the parking space P3. There is provided a disembarkation space P1 and a boarding space P2 in which the occupant 9 rides on the vehicle 2 to be dismissed from the parking space P3.

In the illustrated example, one getting-off space P1 and one getting-on space P2 are arranged, but a plurality of these may be arranged. Further, a plurality of parking spaces P3 are arranged in the parking lot 1, and a plurality of parking spaces P3 are arranged side by side. More specifically, in the parking lot 1, a plurality of parking areas PAs in which a plurality of parking spaces P3 are arranged are provided at intervals from each other. An area (road) on which the vehicle 2 can travel is formed between the plurality of parking areas PAs.

Although not shown, the vehicle 2 automatically travels from the disembarkation space P1 to the empty parking space P3 in response to the warehousing instruction after the occupant 9 disembarks from the vehicle 2 in the disembarkation space P1 at the time of warehousing. The vehicle automatically stops (parks) in the parking space P3. Then, as shown in FIG. 1, at the time of leaving the garage, the vehicle 2 automatically travels from the parking space P3 to the boarding space P2 and automatically stops at the boarding space P2 in response to the warehousing instruction. The warehousing instruction and the warehousing instruction are realized, for example, by the operation of the terminal device 3 by the occupant 9. The terminal device 3 may be a mobile terminal carried by the occupant 9, or may be a fixed terminal installed in the parking lot 1.

[Vehicle condition estimation system configuration]
In such an automatic valley parking system, the vehicle state estimation system 100 estimates the position and orientation of the vehicle 2 that automatically travels toward the set destination P in the parking lot 1 before starting. In the present embodiment, the "destination P" is a place designated by the control device 10 that manages the parking lot 1. At the time of warehousing, an empty parking space P3 (if there are a plurality of empty parking spaces P3, any one of them) is designated as the destination P. At the time of delivery, the boarding space P2 is designated as the destination P. For example, when the boarding space P2 is in use and the disembarking space P1 is vacant, the disembarking space P1 may be designated as the destination P at the time of delivery. When there are a plurality of empty parking spaces P3, the occupant 9 may specify one of them by operating the terminal device 3. Further, FIG. 1 shows an example in which one disembarkation space P1 and one boarding space P2 are provided, but when a plurality of disembarkation space P1 and one boarding space P2 are provided, each of them Either of the plurality of disembarkation spaces P1 and the plurality of boarding spaces P2 may be designated as the destination P at the time of delivery. Alternatively, the same one space may be shared by the getting-off space P1 and the boarding space P2.

The vehicle state estimation method is a method of estimating the position and orientation of the vehicle 2 before starting by automatic driving by using the hardware and software constituting the vehicle state estimation system 100. The vehicle state estimation program is, for example, a program executed by a computer included in the vehicle state estimation system 100 to realize a vehicle state estimation function.

As shown in FIGS. 1 to 3, the vehicle state estimation system 100 (vehicle state estimation method / vehicle state estimation program) includes a travel control unit 21 (travel control step / travel control function) that controls the travel of the vehicle 2. An operation state determination unit 27 for determining the operation state of the vehicle 2, a route setting unit 13 (route setting step / route setting function) for setting the traveling route R of the vehicle 2 to the destination P, and a parking lot 1 are provided. The feature information acquisition unit 24 (feature information acquisition step / feature information acquisition function) for acquiring feature information including the absolute position information (position information) of the feature O, and the camera C mounted on the vehicle 2. Peripheral image acquisition unit 26 (peripheral image acquisition step / peripheral image acquisition function) that acquires the peripheral image of the vehicle 2 photographed (detected) by (photographing device), and image recognition processing of the feature O included in the peripheral image. It is provided with an image recognition unit 25 (image recognition step / image recognition function) for performing the above, and an estimation unit 23 (estimation step / estimation function) for estimating the position of the vehicle 2 in the parking lot 1. Here, the camera C is an example of the peripheral detection sensor SA. Similarly, the peripheral image is an example of peripheral information, and the image recognition process (image recognition step / image recognition function) is an example of recognition processing (recognition step / recognition function).

Here, the "absolute position information of the feature O" means information indicating the absolute position of the feature O in the parking lot 1. The "absolute position of the feature O" is a position on the fixed coordinates fixed to the ground in the parking lot 1. The fixed coordinates may be coordinates set only in the parking lot 1 or only in a specific area, or may be geographic coordinates represented by latitude and longitude. In the present embodiment, the absolute position of the feature O is specified by using the coordinate system set with an arbitrary reference position in the parking lot 1 as the origin as the absolute coordinate system (see FIG. 4).

In addition to the absolute position information of the feature O, the feature information also includes information such as the type of the feature O. Types of feature O include three-dimensional features and flat features. Three-dimensional features include road signs and curb blocks. Flat features include road markings and other painted features provided on the ground, such as markings on the ground. As shown in FIG. 3, the feature O has a marker M (identification target) provided on the outer ground surface of each parking space P3 corresponding to each of the plurality of parking spaces P3 installed in the parking lot 1. A pair of parking frame lines L (corresponding to a strip-shaped feature) provided on the ground surface in the parking lot 1 and partitioning both sides of the parking space P3 are included. Therefore, in this example, the feature information includes absolute position information (position information) indicating the absolute positions of the marker M and the parking frame line L in the parking lot 1. The marker M may be provided corresponding to each of the plurality of parking spaces P3, and may be provided inside the parking space P3, for example. Alternatively, the marker M may be provided both outside and inside the parking space P3. Further, in this example, the feature information includes individual directional information (direction information) indicating the extending direction of each of the parking frame lines L provided on the ground surface in the parking lot 1. Here, the extending direction of the parking frame line L, that is, the "extending direction of the feature O" is the direction on the fixed coordinates fixed to the ground in the parking lot 1. In the following, the extending direction of the feature O may be referred to as "absolute direction of the feature O". In addition to the above, the feature information may include at least one of shape information indicating the shape of the feature O and color information indicating the color of the feature O. Further, the parking frame line L is a band-shaped sign provided on the ground surface in the parking lot 1, and is an example of a band-shaped feature.

In the present embodiment, the vehicle state estimation system 100 includes a control device 10 that controls the entire parking lot 1 and a vehicle control device 20 that controls the vehicle 2. Each of the control device 10 and the vehicle control device 20 includes a processor (system LSI, DSP (Digital Signal Processor), etc.) and an arithmetic processing unit configured as an ECU (Electronic Control Unit).

The control device 10 is installed in the parking lot 1 and is configured to be communicative with the vehicle control device 20. The control device 10 includes a parking lot management unit 11 that manages the availability of the parking space P3 in the parking lot 1, the map information of the parking lot 1, the type of the feature O provided in the parking lot 1, and its absolute position. It includes a data storage unit 12 for storing feature information such as information (absolute position data). In this example, the above-mentioned route setting unit 13 is provided in the control device 10. However, without being limited to such a configuration, at least one of the data storage unit 12 and the route setting unit 13 may be provided in the vehicle control device 20.

The parking lot management unit 11 manages the availability of the parking space P3 by a photographing device (not shown) for photographing the parking space P3 and a sensor (not shown) for detecting the presence or absence of the vehicle 2 in the parking space P3. In addition, the parking lot management unit 11 manages information that identifies the vehicle 2 that is the target of the warehousing instruction and the vehicle 2 that is the target of the warehousing instruction, for example, identification information and position information of each vehicle 2.

The data storage unit 12 stores feature information including information such as the absolute position of the feature O and the type of the feature O. In the present embodiment, the data storage unit 12 stores individual direction information (direction information) indicating the extending direction for each of the plurality of parking borders L as feature information. The data storage unit 12 includes, for example, a volatile main storage device, a non-volatile auxiliary storage device, and the like.

The route setting unit 13 sets the travel route R of the vehicle 2 to the destination P. When the warehousing instruction is given, the route setting unit 13 sets the traveling route R from the disembarkation space P1 where the vehicle 2 subject to the warehousing instruction is parked to the parking space P3 designated as the warehousing destination. To do. When the exit instruction is given, the route setting unit 13 sets the travel route R from the parking space P3 to the boarding space P2 in which the vehicle 2 (parked vehicle) subject to the exit instruction is parked.

The vehicle control device 20 is installed in the vehicle 2 and is configured to be communicative with the control device 10. The vehicle control device 20 includes a vehicle position information acquisition unit 22 that acquires vehicle position information regarding the position of the vehicle. In this example, the above-mentioned traveling control unit 21, operation state determination unit 27, feature information acquisition unit 24, peripheral image acquisition unit 26, image recognition unit 25, and estimation unit 23 are provided in the vehicle control device 20. There is.

The travel control unit 21 controls traveling, steering, shifting, stopping, and the like of the vehicle 2 based on the detection results of various sensors (not shown). The travel control unit 21 includes information on the travel route R to the destination P acquired from the route setting unit 13, the vehicle position information acquired by the vehicle position information acquisition unit 22, and peripheral image information captured by the camera C. Based on the above, the operation of the vehicle 2 is controlled until the destination P is reached.

The own vehicle position information acquisition unit 22 detects the GPS receiver G mounted on the vehicle 2, the orientation sensor S1 (for example, a gyro sensor that detects the turning angle of the vehicle 2), and the rotation sensor S2 (for example, the rotation speed of the wheels). Use a wheel speed sensor) or the like to acquire vehicle position information.

The operation state determination unit 27 determines the operation state of the vehicle 2 such as running, steering, shifting, and stopping. Here, the operation state determination unit 27 determines whether or not the vehicle 2 is stopped in the parking lot 1. Thereby, at least, it can be determined whether or not the vehicle 2 is in the state before the start by the automatic traveling. The operation state determination unit 27 determines the operation state of the vehicle 2 by using various sensors such as the GPS receiver G, the direction sensor S1, and the rotation sensor S2. However, without being limited to such a configuration, the operating state determination unit 27 cooperates with the vehicle position information acquisition unit 22 that acquires the vehicle position information using the above-mentioned various sensors, and the vehicle itself. The operating state of the vehicle 2 may be determined based on the information obtained from the position information acquisition unit 22, or the vehicle 2 may be determined based on the control state of the vehicle 2 by the travel control unit 21 in cooperation with the travel control unit 21. The operating state of the above may be determined.

The feature information acquisition unit 24 acquires feature information from the data storage unit 12 of the control device 10. As described above, the feature information includes absolute position information, type information, orientation information, and the like of the feature O.

The peripheral image acquisition unit 26 acquires a peripheral image of the vehicle 2 taken by the camera C mounted on the vehicle 2. As shown in FIG. 3, in the present embodiment, the front side and the left and right sides of the vehicle 2 are photographed by the camera C with reference to the traveling direction of the vehicle 2. The area indicated by the virtual line (dashed-dotted line) in FIG. 3 is the shooting area CA (detection area) of the camera C. In the present embodiment, the camera C includes a front camera for photographing the front of the vehicle 2 and a side camera for photographing the side of the vehicle 2. The vehicle 2 may also be equipped with a front camera for photographing the front, a back camera for photographing the rear, and the like, and these may be used as the camera C.

As shown in FIG. 3, in the present embodiment, when the vehicle 2 is stopped at the target space PT, which is one of the parking spaces P3, the photographing area CA by the camera C is the marker M (corresponding to the target space PT). In the illustrated example, the second marker M2) and the markers M corresponding to each of the pair of parking spaces P3 adjacent to the target space PT (in the illustrated example, the first marker M1 and the third marker M3) may be included. It is set in the range. Further, in this example, while the vehicle 2 is stopped at the target space PT, the shooting area CA by the camera C is a pair of parking frame lines L that divide both sides of the target space PT (in the illustrated example, the first parking frame). The range is set so as to include the line L1 and the second parking frame line L2). Here, for convenience of explanation, in order to distinguish from other parking spaces P3, the parking space P3 in which the vehicle 2 before starting by automatic running is stopped is referred to as a target space PT among the plurality of parking spaces P3. However, there is no structural difference between the target space PT and the other parking space P3.

The image recognition unit 25 performs image recognition processing of the feature O included in the peripheral image from the peripheral image acquired by the peripheral image acquisition unit 26. Specifically, the image recognition unit 25 recognizes at least one of the position, type, and orientation of the recognized feature O in the peripheral image. In the present embodiment, the image recognition unit 25 performs image recognition processing on a plurality of markers M that can be included in the shooting area CA, and recognizes at least an absolute position (position) of each of the plurality of marker Ms. .. Further, the image recognition unit 25 performs image recognition processing on a pair of parking borders L that can be included in the shooting area CA, and each of the pair of parking borders L has at least an absolute orientation (extended orientation). Recognize.

The estimation unit 23 estimates the position of the vehicle 2 in the parking lot 1, more specifically, the position of the vehicle 2 stopped at any position in the parking lot 1. The "position of the vehicle 2" is specified with reference to a predetermined position such as the center position of the vehicle 2 in a plan view. The position indicated by (Xf, Yf) in FIG. 4 is the position of the vehicle 2. Further, the estimation unit 23 estimates the direction of the vehicle 2 in the parking lot 1, more specifically, the direction of the vehicle 2 stopped at any place in the parking lot 1. The "direction of the vehicle 2" is specified with reference to a specified direction such as the front-rear direction and the width direction of the vehicle 2. The direction indicated by "Df" in FIG. 5 is the direction of the vehicle 2, and in this example, the front-rear direction of the vehicle 2 is the direction Df of the vehicle 2.

[Estimation of vehicle position]
Hereinafter, the estimation of the vehicle position by the vehicle state estimation system 100 will be described in detail. FIG. 4 is an explanatory diagram of a process of estimating the position (Xf, Yf) of the stopped vehicle 2.

In the vehicle state estimation system 100, the image recognition unit 25 is among the features O included in the peripheral image taken while the vehicle is stopped when the operation state determination unit 27 determines that the vehicle 2 is stopped. , Marker M (identification target feature) is recognized. Then, the estimation unit 23 is a stopped vehicle based on the recognition position (X', Y') which is the position of the marker M based on the image recognition result and the absolute position information of the marker M included in the feature information. Estimate the position of 2 (Xf, Yf). In FIG. 4, the position of the marker M indicated by the virtual line (dashed line) is the recognition position (X', Y') of the marker M based on the image recognition result, and the position of the marker M indicated by the solid line is It is the absolute position (X, Y) of the marker M based on the absolute position information. In the present embodiment, the estimation unit 23 calculates an error by matching the recognition position (X', Y') about the marker M with the absolute position (X, Y) about the marker M, and based on this error. The position (Xf, Yf) of the vehicle 2 is corrected. Then, the corrected position of the vehicle 2 (Xf, Yf) is estimated to be the position of the stopped vehicle 2 (Xf, Yf).

FIG. 3 shows a state in which the vehicle 2 is stopped in the target space PT, and the image recognition unit 25 sets the first marker M1, the second marker M2, and the third marker M3 included in the photographing area CA. recognize. Then, as shown in FIG. 4, the estimation unit 23 sets the first recognition position (X1', Y1') for each of the first marker M1, the second marker M2, and the third marker M3 based on the image recognition result. , The second recognition position (X2', Y2'), and the third recognition position (X3', Y3') are acquired (calculated). Further, the estimation unit 23 sets the first absolute position (X1, Y1) and the second absolute position (X2) for each of the first marker M1, the second marker M2, and the third marker M3 based on the absolute position information. Y2) and the third absolute position (X3, Y3) are acquired.

In the present embodiment, the image recognition unit 25 recognizes a plurality of markers M1 to M3 included in the peripheral image, and the recognition positions (X1', Y1') of each of the plurality of markers M1 to M3 are ~. There is a difference between the relative positional relationship of the plurality of markers M1 to M3 based on (X3', Y3') and the relative positional relationship of the plurality of markers M1 to M3 based on the absolute position information of the markers M included in the feature information. In this case, the estimation unit 23 has the recognition positions (X1', Y1') to (X3', Y3') for each of the plurality of markers M1 to M3 and the absolute positions (X1, Y1) shown in the absolute position information. Recognizing positions (X1', Y1') to (X3', Y3') of the plurality of markers M1 to M3 so that the deviation from to (X3, Y3) is the same for the plurality of markers M1 to M3. Is adapted to the absolute positions (X1, Y1) to (X3, Y3) shown in the absolute position information, and the recognition positions (X1', Y1') to (X3', Y3) of the plurality of markers M1 to M3 after the adaptation are adapted. The position (Xf, Yf) of the vehicle 2 with reference to') is estimated to be the position (Xf, Yf) of the vehicle 2.

The estimation unit 23 is based on, for example, a pair of markers M (for example, the first marker M1 and the second marker M2), and the relative positional relationship based on the recognition position (X', Y') based on the image recognition result and the absolute position information. It is determined whether or not there is a difference from the relative positional relationship based on the absolute position (X, Y). As an example, regarding the relative positional relationship between the first marker M1 and the second marker M2 based on the recognition position (X', Y'), the distance between these two points is "2.2 meters" and the absolute position (X). , Y), consider the case where the distance between these two points is "2 meters" with respect to the relative positional relationship between the first marker M1 and the second marker M2. In this case, the estimation unit 23 has a difference between the "relative positional relationship based on the recognition position (X', Y') based on the image recognition result and the relative positional relationship based on the absolute position (X, Y) based on the absolute position information. Is determined. Then, the estimation unit 23 calculates the deviation (difference in the distance between the two points) of these two relative positional relationships. In the above case, this deviation is "0.2 meters". Then, the estimation unit 23 deviates from the recognition position (X', Y') and the absolute position (X, Y) for each of the first marker M1 and the second marker M2 based on the deviation of the relative positional relationship. The positions of the vehicle 2 are corrected by adjusting the respective positions so that the amounts are the same, that is, "0.1 meters each". In other words, the position (Xf, Yf) of the vehicle 2 with respect to the recognition position (X', Y') of the marker M after conforming is estimated to be the position (Xf, Yf) of the vehicle 2. In the above description, an example of estimating the position (Xf, Yf) of the vehicle 2 based on the relative positional relationship between the first marker M1 and the second marker M2 has been described, but the first marker M1 to the third marker M3 The position (Xf, Yf) of the vehicle 2 may be estimated based on the relative positional relationship of any two of them. Alternatively, the position (Xf, Yf) of the vehicle 2 may be estimated based on all the relative positional relationships of the first markers M1 to the third marker M3.

Here, if the difference between the relative positional relationship based on the recognition position (X', Y') and the relative positional relationship based on the absolute position (X, Y) is too large, the image recognition process is appropriately performed. It may not be. When the position (Xf, Yf) of the vehicle 2 is estimated under such a situation, the position where the vehicle 2 is actually stopped may be significantly different from the estimated position of the vehicle 2. Therefore, in the present embodiment, the estimation unit 23 has a relative positional relationship based on the recognition position (X', Y') based on the image recognition result and a relative positional relationship based on the absolute position (X, Y) based on the absolute position information. If the difference (deviation) is greater than or equal to the specified value (greater than or equal to the specified value), it is determined that the recognition position (X', Y') and the absolute position (X, Y) cannot be matched, and the vehicle 2 It is configured so that the position (Xf, Yf) is not estimated. As a result, it is possible to prevent incorrect estimation from being performed. The above "specified value" may be set based on, for example, the width of the parking space P3. For example, the "specified value" may be set to "2.5 meters to 3.5 meters".

[Estimation of vehicle orientation]
Next, the estimation of the vehicle orientation by the vehicle state estimation system 100 will be described in detail. FIG. 5 is an explanatory diagram of a process of estimating the direction Df of the stopped vehicle 2.

In the vehicle state estimation system 100, the image recognition unit 25 is among the features O included in the peripheral image taken while the vehicle is stopped when the operation state determination unit 27 determines that the vehicle 2 is stopped. , Recognize parking border L (belt-shaped feature). Then, the estimation unit 23 is stopped based on the recognition direction D'which is the extension direction of the parking frame line L based on the image recognition result and the individual direction information of the parking frame line L included in the feature information. The directional Df of the vehicle 2 is estimated. In FIG. 5, the extending direction of the parking frame line L indicated by the virtual line (dashed-dotted line) is the recognition direction D'of the parking frame line L based on the image recognition result, and the parking frame line L shown by the solid line. The extending direction is the absolute direction D of the parking frame line L based on the individual direction information. In the present embodiment, the estimation unit 23 calculates an error by adapting the recognition direction D'for the parking frame line L to the absolute direction D for the parking frame line L, and the direction of the vehicle 2 is based on this error. Correct Df. Then, the corrected directional Df of the vehicle 2 is estimated to be the directional Df of the stopped vehicle 2.

FIG. 3 shows a state in which the vehicle 2 is stopped in the target space PT, and the image recognition unit 25 recognizes the first parking frame line L1 and the second parking frame line L2 included in the shooting area CA. .. Then, as shown in FIG. 5, the estimation unit 23 sets the first recognition direction D1'and the second recognition direction D2' for each of the first parking frame line L1 and the second parking frame line L2 based on the image recognition result. Is acquired (calculated). Further, the estimation unit 23 acquires the first absolute direction D1 and the second absolute direction D2 for each of the first parking frame line L1 and the second parking frame line L2 based on the individual direction information.

In the present embodiment, the estimation unit 23 estimates the direction Df of the stopped vehicle 2 for any one of the pair of parking borders L1 and L2 recognized by the image recognition process. For example, when the first parking frame line L1 is targeted, the first recognition direction D1'for the first parking frame line L1 is adapted to the first absolute direction D1 for the first parking frame line L1. The direction Df of the vehicle 2 based on the first recognition direction D1'of the first parking frame line L after conforming is estimated as the direction Df of the vehicle 2. Similarly, the direction Df of the vehicle 2 can be estimated when the second parking frame line L2 is targeted. Further, without being limited to the above configuration, for example, the estimation unit 23 targets both the pair of parking frame lines L1 and L2, and has absolute directions D1 and D2 for the respective recognition directions D1'and D2'. The first recognition direction D1'is based on the average value of the errors, that is, so that the errors between the respective recognition directions D1'and D2'and the absolute directions D1 and D2 are the same amount. And the second recognition direction D2'may be corrected by adapting to the absolute directions D1 and D2, respectively.

[Control procedure]
Next, a control procedure for estimating the position (Xf, Yf) and the direction Df of the automatically traveling vehicle 2 before starting will be described with reference to FIG.

First, the vehicle state estimation system 100 determines whether or not the vehicle 2 is stopped (# 1). When the vehicle state estimation system 100 determines that the vehicle 2 is not stopped (# 1: No), it determines that the vehicle 2 is running and ends the process. When the vehicle state estimation system 100 determines that the vehicle 2 is stopped (# 1: Yes), the vehicle state estimation system 100 acquires an image of the surroundings of the vehicle 2 (# 2). Then, the vehicle state estimation system 100 recognizes the feature O included in the peripheral image (# 3), and acquires the recognition position (X', Y') based on the image recognition result for the recognized feature O. (# 4). Further, the vehicle state estimation system 100 has acquired the recognition position (X', Y') from the absolute position information of the feature O included in the feature information by acquiring the feature information (# 5). The absolute position (X, Y) of the feature O is acquired (# 6).

Then, the vehicle state estimation system 100 can match these based on the recognition position (X', Y') and the absolute position (X, Y) of the feature O obtained in steps 4 and 6 above. It is determined whether or not it is (# 7). Here, as described above, it is suitable when the difference between the relative positional relationship based on the recognition position (X', Y') and the relative positional relationship based on the absolute position (X, Y) is less than the specified value. Judge that it is possible. When the vehicle state estimation system 100 determines that the vehicle cannot be adapted (# 7: No), it notifies an error (# 8) and ends the process. When the vehicle state estimation system 100 determines that it is compatible (# 7: Yes), it acquires the recognition direction D'based on the image recognition result for the feature O recognized in step 3 above (# 9). Further, the vehicle state estimation system 100 acquires the absolute direction D of the feature O for which the recognition direction D'has been acquired from the individual direction information of the feature O included in the feature information (# 10). After that, the vehicle state estimation system 100 adjusts the recognition direction D'to the absolute direction D (# 11) and corrects the recognition direction D'(# 12). Then, the vehicle state estimation system 100 adapts the recognition position (X', Y') to the absolute position (X, Y) (# 13) and corrects the recognition position (X', Y') (# 14). ..

The vehicle state estimation system 100 estimates the position (Xf, Yf) and direction Df of the stopped vehicle 2 by performing the process according to the above procedure. However, the vehicle state estimation system 100 is not limited to the above processing procedure, and the vehicle state estimation system 100 estimates the position (Xf, Yf) of the stopped vehicle 2 and the direction Df of the stopped vehicle 2. The processing may be performed separately, or only one of the processing may be performed.

[Target of image recognition processing]
As described above with reference to FIG. 3, in the vehicle state estimation system 100, when the vehicle 2 is stopped at the target space PT, the shooting area CA by the camera C is the second marker M2 corresponding to the target space PT. The range is set so as to include the first marker M1 and the third marker M3 corresponding to each of the pair of parking spaces P3 adjacent to the target space PT. Further, in this example, the shooting area CA of the camera C is set to a range that can include the first parking frame line L1 and the second parking frame line L2 that partition both sides of the target space PT. Then, the image recognition unit 25 is configured to perform image recognition processing on a plurality of markers M and a pair of parking borders L that can be included in the shooting area CA.

For example, FIG. 7 shows a state in which another parked vehicle 8 exists in the parking space P3 adjacent to the target space PT. In the example shown in FIG. 7, the first marker M1 is hidden by another parked vehicle 8, and the image recognition process for the first marker M1 is difficult (first marker M1: NG). Further, the paint or the like forming the second parking frame line L2 is disappearing due to aged deterioration or the like, and the image recognition processing for the second parking frame line L2 is in a difficult state (second parking). Border L2: NG).

However, as described above, the image recognition unit 25 in the vehicle state estimation system 100 targets a plurality of markers M and a pair of parking borders L that can be included in the shooting area CA, and thus a plurality of markers. Of M and the pair of parking borders L, those capable of image recognition processing can be recognized and used for estimating the position and orientation of the vehicle 2. In the example shown in FIG. 7, the image recognition processing for the second marker M2, the third marker M3, and the first parking frame line L1 is possible (second marker M2, third marker M3, first parking frame). Line L1: OK), the image recognition unit 25 recognizes these. Then, the estimation unit 23 can estimate the position of the vehicle 2 based on the second marker M2 and the third marker M3, and can estimate the direction of the vehicle 2 based on the first parking frame line L1. it can.

[Second Embodiment]
Next, the vehicle state estimation system (vehicle state estimation method, vehicle state estimation program) according to the second embodiment will be described. In the present embodiment, the estimation mode of the directional Df of the vehicle 2 is different from that in the first embodiment. Hereinafter, the differences between the present embodiment and the first embodiment will be mainly described. The points not particularly described are the same as those of the first embodiment, and the same reference numerals as those of the first embodiment are used.

In the present embodiment, the direction Df of the vehicle 2 is not estimated based on the parking frame line L (strip-shaped feature) as shown in FIG. 5, but is based on the marker M (identification target feature) as shown in FIG. The directional Df of the vehicle 2 is estimated.

In the present embodiment, the image recognition unit 25 includes a plurality of markers M (not shown) included in the peripheral image taken while the vehicle is stopped when the operation state determination unit 27 determines that the vehicle 2 is stopped. In the example of, the first marker M1 and the second marker M2) are recognized. Then, the estimation unit 23 includes the recognition inter-feature direction D ″, which is the extending direction of the line connecting the plurality of markers M1 and M2 based on the image recognition result, and the plurality of markers M1 and M2 included in the feature information. The directional Df of the stopped vehicle 2 is estimated based on the absolute position information (position information).

In FIG. 8, the position of the marker M indicated by the virtual line (two-point chain line) is the recognition position (X', Y') of the marker M based on the image recognition result, and the position of the marker M indicated by the solid line is It is the absolute position (X, Y) of the marker M based on the absolute position information. The first absolute position (X1, Y1) of the first marker M1 and the second absolute position (X2, Y2) of the second marker M2 are known from the absolute value information included in the feature information. Therefore, the extending direction of the line connecting the first absolute position (X1, Y1) and the second absolute position (X2, Y2) can be calculated as the absolute direction D. Further, the extending direction of the line connecting the first recognition position (X1', Y1') of the first marker M1 and the second recognition position (X2', Y2') of the second marker M2 based on the image recognition result is also obtained. It can be calculated as the recognized feature-to-feature orientation D ″. Then, the estimation unit 23 adapts the recognized feature-to-feature orientation D ″ to the absolute orientation D, and sets the orientation Df of the vehicle 2 based on the recognized feature-to-object orientation D ″ after the adaptation to the orientation of the vehicle 2. Estimated as Df.

According to this configuration, even when the feature information does not include individual directional information (direction information) indicating the extending direction of the parking frame line L provided on the ground surface in the parking lot 1. , The direction Df of the vehicle 2 can be estimated. Alternatively, even if the feature information includes such individual direction information (direction information), but the parking frame line L (belt-shaped feature) cannot be image-recognized due to aged deterioration or the like, the direction of the vehicle 2 Df can be estimated.

[Other Embodiments]
Next, a vehicle state estimation system, a vehicle state estimation method, and other embodiments of the vehicle state estimation program will be described.

(1) In the above embodiment, a case where the vehicle state estimation system 100 estimates the position of the vehicle 2 in a situation where the vehicle 2 is parked in the parking space P3 has been described as an example. However, the present invention is not limited to such an example, and the identification target area included in the photographed peripheral image regardless of where the vehicle 2 is parked in the parking lot 1 other than the parking space P3. If the object M can be image-recognized, the vehicle state estimation system 100 can estimate the position of the vehicle 2. For example, even if the vehicle 2 is stopped in the disembarkation space P1, if the identification target feature M included in the photographed peripheral image can be recognized as an image, the vehicle state estimation system 100 can use the vehicle 2 The position can be estimated.

(2) In the above embodiment, an example in which the estimation unit 23 estimates both the position (Xf, Yf) of the stopped vehicle 2 and the direction Df of the vehicle 2 has been described. However, without being limited to such an example, the estimation unit 23 may be configured to estimate the direction Df of the vehicle 2 without estimating the position (Xf, Yf) of the stopped vehicle 2. On the contrary, the estimation unit 23 may be configured to estimate the position (Xf, Yf) of the vehicle 2 without estimating the direction Df of the stopped vehicle 2.

(3) In the above embodiment, the estimation unit 23 has a plurality of deviations between the recognition positions (X', Y') for each of the plurality of markers M and the absolute positions (X, Y) shown in the absolute position information. The recognition position (X', Y') of each of the plurality of markers M is adapted to the absolute position (X, Y) shown in the absolute position information so that the same amount is obtained in the marker M of An example of estimating (Xf, Yf) has been described. However, the estimation unit 23 is not limited to such an example, and the estimation unit 23 adapts the recognition position (X', Y') to the absolute position (X, Y) for one marker M. By doing so, the position (Xf, Yf) of the vehicle 2 may be estimated.

(4) In the above embodiment, by photographing the front and left and right sides of the vehicle 2 with the camera C including the front camera and the side camera, a plurality of markers M and a pair of parking lots are parked in the photographing area CA of the camera C. An example in which the border L is included has been described. However, without being limited to such an example, the camera C is composed of one photographing device having a wide-angle photographing function (for example, a function capable of photographing a range of 180 ° or more), whereby a plurality of markers M and a plurality of markers M and A pair of parking borders L may be included in the shooting area CA of the camera C.

(5) In the above embodiment, an example in which a camera C (photographing device) using visible light is used as a peripheral detection sensor has been described. However, the peripheral detection sensor is not limited to such an example, and may be a sensor using invisible light such as infrared rays or ultraviolet rays, electromagnetic waves of various wavelengths, sound waves, or the like. For example, as a peripheral detection sensor, various sensors such as a millimeter wave radar, sonar, and LIDAR (Light Detection and Ranging / Laser Imaging Detection and Ranging) that transmit exploration waves to a predetermined range around the vehicle 2 can be used. Even when a sensor other than the camera C is used as the peripheral detection sensor, the peripheral detection sensor detects the peripheral information of the vehicle 2 regarding the detection target of the sensor. In this case, the feature included in the surrounding information is a feature that can be detected by the peripheral detection sensor. Then, the vehicle state estimation system 100 (vehicle state estimation method / vehicle state estimation program) is a peripheral information acquisition unit (peripheral information acquisition step / peripheral information acquisition function) that acquires peripheral information of the vehicle 2 detected by the peripheral detection sensor. It also includes a recognition unit (recognition step / recognition function) that performs recognition processing of the feature O included in the surrounding information.

(6) In the above embodiment, an example of estimating the position (Xf, Yf) of the vehicle 2 based on the position of the marker M (identification target feature) in the feature O has been described. However, without being limited to such an example, the position (Xf, Yf) of the vehicle 2 may be estimated based on the position of the parking frame line L (belt-shaped feature) in the feature O. In this case, the parking frame line L also functions as an identification target feature.

(7) In the above embodiment, an example in which the parking frame line L (belt-shaped feature) of the feature O has its absolute direction D as individual direction information has been described. However, the present invention is not limited to such an example, and the marker M (identification target feature) of the feature O may have its absolute orientation D as individual orientation information. In this case, the direction Df of the vehicle 2 can be estimated based on one marker M.

(8) In the above embodiment, the parking frame line L has been described as an example of the strip-shaped feature. However, the direction Df of the vehicle 2 may be estimated by using, for example, a temporary stop line or a lane that divides the traveling area of the vehicle 2 as a band-shaped feature without being limited to such an example. Further, in the above, a band-shaped sign is given as an example of the band-shaped feature, and a paint feature such as a temporary stop line or a parking frame line L is exemplified as the band-shaped sign. However, the strip-shaped feature is not limited to such an example, and includes a three-dimensional three-dimensional feature in addition to the flat feature provided on the ground surface. The three-dimensional strip-shaped feature has a strip-shaped (straight line) shape detected by the peripheral detection sensor even if the three-dimensional shape such as a columnar shape or a tubular shape is not a strip-shaped shape. A part of the shape detected by the peripheral detection sensor is strip-shaped (straight). That is, as long as it has a linear portion in a part, it can be a strip-shaped feature even if the other portion is non-linear. For example, three-dimensional features such as guide signs, guardrails, and fences can also be strip-shaped features.

(9) The configurations disclosed in each of the above-described embodiments can be applied in combination with the configurations disclosed in other embodiments as long as there is no contradiction. With respect to other configurations, the embodiments disclosed herein are merely exemplary in all respects. Therefore, various modifications can be made as appropriate without departing from the gist of the present disclosure.

[Outline of the above embodiment]
Hereinafter, the vehicle condition estimation system, the vehicle condition estimation method, and the vehicle condition estimation program described above will be described.

A vehicle state estimation system (100) that estimates the direction of a vehicle (2) that automatically travels toward a set destination (P) in the parking lot (1) before starting.
An operation state determination unit (27) that determines the operation state of the vehicle (2),
The feature information acquisition unit (24) for acquiring feature information including the position information of the feature (O) provided in the parking lot (1), and
Peripheral information acquisition unit (26) that acquires peripheral information of the vehicle (2) detected by the peripheral detection sensor (C) mounted on the vehicle (2), and
A recognition unit (25) that performs recognition processing of the feature (O) included in the surrounding information, and
An estimation unit (23) for estimating the orientation of the vehicle (2) in the parking lot (1) is provided.
The feature information includes individual orientation information indicating the extending orientation of each of the strip-shaped features (L) provided in the parking lot (1).
The operating state determination unit (27) determines whether or not the vehicle (2) is stopped in the parking lot (1).
When the operating state determination unit (27) determines that the vehicle (2) is stopped, the recognition unit (25) includes the feature included in the peripheral information detected while the vehicle is stopped. Of (O), the strip-shaped feature (L) is recognized and
The estimation unit (23) has a recognition direction (D') which is the extension direction of the strip-shaped feature (L) based on the recognition result, and the strip-shaped feature (L) included in the feature information. The direction (Df) of the stopped vehicle (2) is estimated based on the individual direction information.

According to this configuration, when the vehicle is stopped, the recognition direction (D') of the band-shaped feature (L) based on the recognition result of the band-shaped feature (L) and the individual direction included in the feature information. The orientation (Df) of the vehicle (2) can be appropriately estimated based on the information. Therefore, according to this configuration, the direction (Df) before the start of the vehicle (2) in the case of automatic driving in the parking lot (1), in other words, the initial stage of the vehicle (2) at the starting point of the automatic driving. The orientation (Df) can be estimated appropriately.

here,
When the operating state determination unit (27) determines that the vehicle (2) is stopped, the recognition unit (25) includes the feature included in the peripheral information detected while the vehicle is stopped. Among (O), the identification target feature (M) provided corresponding to each of the plurality of parking spaces (P3) installed in the parking lot (1) is recognized.
The estimation unit (23) has a recognition position (X', Y') which is a position of the identification target feature (M) based on the recognition result, and the identification target feature (M) included in the feature information. It is preferable to estimate the position (Xf, Yf) of the stopped vehicle (2) based on the position information of the above.

According to this configuration, when the vehicle (2) is stopped, the identification target feature (M) provided corresponding to each of the plurality of parking spaces (P3) installed in the parking lot (1). ) Based on the recognition position (X', Y') of the identification target feature (M) and the position information of the identification target feature (M) included in the feature information, the vehicle (2). ) Position can be estimated appropriately. Therefore, according to this configuration, the position before the start of the vehicle (2) in the case of automatic driving in the parking lot (1), in other words, the initial position of the vehicle (2) which is the starting point of the automatic driving (2). Xf, Yf) can be estimated appropriately.

Also,
When the operation state determination unit (27) determines that the vehicle (2) is stopped, the recognition unit (25) includes a plurality of the recognition units (25) included in the peripheral information detected while the vehicle is stopped. Recognize the identification target feature (M) and
The estimation unit (23) is included in the recognition feature-to-feature orientation (D ″), which is the extending direction of the line connecting the plurality of identification target features (M) based on the recognition result, and the feature information. It is preferable to estimate the direction (Df) of the stopped vehicle (2) based on the position information of the plurality of identification target features (M).

According to this configuration, when the vehicle is stopped, the recognition location is the extending direction of the line connecting the plurality of identification target features (M) based on the recognition results of the plurality of identification target features (M). The direction (Df) of the vehicle (2) can be appropriately estimated based on the inter-object direction (D ″) and the position information of a plurality of identification target features (M) included in the feature information. .. Therefore, according to this configuration, in addition to the position (Xf, Yf) before the start of the vehicle (2) in the case of automatic driving in the parking lot (1), the direction (Df) before the start of the vehicle (2). ), In other words, the initial direction (Df) of the vehicle (2) at the starting point of automatic driving can be appropriately estimated.

Also,
When the recognition unit (25) recognizes a plurality of the identification target features (M) included in the peripheral information, the recognition positions of the plurality of identification target features (M) (M) The relative positional relationship of the plurality of identification target features (M) based on X', Y') and the plurality of identification targets based on the position information of the identification target feature (M) included in the feature information. If there is a difference from the relative positional relationship of the feature (M),
The estimation unit (23) has a plurality of deviations between the recognition positions (X', Y') and the positions (X, Y) indicated in the position information for each of the plurality of identification target features (M). The recognition positions (X', Y') of each of the plurality of identification target features (M) are indicated in the position information so that the same amount is obtained in the identification target feature (M). , Y), and the position of the vehicle (2) with respect to the recognition position (X', Y') of the plurality of identification target features (M) after the conformation is set to the position of the vehicle (, Y). It is preferable to estimate it as Xf, Yf).

An error is often included in the recognition position (X', Y') (relative position of the feature with respect to the peripheral detection sensor) of the feature (O) based on the recognition result by the recognition unit (25). Therefore, when the position (Xf, Yf) of the vehicle (2) is estimated based on the recognition position (X', Y') for one identification target feature (M), the estimated position (Xf) of the vehicle (2) is estimated. , Yf) may contain a relatively large error. However, according to this configuration, the deviation between the recognition position (X', Y') for each of the plurality of identification target features (M) and the position (X, Y) indicated in the position information is a plurality of identification targets. The recognition positions (X', Y') of each of the plurality of identification target features (M) are adapted to the positions (X, Y) indicated in the position information so that the same amount is obtained in the feature (M). Based on this, the positions (Xf, Yf) of the vehicle (2) are estimated. Therefore, it is possible to average the errors of the recognition positions (X', Y') for each of the plurality of identification target features (M). As a result, it is possible to reduce the influence of the error of the recognition position (X', Y') for each identification target feature (M) and estimate the vehicle position (Xf, Yf) with higher accuracy. ..

Also,
A plurality of the parking spaces (P3) are arranged side by side,
While the vehicle (2) is parked in the target space (PT), which is one of the parking spaces (P3), the detection area (CA) by the peripheral detection sensor (C) is the target space (PT). In a range that can include the identification target feature (M) corresponding to the above and the identification target feature (M) corresponding to each of the pair of parking spaces (P3) adjacent to the target space (PT). Set,
It is preferable that the recognition unit (25) performs the recognition process on a plurality of the identification target features (M) that can be included in the detection area (CA).

According to this configuration, a relatively wide range that can include a plurality of identification target features (M) can be included in the detection area (CA), and the plurality of identification target features (M) can be included in the recognition process. can do. Therefore, for example, even under a situation where a part of the plurality of identification target features (M) is hidden in the shadow or difficult to see due to aged deterioration or the like, the identification target feature (25) by the recognition unit (25) It is possible to realize a configuration in which the recognition result of M) can be easily obtained.

Also,
The feature (O) includes a pair of strip-shaped features (L) provided in the parking lot (1) and partitioning both sides of the parking space (P3).
While the vehicle (2) is parked in the parking space (P3), the detection area (CA) by the peripheral detection sensor (C) divides both sides of the parking space (P3) into a pair of strips. Set in a range that can include the object (L),
It is preferable that the recognition unit (25) performs the recognition process on the pair of strip-shaped features (L) that can be included in the detection area (CA).

According to this configuration, a relatively wide range that can include a pair of strip-shaped features (L) can be included in the detection area (CA), and the pair of strip-shaped features (L) are targeted for recognition processing. Can be done. Therefore, for example, even under a situation where one of the pair of strip-shaped features (L) is hidden in the shadow or difficult to see due to aged deterioration or the like, the strip-shaped feature (L) by the recognition unit (25) It is possible to realize a configuration that makes it easy to obtain the recognition result of.

Further, as a vehicle state estimation system (100) having another characteristic configuration,
A vehicle state estimation system (100) that estimates the position of a vehicle (2) that automatically travels toward a set destination (P) in the parking lot (1) before starting.
An operation state determination unit (27) that determines the operation state of the vehicle (2),
The feature information acquisition unit (24) for acquiring feature information including the position information of the feature (O) provided in the parking lot (1), and
Peripheral information acquisition unit (26) that acquires peripheral information of the vehicle (2) detected by the peripheral detection sensor (C) mounted on the vehicle (2), and
A recognition unit (25) that performs recognition processing of the feature (O) included in the surrounding information, and
It is provided with an estimation unit (23) for estimating the position of the vehicle (2) in the parking lot (1).
The operating state determination unit (27) determines whether or not the vehicle (2) is stopped in the parking lot (1).
When the operating state determination unit (27) determines that the vehicle (2) is stopped, the recognition unit (25) includes the feature included in the peripheral information detected while the vehicle is stopped. Among (O), the identification mark (M) provided corresponding to each of the plurality of parking spaces (P3) installed in the parking lot (1) is recognized as an identification target feature.
The estimation unit (23) has a recognition position (X', Y') which is a position of the identification mark (M) based on the recognition result, and the position information of the identification mark (M) included in the feature information. Based on the above, the positions (Xf, Yf) of the stopped vehicle (2) are estimated.

According to this configuration, when the vehicle is stopped, the recognition result of the identification mark (M) provided corresponding to each of the plurality of parking spaces (P3) installed in the parking lot (1) Appropriately estimate the position of the vehicle (2) based on the recognition position (X', Y') of the identification mark (M) based on the position information of the identification mark (M) included in the feature information. Can be done. Therefore, according to this configuration, the position before the start of the vehicle (2) in the case of automatic driving in the parking lot (1), in other words, the initial position of the vehicle (2) which is the starting point of the automatic driving (2). Xf, Yf) can be estimated appropriately.

The various technical features of the vehicle state estimation system described above can also be applied to a vehicle state estimation method and a vehicle state estimation program. For example, the vehicle state estimation method can be a method having the features of the vehicle state estimation system described above. Further, the vehicle state estimation program can realize a function corresponding to the features of the vehicle state estimation system described above in the computer. As a matter of course, these vehicle state estimation methods and vehicle state estimation programs can also exert the effects of the vehicle state estimation system described above. Further, various additional features exemplified as a preferred embodiment of the vehicle state estimation system can be incorporated into these vehicle state estimation methods and vehicle state estimation programs, and the method and the program are included in the respective additional features. Corresponding action and effect can also be achieved.

Such a vehicle condition estimation method is
It is a vehicle state estimation method that estimates the direction of the vehicle (2) that automatically travels toward the set destination (P) in the parking lot (1) before starting.
The operation state determination step for determining the operation state of the vehicle (2) and
A feature information acquisition step for acquiring feature information including the position information of the feature (O) provided in the parking lot (1), and
A peripheral information acquisition step for acquiring peripheral information of the vehicle (2) detected by the peripheral detection sensor (C) mounted on the vehicle (2), and
A recognition step for performing recognition processing of the feature (O) included in the surrounding information, and
A step of estimating the orientation of the vehicle (2) in the parking lot (1) is provided.
The feature information includes individual orientation information indicating the extending orientation of each of the strip-shaped features (L) provided in the parking lot (1).
In the operation state determination step, it is determined whether or not the vehicle (2) is stopped in the parking lot (1).
In the recognition step, among the features (O) included in the peripheral information detected while the vehicle is stopped when the vehicle (2) is determined to be stopped by the operation state determination step. Recognizing the strip-shaped feature (L),
In the estimation step, the recognition orientation (D'), which is the extending orientation of the strip-shaped feature (L) based on the recognition result, and the individual orientation information of the strip-shaped feature (L) included in the feature information. Based on the above, the direction (Df) of the stopped vehicle (2) is estimated.

In addition, as a vehicle state estimation method having other characteristic configurations,
It is a vehicle state estimation method that estimates the position of a vehicle (2) that automatically travels toward a set destination (P) in the parking lot (1) before starting.
The operation state determination step for determining the operation state of the vehicle (2) and
A feature information acquisition step for acquiring feature information including the position information of the feature (O) provided in the parking lot (1), and a feature information acquisition step.
A peripheral information acquisition step for acquiring peripheral information of the vehicle (2) detected by the peripheral detection sensor (C) mounted on the vehicle (2), and
A recognition step for performing recognition processing of the feature (O) included in the surrounding information, and
An estimation step for estimating the position of the vehicle (2) in the parking lot (1) is provided.
In the operation state determination step, it is determined whether or not the vehicle (2) is stopped in the parking lot (1).
In the recognition step, among the features (O) included in the peripheral information detected while the vehicle is stopped when the vehicle (2) is determined to be stopped in the operation state determination step. Recognize the identification mark (M) provided corresponding to each of the plurality of parking spaces (P3) installed in the parking lot (1).
In the estimation step, the recognition position (X', Y'), which is the position of the identification mark (M) based on the recognition result, and the position information of the identification mark (M) included in the feature information are used. The position (Xf, Yf) of the stopped vehicle (2) is estimated.

In addition, the vehicle condition estimation program
It is a vehicle state estimation program that estimates the direction of the vehicle (2) that automatically travels toward the set destination (P) in the parking lot (1) before starting.
An operation state determination function for determining the operation state of the vehicle (2) and
A feature information acquisition function for acquiring feature information including the position information of the feature (O) provided in the parking lot (1), and
Peripheral information acquisition function that acquires peripheral information of the vehicle (2) detected by the peripheral detection sensor (C) mounted on the vehicle (2), and
A recognition function that performs recognition processing for the feature (O) included in the surrounding information, and
A computer is realized with an estimation function for estimating the orientation of the vehicle (2) in the parking lot (1).
The feature information includes individual orientation information indicating the extending orientation of each of the strip-shaped features (L) provided in the parking lot (1).
In the operating state determination function, it is determined whether or not the vehicle (2) is stopped in the parking lot (1).
In the recognition function, when the vehicle (2) is determined to be stopped by the operation state determination function, among the features (O) included in the peripheral information detected while the vehicle is stopped, Recognizing the strip-shaped feature (L),
In the estimation function, the recognition orientation (D'), which is the extending orientation of the strip-shaped feature (L) based on the recognition result, and the individual orientation information of the strip-shaped feature (L) included in the feature information. Based on the above, the direction (Df) of the stopped vehicle (2) is estimated.

In addition, as a vehicle state estimation program having other characteristic configurations,
It is a vehicle state estimation program that estimates the position of the vehicle (2) that automatically travels toward the set destination (P) in the parking lot (1) before starting.
An operating state function for determining the operating state of the vehicle (2) and
A feature information acquisition function for acquiring feature information including the position information of the feature (O) provided in the parking lot (1), and
Peripheral information acquisition function that acquires peripheral information of the vehicle (2) detected by the peripheral detection sensor (C) mounted on the vehicle (2), and
A recognition function that performs recognition processing for the feature (O) included in the surrounding information, and
A computer is realized with an estimation function for estimating the position of the vehicle (2) in the parking lot (1).
In the operation state determination function, it is determined whether or not the vehicle (2) is stopped in the parking lot (1).
In the recognition function, among the features (O) included in the peripheral information detected while the vehicle is stopped when the vehicle (2) is determined to be stopped by the operation state determination function. Recognize the identification mark (M) provided corresponding to each of the plurality of parking spaces (P3) installed in the parking lot (1).
The estimation function is based on the recognition position (X', Y'), which is the position of the identification mark (M) based on the recognition result, and the position information of the identification mark (M) included in the feature information. The position (Xf, Yf) of the stopped vehicle (2) is estimated.

The technology according to the present disclosure can be used in a vehicle state estimation system that estimates the direction of a vehicle that automatically travels toward a set destination before starting, a vehicle state estimation method, and a vehicle state estimation program.

100: Vehicle condition estimation system 1: Parking lot 2: Vehicle 23: Estimate unit 24: Feature information acquisition unit 25: Image recognition unit (recognition unit)
26: Peripheral image acquisition unit (peripheral information acquisition unit)
27: Operating state determination unit C: Camera (peripheral detection sensor)
CA: Shooting area (detection area)
O: Feature L: Parking frame (belt-shaped feature)
M: Marker (feature to be identified)
P: Destination P3: Parking space PT: Target space Xf, Yf: Vehicle position X, Y: Absolute position X', Y': Recognition position Df: Vehicle direction D: Absolute direction D': Recognition direction D' : Recognized feature-to-feature orientation

Claims (11)

It is a vehicle condition estimation system that estimates the direction of a vehicle that automatically travels toward a set destination before starting in a parking lot.
An operation state determination unit that determines the operation state of the vehicle,
A feature information acquisition unit that acquires feature information including the location information of features provided in the parking lot,
Peripheral information acquisition unit that acquires peripheral information of the vehicle detected by the peripheral detection sensor mounted on the vehicle, and
A recognition unit that performs recognition processing of the feature included in the surrounding information,
An estimation unit for estimating the orientation of the vehicle in the parking lot is provided.
The feature information includes individual orientation information indicating the extending orientation of each of the strip-shaped features provided in the parking lot.
The operating state determination unit determines whether or not the vehicle is stopped in the parking lot, and determines whether or not the vehicle is stopped.
When the operating state determination unit determines that the vehicle is stopped, the recognition unit recognizes the strip-shaped feature among the features included in the peripheral information detected while the vehicle is stopped. And
The estimation unit is the vehicle that is stopped based on the recognition orientation that is the extension orientation of the strip-shaped feature based on the recognition result and the individual orientation information of the strip-shaped feature included in the feature information. Vehicle condition estimation system that estimates the direction of the vehicle. The recognition unit is installed in the parking lot among the features included in the peripheral information detected while the vehicle is stopped when the operation state determination unit determines that the vehicle is stopped. Recognize the identification target features provided for each of the multiple parking spaces
The estimation unit determines the position of the stopped vehicle based on the recognition position which is the position of the identification target feature based on the recognition result and the position information of the identification target feature included in the feature information. The vehicle condition estimation system according to claim 1. It is a vehicle condition estimation system that estimates the position of a vehicle that automatically travels toward a set destination before starting in a parking lot.
An operation state determination unit that determines the operation state of the vehicle,
A feature information acquisition unit that acquires feature information including the location information of features provided in the parking lot,
Peripheral information acquisition unit that acquires peripheral information of the vehicle detected by the peripheral detection sensor mounted on the vehicle, and
A recognition unit that performs recognition processing of the feature included in the surrounding information,
An estimation unit for estimating the position of the vehicle in the parking lot is provided.
The operating state determination unit determines whether or not the vehicle is stopped in the parking lot, and determines whether or not the vehicle is stopped.
The recognition unit is installed in the parking lot among the features included in the peripheral information detected while the vehicle is stopped when the operation state determination unit determines that the vehicle is stopped. Recognize the identification signs provided corresponding to each of the multiple parking spaces as the identification target features,
The estimation unit estimates the position of the stopped vehicle based on the recognition position which is the position of the identification mark based on the recognition result and the position information of the identification mark included in the feature information. Vehicle condition estimation system. When the operating state determination unit determines that the vehicle is stopped, the recognition unit recognizes a plurality of the identification target features included in the peripheral information detected while the vehicle is stopped.
The estimation unit includes the recognition inter-feature direction, which is the extending direction of the line connecting the plurality of identification target features based on the recognition result, and the position information of the plurality of identification target features included in the feature information. The vehicle state estimation system according to claim 2 or 3, wherein the direction of the stopped vehicle is estimated based on the above. When the recognition unit recognizes a plurality of the identification target features included in the peripheral information, and the identification target features are a plurality of identification target features based on the recognition positions of the plurality of identification target features. When there is a difference between the relative positional relationship and the relative positional relationship of the plurality of identification target features based on the position information of the identification target feature included in the feature information,
The estimation unit performs a plurality of the identifications so that the deviation between the recognition position and the position indicated in the position information for each of the plurality of identification target features is the same in the plurality of identification target features. The recognition position of each of the target features is adapted to the position indicated in the position information, and the position of the vehicle based on the recognition position of the plurality of identification target features after the adaptation is set to the position of the vehicle. The vehicle condition estimation system according to any one of claims 2 to 4. Multiple parking spaces are arranged side by side,
While the vehicle is parked in the target space, which is one of the parking spaces, the detection area by the peripheral detection sensor is the identification target feature corresponding to the target space and a pair of pairs adjacent to the target space. It is set in a range that can include the identification target feature corresponding to each of the parking spaces.
The vehicle state estimation system according to any one of claims 2 to 5, wherein the recognition unit performs the recognition process on a plurality of the identification target features that can be included in the detection area. The feature includes a pair of strip-shaped features provided in the parking lot and partitioning both sides of the parking space.
While the vehicle is parked in the parking space, the detection area by the peripheral detection sensor is set to a range that can include the pair of strip-shaped features that partition both sides of the parking space.
The vehicle state estimation system according to any one of claims 2 to 6, wherein the recognition unit performs the recognition process on a pair of the strip-shaped features that can be included in the detection area. It is a vehicle condition estimation method that estimates the direction of a vehicle that automatically travels toward a set destination before starting in a parking lot.
The operation state determination step for determining the operation state of the vehicle and
A feature information acquisition step for acquiring feature information including the position information of the feature provided in the parking lot, and
A peripheral information acquisition step for acquiring peripheral information of the vehicle detected by a peripheral detection sensor mounted on the vehicle, and
A recognition step for performing recognition processing of the feature included in the surrounding information, and
It comprises an estimation step of estimating the orientation of the vehicle in the parking lot.
The feature information includes individual orientation information indicating the extending orientation of each of the strip-shaped features provided in the parking lot.
In the operation state determination step, it is determined whether or not the vehicle is stopped in the parking lot.
In the recognition step, when it is determined by the operation state determination step that the vehicle is stopped, the strip-shaped feature is recognized among the features included in the peripheral information detected while the vehicle is stopped. And
In the estimation step, the stopped vehicle is based on the recognition direction which is the extension direction of the strip-shaped feature based on the recognition result and the individual direction information of the strip-shaped feature included in the feature information. Vehicle condition estimation method that estimates the direction of the vehicle. It is a vehicle condition estimation method that estimates the position of a vehicle that automatically travels toward a set destination before starting in a parking lot.
The operation state determination step for determining the operation state of the vehicle and
A feature information acquisition step for acquiring feature information including the position information of the feature provided in the parking lot, and
A peripheral information acquisition step for acquiring peripheral information of the vehicle detected by a peripheral detection sensor mounted on the vehicle, and
A recognition step for performing recognition processing of the feature included in the surrounding information, and
It comprises an estimation step of estimating the position of the vehicle in the parking lot.
In the operation state determination step, it is determined whether or not the vehicle is stopped in the parking lot.
In the recognition step, when it is determined that the vehicle is stopped in the operation state determination step, it is installed in the parking lot among the features included in the peripheral information detected while the vehicle is stopped. Recognize the identification signs provided for each of the multiple parking spaces
In the estimation step, the position of the stopped vehicle is estimated based on the recognition position which is the position of the identification mark based on the recognition result and the position information of the identification mark included in the feature information. Vehicle condition estimation method. It is a vehicle condition estimation program that estimates the direction of a vehicle that automatically travels toward a set destination before starting in the parking lot.
The operation state determination function for determining the operation state of the vehicle and
A feature information acquisition function that acquires feature information including the location information of features provided in the parking lot, and
Peripheral information acquisition function that acquires peripheral information of the vehicle detected by the peripheral detection sensor mounted on the vehicle, and
A recognition function that performs recognition processing of the feature included in the surrounding information,
A computer is realized with an estimation function for estimating the orientation of the vehicle in the parking lot.
The feature information includes individual orientation information indicating the extending orientation of each of the strip-shaped features provided in the parking lot.
The operating state determination function determines whether or not the vehicle is stopped in the parking lot, and determines whether or not the vehicle is stopped.
In the recognition function, when the vehicle is determined to be stopped by the operation state determination function, the strip-shaped feature is recognized among the features included in the peripheral information detected while the vehicle is stopped. And
In the estimation function, the stopped vehicle is based on the recognition direction which is the extension direction of the strip-shaped feature based on the recognition result and the individual direction information of the strip-shaped feature included in the feature information. Vehicle condition estimation program that estimates the direction of the vehicle. It is a vehicle condition estimation program that estimates the position of a vehicle that automatically travels toward a set destination before starting in a parking lot.
The operation state function for determining the operation state of the vehicle and
A feature information acquisition function that acquires feature information including the location information of features provided in the parking lot, and
Peripheral information acquisition function that acquires peripheral information of the vehicle detected by the peripheral detection sensor mounted on the vehicle, and
A recognition function that performs recognition processing of the feature included in the surrounding information,
A computer is realized with an estimation function for estimating the position of the vehicle in the parking lot.
The operating state determination function determines whether or not the vehicle is stopped in the parking lot, and determines whether or not the vehicle is stopped.
In the recognition function, when the vehicle is determined to be stopped by the operation state determination function, it is installed in the parking lot among the features included in the peripheral information detected while the vehicle is stopped. Recognize the identification signs provided for each of the multiple parking spaces
In the estimation function, the position of the stopped vehicle is estimated based on the recognition position which is the position of the identification mark based on the recognition result and the position information of the identification mark included in the feature information. Vehicle condition estimation program.

Images