DE102019106520B4 - Parking assistance device, parking assistance method and parking assistance program - Google Patents

Abstract

Parking assistance device (20), comprising:a scan result extraction interface (27) that obtains a scan result from a sensor (28) mounted on a vehicle (4);a top view image generator (26) that is generated from the scan result extraction interface (27 ) the scanning result obtained generates a top view image which shows the vehicle (4) seen from above; anda top view image setting interface (23) which, based on the relative positions of an operator (3) remotely operating the vehicle (4) and a parking target (5), sets an orientation of the parking target (5) in the top view image relative to a display screen ( 34) determines when the vehicle (4) is parked at the parking target (5), wherein the top view image setting interface (23) sets the orientation of the top view image so that the orientation of the longer side of the parking target (5) in the top view image corresponds to the orientation the longer side of the display screen (34) is aligned when the vehicle (4) moves from a position spaced from the parking target (5) in the direction of the longer side of the parking target (5).

Description

background

1. Area

The disclosure relates to a parking assistance technology and, more particularly, to a parking assistance device, a parking assistance method, and a parking assistance program for assisting a user attempting to park a vehicle.

2. Description of the related field

DE 10 2016 118 967 A1 relates to a method for remote-controlled maneuvering of a motor vehicle, in which a radio connection is provided between a communication device of the motor vehicle and a mobile terminal external to the motor vehicle, a control signal is transmitted from the mobile terminal to the communication device via the radio connection and the motor vehicle is controlled by means of a control device depending on the control signal is maneuvered autonomously, wherein during maneuvering a position and/or an orientation of the motor vehicle relative to a position and/or an orientation of the mobile terminal is determined based on the radio connection.

DE 10 2015 002 438 A1 relates to a motor vehicle and method for operating a motor vehicle for carrying out an automatic parking process, wherein environmental data of an environment of the motor vehicle are continuously recorded while the motor vehicle is moving in a direction of travel at a speed only below a predetermined limit value. Furthermore, under the condition that a predetermined first operating action carried out by the user is detected, results of the detection carried out are displayed as a schematic representation on a display device of the motor vehicle. This makes it possible to search for a parking space that is tailored to the situation.

• - Bereitstellen von Umfelddaten, die wenigstens einen potentiell durch das Kraftfahrzeug befahrbaren Parkbereich im Umfeld des Kraftfahrzeugs betreffen, an das Assistenzsystem,
• - Übertragen der Umfelddaten und/oder von aus den Umfelddaten berechneten Daten an eine von dem Kraftfahrzeug separat ausgebildete Kommunikationseinrichtung,
• - Anzeige einer in Abhängigkeit der Umfelddaten und/oder der berechneten Daten ermittelten, graphischen Darstellung des Parkbereichs durch die Kommunikationseinrichtung,
• - Erfassen wenigstens einer an der Kommunikationseinrichtung vorgenommenen, den Parkvorgang betreffenden Bedieneingabe,
• - Bestimmen einer eine Zielposition und/oder eine Zielausrichtung des Kraftfahrzeugs in dem Parkbereich oder in einem der Parkbereiche beschreibenden Zielpose, in die das Kraftfahrzeug durch den Parkvorgang geführt werden soll, in Abhängigkeit der erfassten Bedieneingabe, und
• - Führen des Kraftfahrzeugs in die Zielpose durch zumindest teilautomatisierte Längs- und/oder Querführung des Kraftfahrzeugs und/oder Gabe von Fahrhinweisen an einen Fahrer des Kraftfahrzeugs zur Führung des Kraftfahrzeugs in die Zielpose.

DE 10 2014 014 242 A1 discloses a method for operating an assistance system to support a parking process of a motor vehicle, comprising the steps:

• - Providing the assistance system with environmental data relating to at least one parking area in the vicinity of the motor vehicle that can potentially be accessed by the motor vehicle,
• - transmitting the environment data and/or data calculated from the environment data to a communication device designed separately from the motor vehicle,
• - Display of a graphical representation of the parking area determined depending on the surrounding data and/or the calculated data by the communication device,
• - detecting at least one operating input made on the communication device and relating to the parking process,
• - Determining a target pose describing a target position and/or a target orientation of the motor vehicle in the parking area or in one of the parking areas, into which the motor vehicle is to be guided through the parking process, depending on the detected operating input, and
• - Guiding the motor vehicle into the target pose by at least partially automated longitudinal and/or transverse guidance of the motor vehicle and/or giving driving instructions to a driver of the motor vehicle to guide the motor vehicle into the target pose.

DE 10 2013 201 624 A1 discloses a method for transmitting data on a mobile terminal to a function of a vehicle, comprising: determining the data to be transmitted to a function of a vehicle; determining the position of the mobile terminal in relation to a location associated with the function of the vehicle; determining the direction in which the associated location is located based on the position of the mobile terminal; Displaying a control on a touch-sensitive display of the mobile terminal, the control comprising an indication pointing in the direction in which the associated location is located, based on the position of the mobile terminal; receiving an activation input of the control from a user on the touch-sensitive display of the mobile terminal; In response to receiving the activation input: transmitting the data from the mobile terminal to the function of the vehicle.

In order to provide a user with a top view video of a parking lot displayed on a display screen in real time, even when their own vehicle is moving, in JP 2013-006548 A proposed that a parking assistance device include a parking target position detection means that uses a first plan view video to detect a partial target position for parking a vehicle, a viewpoint position determination means that determines a viewpoint position, a rotation amount determination means that determines a rotation amount, a viewpoint transformation table generation means that a viewpoint transformation table for rotating the first Top view videos generated based on viewpoint position and rotation amount, and a second top view video generating means that generates a second top view video based on the viewpoint transformation table from the first top view video.

Recently, various functions have been built into a vehicle to improve the comfort of a vehicle for the user. For example, Patent Document 1 discloses a technology aimed at helping the driver of a vehicle stop the vehicle to adjust his position to a target on a road surface by capturing moving images of a scene therearound containing the target. Generate positioning moving images used for positioning the vehicle and display the thus obtained positioning moving images on a display monitor.

• [Patentdokument 1] JP 2014 - 100 924 A
• [Patentdokument 2] JP 2013 - 6548 A

Further, Patent Document 2 discloses a technology aimed at assisting in parking a vehicle by generating for the user a real-time bird's-eye view in which a parking space is shown on a display screen even when the driver's vehicle is in motion emotional.

• [Patent Document 1] JP 2014 - 100 924 A
• [Patent Document 2] JP 2013 - 6548 A

The inventors have recognized a disadvantage in the related field that it cannot produce an image that is easily understood by the operator when a vehicle equipped with an automatic parking function is remotely controlled from outside the vehicle.

The present invention addresses the problem discussed above, one object of which is to improve the comfort of the user of a vehicle.

Summary

A parking assistance device according to an embodiment of the present disclosure is defined in claim 1.

Another embodiment of the disclosure also relates to a parking assistance device. The device is defined in claim 2.

Another embodiment of the present disclosure also relates to a parking assistance method. The method is defined in claim 5.

Yet another embodiment of the present disclosure also relates to a parking assistance method. The method is defined in claim 6

Optional combinations of the above-mentioned components and implementations of the invention in the form of methods, devices, systems, recording media and computer programs may also be embodied as additional embodiments of the present invention.

According to the present disclosure, comfort for the user of a vehicle is improved.

Brief description of the drawings

• 1 eine Konfiguration eines Parkunterstützungssystems gemäß der ersten Ausführungsform zeigt;
• 2 beispielhaft relative Positionen des Parkziels, des Fahrzeugs und des Betreibers zeigt;
• 3 ein Verfahren einer Koordinatentransformation zeigt;
• 4 einen Bereich, der die Position des Betreibers in dem Zielursprungkoordinatensystem angibt, zeigt;
• 5 ein Beispiel der Bezugnahmetabelle A zeigt, auf die durch den Parametergenerator in der normalen Betriebsart Bezug genommen wird;
• 6 ein Beispiel der Bezugnahmetabelle B zeigt, auf die durch den Parametergenerator 23 in der angepassten Betriebsart Bezug genommen wird;
• 7 ein Beispiel eines auf dem Berührungsbildschirm des Smartphones angezeigten Anzeigebilds zeigt;
• 8 ein Beispiel eines auf dem Berührungsbildschirm des Smartphones angezeigten Anzeigebilds zeigt;
• 9 ein Beispiel eines auf dem Berührungsbildschirm des Smartphones angezeigten Anzeigebilds zeigt;
• 10 ein Beispiel eines auf dem Berührungsbildschirm des Smartphones angezeigten Anzeigebilds zeigt;
• 11 ein Ablaufplan ist, der die Schritte in dem Parkunterstützungsverfahren gemäß der Ausführungsform zeigt;
• 12 ein Ablaufplan ist, der die in 11 gezeigte Einzelheit S16 zeigt;
• 13 ein Ablaufplan ist, der die Schritte des Parkunterstützungsverfahrens gemäß der Ausführungsform zeigt;
• 14 eine Konfiguration eines Parkunterstützungssystems gemäß der zweiten Ausführungsform zeigt; und
• 15 ein Beispiel zeigt, in dem Parkplätze unter einem Winkel zu einer Durchfahrt vorgesehen sind.

Embodiments will now be described, by way of example only, with reference to the accompanying drawings, which are intended to be exemplary and not restrictive, and in which like elements in several figures are provided with like reference numerals, in which:

• 1 shows a configuration of a parking assistance system according to the first embodiment;
• 2 shows exemplary relative positions of the parking target, the vehicle and the operator;
• 3 shows a method of coordinate transformation;
• 4 shows an area indicating the operator's position in the target origin coordinate system;
• 5 shows an example of the reference table A referenced by the parameter generator in the normal mode;
• 6 shows an example of the reference table B referenced by the parameter generator 23 in the customized mode;
• 7 shows an example of a display image displayed on the touch screen of the smartphone;
• 8th shows an example of a display image displayed on the touch screen of the smartphone;
• 9 shows an example of a display image displayed on the touch screen of the smartphone;
• 10 shows an example of a display image displayed on the touch screen of the smartphone;
• 11 is a flowchart showing the steps in the parking assistance method according to the embodiment;
• 12 is a schedule that shows the in 11 shown detail S16 shows;
• 13 is a flowchart showing the steps of the parking assistance method according to the embodiment;
• 14 shows a configuration of a parking assistance system according to the second embodiment; and
• 15 shows an example in which parking spaces are provided at an angle to a driveway.

Detailed description

A description will now be given of a technology embodying the present disclosure, wherein when a vehicle provided with an automatic parking function is remotely controlled by the user of the vehicle from outside the vehicle and automatically arrives at a parking destination, e.g. B. in a parking space in a parking area, is automatically parked, a top view image is displayed on a terminal used by the user, which shows the vehicle seen from above, to help the operator check the scene of parking outside the vehicle .

In vehicles of the related field, a top view image is displayed on a display device mounted in the vehicle to let the driver or a passenger in the vehicle see the image, and thus becomes with the position of the vehicle at the center and in the same orientation as the vehicle displayed. However, the operator may find it difficult to understand the position relative to the parking destination if the same top view is displayed on the operator's terminal outside the vehicle.

To address the problem, the embodiment adjusts the orientation of a parking target in the top view image based on the relative positions of the operator and the parking target relative to the display screen. This helps the operator to correctly understand the relative positions of the operator and the parking target and check the movement of the automatically parked vehicle. Accordingly, the comfort for the operator is improved. Further, according to the embodiment, the operator can correctly and efficiently visually inspect the situation around the vehicle and the parking destination, so that the safety of automatic parking is improved.

[First Embodiment]

1 shows the configuration of a parking assistance system according to the first embodiment. A parking assistance system 1 includes a vehicle-mounted ECU 20 mounted in a vehicle, a smartphone 30 exemplifying a user terminal used by the user of the vehicle, a server 10 for exchanging data between the vehicle-mounted ECU 20 and the Smartphone 30 and a communication line 2 connecting these devices.

The server 10 includes a communication interface 11 and a data exchange server 12. The communication interface 11 controls communication with the vehicle-mounted ECU 20 and with the smartphone 20 via the communication line 2. The data exchange server 12 registers in advance the correspondence between the vehicle-mounted ECU 20 , which is mounted in the vehicle, and the smartphone 30 used by the user of the vehicle. The data exchange server 12 mediates the data exchange between the vehicle-mounted ECU 20 and the registered smartphone 30. In another example, the vehicle-mounted ECU 20 and the smartphone 30 exchange data directly in wireless communication. In this case, the server 10 cannot be provided.

The vehicle-mounted ECU 20 includes a communication interface 21, a display image generator 22, a parameter generator 23, a parking target position detector 24, an operator position detector 25, a top view image generator 26, and a camera interface 27. The features are in hardware such as a CPU, memory, or other LSIs of any computer and implemented in software such as a program loaded into memory. The figure shows functional blocks that are implemented through the interaction of these elements. Thus, those skilled in the art will understand that the functional blocks can be implemented in a variety of ways by hardware alone or by a combination of hardware and software.

The communication interface 21 controls communication with the server 10 and with the smartphone 30 via the communication line 2. The camera interface 27 functions as a scanning result acquisition interface that acquires an image from the vehicle-mounted camera 28 that captures a scene around the vehicle . In order to produce a top view image showing the vehicle viewed from above, the camera 28 is provided at four positions, front, rear, left and right of the vehicle to capture images relative to the vehicle in the four directions including forward, backward, left and right. The top view image generator 26 generates a top view image that shows the vehicle seen from above from the images around the vehicle, which are obtained from the cameras 28 through the camera interface 27. If desired, the top view image generator 26 can generate a top view image using known technology. Alternatively, the top view image generator 26 may generate a top view image from a scan result captured by a sensor other than the camera 28.

The parking target position detector 24 detects a parking target position from the images around the vehicles. The parking target position detector 24 can detect a parking space to be the parking target by referring to the images around the vehicle captured by the cameras 28 and detecting white lines drawn on the ground of the parking area. Alternatively, the parking target position detector 24 may detect a parking target position using information other than captured images, or may detect a parking target position using other known technologies, if desired.

The operator position detector 25 detects the position of the operator. The operator position detector 25 may detect the operator's position by referring to the images around the vehicle captured by the cameras 28, or may obtain the position of the smartphone 30 from the smartphone 30 and use the acquired position as the operator's position. In the former case, the current position of the operator can be detected by starting to track the movement of the operator since a time when the operator left the driver's seat of the vehicle 4. In the latter case, the smartphone 30 can detect the position from a GPS. Alternatively, the relative positions of the smartphone 30 and the vehicle or parking lot may be obtained by analyzing the image captured by the camera provided on the rear surface of the smartphone 30. These methods deal with a case where a plurality of passengers have exited the vehicle 4 and are shown in the image captured by the camera 28 by identifying who the operator of the smartphone 30 is.

The parameter generator 23 functions as a top view image setting interface that determines the orientation of the parking destination in the top view image relative to the display screen based on the parking target position detected by the parking target position detector 24 and the operator's position detected by the operator position detector 25, and a parameter for setting of the top view image to the specific orientation. More specifically, the parameter generator 23 generates as a parameter an angle through which the top view image is rotated.

The parameter generator 23 maintains a reference table that defines the correspondence between i) the relative positions of the operator and the parking target and ii) the rotation angle of the top view image. The parameter generator 23 is capable of determining the orientation of the parking target in the top view image relative to the display screen based on the relative positions of the operator and the parking target in accordance with a variety of determination methods. The parameter generator 23 maintains various reference tables depending on the determination methods. The parameter generator 23 switches between the plurality of determination methods by switching between the reference tables based on an operator's operation received from the smartphone 30 via the communication interface 21.

In this embodiment, it is possible to generate a parameter for rotating the top view image according to two determination methods including the “normal mode” and the “custom mode”. In the “normal mode,” the top view image is rotated so that the parking target in the top view image is oriented vertically or horizontally relative to the display screen. In the “custom mode,” the top view image is rotated so that the orientation of the longer side of the parking target in the top view image is aligned with the orientation of the longer side of the display screen. Before an operation is acknowledged by the operator, the “normal operating mode” or the “adapted operating mode” can be selected as a standard operating mode. Furthermore, the parameter generator 23 may be able to switch between three or more determination methods. The parameter will be described in detail later.

The display image generator 22 sets the top view image generated by the top view image generator 26 according to the parameter generated by the parameter generator 23, and generates a display image to be displayed on a touch screen 34 of the smartphone 30. The display image generator 22 rotates the top view image by the rotation angle determined by the parameter generator 23, and also sets the display position and scale of the top view image in accordance with the positions of an operator 3, the vehicle 4, a parking destination 5, etc. That through The display image generated by the display image generator 22 is sent to the smartphone 30 via the communication interface 21.

The smartphone 30 includes a communication interface 31, a UI generator 32, a touch screen interface 33, and a touch screen 34. These features can also be implemented in a variety of ways through hardware alone or through a combination of hardware and software.

The communication interface 31 controls communication with the server 10 and with the ECU 20 mounted in the vehicle via the communication line 2. The UI generator 32 generates a user interface for switching between the “normal operating mode” and the “adapted operating mode”. Further, the UI generator 32 may generate a user interface for the remote control for starting automatic parking of the vehicle, changing the parking destination, or causing the vehicle to stop. The touch screen interface 33 functions as a display interface that synthesizes the display image obtained from the vehicle-mounted ECU 20 via the communication interface 31 and the user interface generated by the UI generator 32, and displays a generated display screen image on the touch screen 34.

2 shows exemplary relative positions of the parking target, the vehicle and the operator. The operator 3 leaves the vehicle 4 near an available parking space in a parking area and instructs the vehicle 4 from the smartphone 30 to automatically park itself. The vehicle-mounted ECU 20 of the vehicle 4 detects the space around the vehicle 4 using a vehicle coordinate system around an origin O near the center of the vehicle 4 and automatically parks the vehicle. The parking target position detector 24 detects the parking space around the point P as the parking target 5. The angle formed by the vector OP and the y-axis of the vehicle coordinate system is α with respect to the vector OP. The angle formed by the vector PO and by the direction to the entrance to the parking lot is β with respect to the vector PO. The counterclockwise direction is defined as the positive direction.

3 shows a method of coordinate transformation. The parameter generator 23 generates a parameter for adjusting the top view image by using, instead of the vehicle coordinate system, a coordinate system defined around the parking destination as the origin, that is, a destination origin coordinate system around the point P in which the direction to the entrance of the parking destination 5 is along as the negative direction the y-axis is defined. The parameter generator 23 transforms source coordinates in the vehicle coordinate system into coordinates in the target origin coordinate system by transforming the source coordinates by a motion vector M = -P and a rotation angle θ = α - β.

In 4 shows an area indicating the operator's position in the target origin coordinate system. In this embodiment, the orientation of the top view image is adjusted depending on which of the eight in 4 the operator is positioned in the areas shown. In the in 2 and 3 In the example shown, the operator position detector 25 has detected that the operator is at the point Q, so that the position at which the operator is belongs to the area “6”.

5 shows an example of the reference table A referenced by the parameter generator 23 in the normal mode. The reference table A defines rotation angles to rotate the top view image so that the orientation of the parking target in the top view image is vertical or horizontal relative to the display screen. A determination as to which area the operator's position belongs to is made in accordance with the condition defining the x-coordinate and the y-coordinate of the position at which the operator is located.

For example, the operator is in the area "6" or "7" if the coordinates of the position where the operator is located has a condition "y < x <0" or a condition "y ≤ -x ≤ 0" fulfill. The parameter generator 23 thus sets the rotation angle φ, by which the top view image is rotated, to “0°”. In this case the in 2 and 3 Top view image shown shown as is. Thus, the vector indicating the direction to the entrance of the parking destination is oriented "downward" and the parking destination is displayed so that it is vertically oriented relative to the display screen.

If the coordinates of the position where the operator is located satisfy a condition “-x > y > 0” or a condition “x ≤ y ≤ 0”, then the operator is in the range “4” or “5”. The parameter generator 23 thus sets the rotation angle cp, by which the top view image is rotated, to “90°”. In this case the in 2 and 3 The top view image shown is rotated 90° counterclockwise and displayed accordingly. Thus, the vector indicating the direction to the entrance of the parking destination is oriented "to the right" and the parking destination is displayed so that it is oriented horizontally relative to the display screen.

If the coordinates of the position where the operator is located satisfy a condition “y > x > 0” or a condition “y ≥ -x ≥ 0”, then the operator is in the area “2” or “3”. The parameter generator 23 thus sets the rotation angle cp, by which the top view image is rotated, to “180°”. In this case the in 2 and 3 The top view image shown is rotated by 180° and displayed accordingly. Thus, the vector indicating the direction to the entrance of the parking destination is oriented "upward" and the parking destination is displayed so that it is vertically oriented relative to the display screen.

If the coordinates of the position where the operator is located satisfy a condition "-x < y <0" or a condition "x ≥ -y ≥ 0", the operator is in the range "8" or "1" . The parameter generator 23 thus sets the rotation angle cp, by which the top view image is rotated, to “-90°”. In this case the in 2 and 3 The top view image shown is rotated 90° clockwise and displayed accordingly. Thus, the vector indicating the direction to the entrance of the parking destination is oriented "to the left" and the parking destination is displayed so that it is oriented horizontally relative to the display screen.

In any case, the top view image is rotated so that the operator is displayed toward the bottom of the screen and the parking destination is displayed toward the top of the screen.

6 shows an example of the reference table B referenced by the parameter generator 23 in the adapted mode. The reference table B defines rotation angles to rotate the top view image so that the orientation of the longer side of the parking target in the top view image is aligned with the orientation of the longer side of the display screen.

For example, the operator is in the range "1", "2", "3" or "4" if the coordinates of the position where the operator is located satisfy a condition "y >0". Thus, the parameter generator 23 sets the rotation angle φ by which the top view image is rotated to “180°” if the vertical direction of the screen is longer than the horizontal direction. In this case the in 2 and 3 The top view image shown is rotated by 180° and displayed accordingly. Thus, the vector indicating the direction to the entrance of the parking destination is oriented "upward" and the parking destination is displayed so that it is vertically oriented relative to the display screen.

If the coordinates of the position where the operator is located meet a condition “y < 0”, then the operator is in the range “5”, “6”, “7” or “8”. Thus, the parameter generator 23 sets the rotation angle φ by which the top view image is rotated to “0°” if the vertical direction of the screen is longer than the horizontal direction. In this case the in 2 and 3 Top view image shown shown as is. Thus, the vector indicating the direction to the entrance of the parking destination is oriented "downward" and the parking destination is displayed so that it is vertically oriented relative to the display screen.

If the coordinates of the position where the operator is located satisfy a condition “x ≥ 0”, then the operator is in the range “1”, “2”, “7” or “8”. Thus, the parameter generator 23 sets the rotation angle φ by which the top view image is rotated to “-90°” if the horizontal direction of the screen is longer than the vertical direction. In this case the in 2 and 3 The top view image shown is rotated 90° clockwise and displayed accordingly. Thus, the vector indicating the direction to the entrance of the parking destination is oriented "to the left" and the parking destination is displayed so that it is oriented horizontally relative to the display screen.

If the coordinates of the position where the operator is located satisfy a condition “x < 0”, then the operator is in the range “3”, “4”, “5” or “6”. Thus, the parameter generator 23 sets the rotation angle φ by which the top view image is rotated to “90°” if the horizontal direction of the screen is longer than the vertical direction. In this case the in 2 and 3 The top view image shown is rotated counterclockwise by 90° and displayed accordingly. Thus, the vector indicating the direction to the entrance of the parking destination is oriented "to the right" and the parking destination is displayed so that it is oriented horizontally relative to the display screen.

In the customized mode, the top view image is rotated as in the normal mode so that the operator is displayed to the bottom of the screen and the parking destination is displayed to the top of the screen.

7 shows an example of a display image displayed on the touch screen 34 of the smartphone 30. The operator instructs the vehicle 4 from the smartphone 30 to automatically park itself and checks the automatic parking scene at a position where the parking target 5 and the vehicle are visible. The display image generated from the top view image by the display image generator 22 is displayed on the touch screen 34 of the smartphone 30. In the example from 7 the normal operating mode is selected. So the top view image is rotated so that the orientation of the parking target 5 relative to the touch screen 34 is vertical when the smartphone 30 is held so that the longer side of the touch screen 34, as in 7 shown is horizontal. The top view image is generated from the image captured by the camera mounted on the vehicle 4 such that the vehicle 4 itself is not captured in the image. The top view image generator 26 obtains the position and orientation of the vehicle 4 and synthesizes a replacement image 40 of the vehicle to match the obtained position and orientation.

The 8th and 9 show examples of a display image displayed on the touch screen 34 of the smartphone 30. In the examples 8th and 9 the adapted operating mode is selected. Thus, the top view image is rotated so that the orientation of the parking target 5 relative to the touch screen 34 is vertical when the smartphone 30 is held so that the longer side of the touch screen 34, as in 8th shown is vertical. When the smartphone 30 is held so that the longer side of the touch screen, as in 9 is shown is horizontal, the top view image is rotated so that the orientation of the parking target 5 relative to the touch screen 34 is horizontal. Normally, the vehicle 4 moves from a position spaced from the parking target 5 in the direction of the longer side of the parking target 5. Thus, by rotating the top view so that the direction of the longer side of the parking target 5 is aligned with the direction of the longer side of the display screen, a large display size is ensured when both the parking target 5 and the vehicle 4 are displayed on the display screen are. Accordingly, visibility for the operator is improved.

10 shows an example of a display image displayed on the touch screen 34 of the smartphone 30. In the example from 10 A user interface 41 generated by the UI generator 32 is displayed on the touch screen 34. When the operator determines the normal mode selection using the user interface 41, the parameter generator 23 refers to the reference table A to generate the normal mode parameter. When the operator determines the selection of the customized mode using the user interface 41, the parameter generator 23 refers to the reference table B to generate the customized mode parameter. This allows the operator to change the orientation of the top view image according to the operator's preference, situation, or the like. Accordingly, the comfort for the operator is improved.

In the in 7-10 In the examples shown, the display image generator 22 adjusts the display position and scale of the top view image so that the positions of both the parking target 5 and the vehicle 4 and the operator 3 are displayed on the display screen. When it is difficult to display the positions of both the parking target 5 and the vehicle 4 and the operator 3 on the display screen, the display image generator 22 may adjust the display position, scale or rotation angle of the top view image to display only one or two of them are displayed on the display screen. If the operator 3 remotely controls the vehicle 4 from a position far away from the vehicle 4 and the parking destination 5, the display image generator 22 cannot display the position of the operator 3 on the display screen and only the positions of the vehicle 4 and the parking destination on the display screen Show 5. Further, only the neighborhood of the vehicle 4 may be displayed on the display screen, or only the neighborhood of the parking destination 5 may be displayed on the display screen while the vehicle 4 is far away from the parking destination 5. While the vehicle 4 is far away from the parking target 5, the top view image can be rotated at an angle that makes the orientation of the parking target 5 non-vertical or horizontal relative to the display screen. In this case, the top view image can also be rotated according to the parameter generated by the parameter generator 23 when the vehicle 4 approaches the parking target 5.

While the position of the vehicle 4 is not shown on the display screen, the display image generator 22 may display an icon of the vehicle 4 near the edge of the screen near the position of the vehicle 4. If the substitute image 40 of the vehicle 4 or an icon of the vehicle 4 is displayed on the display screen, the display mode of the substitute image 40 or the icon may be changed based on the condition of the vehicle 4. For example, the substitute image 40 or the icon of various display modes may be displayed in accordance with whether the vehicle 4 is being driven forward, backward, at a stop, accelerated, decelerated, etc. In addition, the direction of movement of the vehicle 4 can be indicated by an arrow or the like.

The same applies to operator 3. Although the position of the operator 3 is shown on the display screen, the position of the operator may have a substitute image of the operator 3 displayed and an icon of the operator 3 displayed near the edge of the screen while the position of the operator 3 is not shown on the display screen is.

If it is possible, a camera 38 mounted by a camera other than the camera 38 mounted on the vehicle 4 (e.g., a camera provided on the rear surface of the smartphone 30, a camera installed in a parking lot, etc .) To obtain the captured image, the display image generator 22 can generate a display image by reusing the acquired image. For example, an image of a position distant from the vehicle 4 or an image in an area that is blocked by another vehicle or the like and that cannot be captured by the camera 28 may be supplemented with an image acquired by another camera. Furthermore, when an image captured by the camera 28 is transformed into a top view image, an image of a portion that is severely distorted may be replaced with an image acquired by another camera.

11 is a flowchart showing the steps in the parking assistance method according to the embodiment. The top view image generator 26 of the vehicle-mounted ECU 20 generates a top view image (S10) from images around the vehicle captured by the cameras 28. The parking target position detector 24 detects a parking target position (S12) from the top view image, and the operator position detector 25 detects the operator position (S14) in the top view image. The parameter generator 23 generates a parameter for generating a display image by referring to the parking target position and the operator positions (S16). The display image generator 22 generates a display image based on the parameter from the top view image (S18). The above steps are repeated (No in S20) until parking is completed or stopped. When parking is completed or stopped (Yes in S20), the parking assistance process is ended.

12 is a flowchart that shows the in 11 shown detail S16 shows. In order to generate a parameter for generating a display image from the parking target position and from the operator position, the parameter generator 23 first calculates the motion vector M = -P and the rotation angle θ = α - β, which in the coordinate transformation from the vehicle coordinates to the target origin coordinates, which are the parking target position is defined as the origin can be used (S20). The parameter generator 23 carries out the coordinate transformation from the vehicle coordinates to the target origin coordinates (S22). Then, the parameter generator 23 determines a range to which the operator position Q belongs using the reference table A, and calculates the rotation angle φ to the orientation in which the top view image is to be displayed (S24). The above description assumes that the normal mode is selected as the default function. If the customized mode is selected, reference table B is referred to in S24.

13 is a flowchart showing the steps of the parking assistance method according to the embodiment. 13 shows the steps of the parking assistance process that are carried out when the user interface is created for switching between the normal operating mode and the customized operating mode. First, the parameter generator 23 sets the reference table A in the reference type variable T to select the normal mode (S30). In S10 - S18, which are the same as those in 11 The steps shown are, the display image is created from the top view image. If the operator does not use the user interface displayed on the touch screen 34 of the smartphone 30 (No in S23), the above steps are repeated until parking is completed or stopped (No in S20). When parking is completed or stopped (Yes in S20), the parking assistance process is ended. If the operator touches the touch screen 34 of the smartphone 30 to use the user interface (Yes in S32), the reference table B is set in T (S40) when the adjusted mode for aligning the longer side of the parking target with the longer side of the display screen is selected (Yes in S36), and the reference table A is set in T (S38) when the normal mode is selected (No in S36). Control is returned to S10 and the display image is generated from the top view image by referring to the selected reference table.

According to the technology of the embodiment, the top view image is adjusted to the correct orientation and displayed accordingly according to the relative positions of the operator remotely operating the vehicle from outside the vehicle and the parking target. Thus, it is possible to generate a top view image in an orientation that is understood by the operator and improve the convenience of the operator.

[Second Embodiment]

14 shows a configuration of a parking assistance system 6 according to the second embodiment. In the parking assistance system 6 according to the second embodiment, some of the parking assistance system 6 according to FIG 1 First embodiment shown in the vehicle-mounted ECU functions provided in the parking assistance system 1 in the smartphone 30. In the first embodiment, the vehicle-mounted ECU 20 functions as a parking assistance device, while in the second embodiment, the smartphone 30 functions as a parking assistance device direction acts. In this embodiment, the features and operations different from those of the parking assistance system 1 according to the first embodiment will be mainly described, and a description of the features and operations similar to those of the first embodiment will be omitted as appropriate. In 14 those features that have similar functions to those of the first embodiment are denoted by the same reference numerals.

The vehicle-mounted ECU 20 includes a communication interface 21, a top view image generator 26, a parking target position detector 24, and a camera interface 27. The top view image generator 26 generates a top view image for display on the vehicle-mounted display device. The parking target position detector 24 detects a parking target position for automatic parking. The top view image generated by the top view image generator 26 and the information indicating the position of the parking target detected by the parking target position detector 24 are sent to the smartphone 30 via the communication interface 21.

The smartphone 30 includes a communication interface 31, a top view image acquisition interface 35, a parking target position acquisition interface 36, an operator position detection detector 25, a parameter generator 23, a display image generator 22, a UI generator 32, a touch screen interface 33 and a touch screen 34.

The top view image obtaining interface 35 acquires the top view image sent from the vehicle-mounted ECU 20. The parking target position acquisition interface 36 acquires information indicating the parking target position sent from the vehicle-mounted ECU 20. The operations of the other features are similar to those of the first embodiment.

The technology of this embodiment also enables the top view image to be adjusted according to the relative positions of the operator remotely operating the vehicle from outside the vehicle and the parking destination. Thus, it is possible to generate a top view image in an orientation that is easily understood by the operator and improve the convenience of the operator.

Whether each of the features of the vehicle-mounted ECU 20 and the smartphone 30 is generated can be selected, if desired, depending on the specification of the vehicle-mounted ECU 20 and the smartphone 30, etc. Thus, various combinations of features other than the parking assistance system 1 according to the first embodiment or the parking assistance system according to the second embodiment are possible. These combinations are also useful as embodiments of the present disclosure. For example, the top view image generator 26 or the parking target position detector 24 may be provided in the smartphone 30, or the UI generator 32 may be provided in the vehicle-mounted ECU 20.

An explanation based on an exemplary embodiment is described above. The embodiment is intended to be illustrative only and those skilled in the art will understand that various changes could be developed in components and processes and that such changes would also be within the scope of the present invention.

The description of the embodiments assumes that a plurality of rectangular parking spaces are arranged towards their shorter sides, but the technology according to this disclosure is also applicable to the case where parking spaces are at an angle to the passage on which the vehicle is traveling, such as she e.g. B. can be found in a parking area in a road maintenance area. 15 shows an example in which parking spaces are provided at an angle to the driveway. In this case, the parameter generator 23 also determines the orientation of the parking target 5 in the top view image relative to the display screen based on the relative positions of the operator 3 who remotely controls the vehicle and the driving destination 5. The parameter generator 23 can thus determine the orientation of the top view image that the orientation of the parking target 5 in the top view image is vertical or horizontal relative to the display screen. Alternatively, the parameter generator 23 may determine the orientation of the top view image such that the orientation of the passage of the vehicle 4 in the top view image is vertical or horizontal relative to the display screen. In the latter case, the orientation of the passage of the vehicle 4 may be defined as the orientation of a straight line connecting ends of white lines indicating parking spaces.

The orientation of the smartphone 30 may be detected using a scanning result from a gyro sensor for detecting the orientation of the smartphone 330 or an image captured by a camera provided on the rear surface of the smartphone 30, and the display position and the orientation of the top view image can be adjusted based on the operator position and the orientation of the smartphone 30. For example, can the display position of the top view image can be adjusted so that the operator position is near the bottom of the display screen, and the orientation of the top view image can be adjusted so that the smartphone 30 is oriented vertically upward relative to the display screen. This allows the top view image to be displayed at the same relative position as it appears in the actual space viewed by the operator. Further, the orientation of the top view image can be adjusted so that the direction from the operator position to the parking destination or to the vehicle is vertically upward relative to the display screen. Such an operator standpoint operating mode may be selectable in addition to the normal operating mode and the adjusted operating mode.

An embodiment of the present disclosure is summarized below. A parking assistance device according to an embodiment of the disclosure includes: a sensing result obtaining interface that acquires a sensing result from a sensor mounted on a vehicle; a top view image generator that generates a top view image showing the vehicle viewed from above from the scanning result obtained from the scan result obtaining interface; and a top view image setting interface that determines an orientation of the parking destination in the top view image relative to a display screen based on the relative positions of an operator remotely operating the vehicle and the parking destination when the vehicle is parked at a parking destination.

According to this embodiment, the top view image is adjusted to a proper orientation according to the relative positions of the operator remotely operating the vehicle from outside the vehicle and the parking target, so that the operator's convenience is improved.

Another embodiment of the disclosure also relates to a parking assistance device. The apparatus includes: a top view image obtaining interface that acquires a top view image showing a vehicle viewed from above generated from a sensing result from a sensor mounted on the vehicle; a top view image setting interface that determines an orientation of the parking destination in the top view image relative to a display screen based on the relative positions of the operator remotely operating the vehicle and the parking destination when the vehicle is parked at a parking destination; and a display interface that displays the top view image set to the orientation determined by the top view image setting interface on the display screen.

The embodiment also enables the top view image to be adjusted to a proper orientation according to the relative positions of the operator remotely operating the vehicle from outside the vehicle and the parking target, so that the convenience of the operator is improved.

The top view image setting interface may determine the orientation of the top view image so that the orientation of the parking target in the top view image is vertical or horizontal relative to the display screen. This allows the top view image to be rotated and displayed in an orientation that is easily understood by the operator, so that convenience for the operator is improved.

The top view image setting interface may adjust the orientation of the top view image so that the orientation of the longer side of the parking target in the top view image is aligned with the orientation of the longer side of the display screen. This allows the top view image to be rotated and displayed in an orientation that is easily understood by the operator and ensures a large display size when both the position of the parking target and that of the vehicle are displayed on the display screen. Accordingly, visibility for the operator is improved.

The top view image setting interface may determine the orientation of the parking target in the top view image relative to the display screen according to a plurality of determination methods based on the relative positions of the operator and the parking target, and may switch between the plurality of determination operation methods based on an operation of the operator. This allows the operator to change the orientation of the top view image according to the operator preference, situation, or the like. Accordingly, the comfort for the operator is improved.

Another embodiment of the present disclosure relates to a parking assistance method. The method is implemented by a computer and includes: obtaining a sensing result from a sensor mounted on a vehicle; generating a top view image, which is an image showing the vehicle viewed from above, from the acquired scanning result; and determining an orientation of the parking destination in the top view image relative to a display screen based on the relative positions of an operator remotely operating the vehicle and the parking destination when the vehicle is parked at a parking destination.

According to this embodiment, the top view image is created according to the relative positions of the Operator who remotely controls the vehicle from outside the vehicle and the parking destination are set to a correct orientation, so that the comfort for the operator is improved.

Yet another embodiment of the present disclosure relates to a parking assistance method. The method is implemented by a computer and includes: obtaining a top view image showing a vehicle viewed from above and generated from a sensing result from a sensor mounted on the vehicle; determining an orientation of the parking destination in the top view image relative to a display screen based on the relative positions of an operator remotely operating the vehicle and the parking destination when the vehicle is parked at a parking destination; and displaying the top view image set to the particular orientation on the display screen.

The embodiment also enables the top view image to be adjusted to a proper orientation according to the relative positions of the operator remotely operating the vehicle from outside the vehicle and the parking target, so that the convenience of the operator is improved.

Description of reference numbers

1 parking support system, 3 operator, 4 vehicle, 5 parking target, 6 parking support system, 20 vehicle-mounted ECU, 22 display image generator, 23 parameter generator, 24 parking target position detector, 25 operator position detector, 26 top view image generator, 27 camera interface, 28 camera, 30 smartphone, 32 UI generator, 33 touch screen interface, 34, touch screen, 35 top view image acquisition interface, 36 parking target position acquisition interface

Claims (8)

Parking assistance device (20), comprising: a sensing result acquisition interface (27) that acquires a sensing result from a sensor (28) mounted on a vehicle (4); a top view image generator (26) that generates a top view image showing the vehicle (4) viewed from above from the scanning result obtained from the scan result obtaining interface (27); and a top view image setting interface (23) which, based on the relative positions of an operator (3) who remotely controls the vehicle (4) and a parking target (5), sets an orientation of the parking target (5) in the top view image relative to a display screen ( 34) determines when the vehicle (4) is parked at the parking destination (5), where the top view image adjustment interface (23) adjusts the orientation of the top view image so that the orientation of the longer side of the parking target (5) in the top view image is aligned with the orientation of the longer side of the display screen (34) when the vehicle (4) is off moves to a position spaced from the parking target (5) in the direction of the longer side of the parking target (5). Parking assistance device (30), comprising: a top view image acquisition interface (35) that acquires a top view image showing a vehicle (4) viewed from above and generated from a scanning result from a sensor (28) mounted on the vehicle (4); a top view image setting interface (23) which, based on the relative positions of an operator (3) who remotely controls the vehicle (4) and a parking target (5), sets an orientation of the parking target (5) in the top view image relative to a display screen ( 34) determines when the vehicle (4) is parked at the parking destination (5); and a display interface (33) that displays the top view image set to the orientation determined by the top view image setting interface (23) on the display screen (34), wherein the top view image adjustment interface (23) adjusts the orientation of the top view image so that the orientation of the longer side of the parking target (5) in the top view image is aligned with the orientation of the longer side of the display screen (34) when the vehicle (4) is off moves to a position spaced from the parking target (5) in the direction of the longer side of the parking target (5). Parking assistance device (20, 30). Claim 1 or 2 , wherein the top view image setting interface (23) determines the orientation of the top view image so that the orientation of the parking target (5) in the top view image is vertical or horizontal relative to the display screen (34). Parking assistance device (20, 30) according to one of the Claims 1 until 3 , wherein the top view image setting interface (23) controls the orientation of the parking target (5) in the top view image relative to the display screen (34) according to a variety of determination methods based on the relative positions of the operator (3) and the parking target (5), can determine and switches between the plurality of determination methods based on an operation by the operator (3). Parking assistance method implemented by a computer and including: obtaining a sensing result from a sensor (28) mounted on a vehicle (4); Generating a top view image, which shows the vehicle (4) seen from above, from the obtained sampling result; and determining an orientation of the parking target (5) in the top view image relative to a display screen (34) based on the relative positions of an operator (3) remotely operating the vehicle (4) and a parking target (5) when the vehicle ( 4) is parked at the parking target (5), the determining adjusting the orientation of the top view image so that the orientation of the longer side of the parking target (5) in the top view image is aligned with the orientation of the longer side of the display screen (34), if the vehicle (4) moves from a position that is spaced from the parking target (5) in the direction of the longer side of the parking target (5). Parking assistance method implemented by a computer and including: Obtaining a top view image showing a vehicle (4) viewed from above and generated from a scanning result from a sensor (28) mounted on the vehicle (4); Determining an orientation of the parking target (5) in the top view image relative to a display screen (34) based on the relative positions of an operator (3) remotely operating the vehicle (4) and a parking target (5) when the vehicle (4 ) is parked at the parking destination (5); and Displaying the top view image set to the particular orientation on the display screen (34), wherein determining adjusts the orientation of the top view image so that the orientation of the longer side of the parking target (5) in the top view image is aligned with the orientation of the longer side of the display screen (34) when the vehicle (4) moves from a position that is spaced from the parking target (5) in the direction of the longer side of the parking target (5). Parking assistance program that causes a computer to act as: a sensing result acquisition interface (27) that acquires a sensing result from a sensor (28) mounted on a vehicle (4); a top view image generator (26) which generates a top view image showing the vehicle (4) viewed from above from the scanning result obtained from the scanning result obtaining interface (27); and a top view image setting interface (23) which, based on the relative positions of an operator (3) who remotely controls the vehicle (4) and a parking target (5), sets an orientation of the parking target (5) in the top view image relative to a display screen ( 34) determines when the vehicle (4) is parked at the parking destination (5), where the top view image adjustment interface (23) adjusts the orientation of the top view image so that the orientation of the longer side of the parking target (5) in the top view image is aligned with the orientation of the longer side of the display screen (34) when the vehicle (4) is off moves to a position spaced from the parking target (5) in the direction of the longer side of the parking target (5). Parking assistance program that causes a computer to act as: a top view image obtaining interface (35) that acquires a top view image showing a vehicle (4) viewed from above and generated from a sensing result from a sensor (28) mounted on the vehicle (4); a top view image setting interface (23) which, based on the relative positions of an operator (3) who remotely controls the vehicle (4) and a parking target (5), sets an orientation of the parking target (5) in the top view image relative to a display screen ( 34) determines when the vehicle (4) is parked at the parking destination (5); and a display interface (33) that displays the top view image set to the orientation determined by the top view image setting interface (23) on the display screen (34), wherein the top view image adjustment interface (23) adjusts the orientation of the top view image so that the orientation of the longer side of the parking target (5) in the top view image is aligned with the orientation of the longer side of the display screen (34) when the vehicle (4) is off moves to a position spaced from the parking target (5) in the direction of the longer side of the parking target (5).

Images