JP2019104311A - Parking assistance device and vehicle - Google Patents

Abstract

To provide a technology for executing parking assistance using markers in various environments.SOLUTION: A parking assistance device 30 assists parking of a vehicle 100 by using a plurality of markers after the plurality of portable markers have been arranged in a parking target position. An acquisition part 46 acquires an image obtained by imaging an outside of the vehicle 100. A detection part 48 detects the plurality of markers from the image acquired by the acquisition part 46. A determination part 50 determines whether or not an arrangement of the plurality of markers is appropriate based on a positional relationship among the plurality of markers detected by the detection part 48. A generation part 56 generates a parking route based on the positions of the plurality of markers when they are determined to be appropriate by the determination part 50.SELECTED DRAWING: Figure 3

Description

  The present disclosure relates to parking assistance technology, and more particularly, to a parking assistance apparatus and a vehicle that assist parking.

  Conventionally, when a vehicle parks, the parking assistance device performs parking assistance based on image information obtained by performing image processing on a camera image captured by an on-vehicle camera installed at the rear of the vehicle or the like. At this time, a marker indicating a parking area is recognized by the image recognition process. In order to prevent false recognition of the markers, it is known that two markers are provided on the ground and the positional relationship between the two markers is used.

WO 14/156166

  Placing the marker at the parking target position facilitates identification of the parking target position by the image recognition process. However, in order to execute this parking assistance, markers need to be installed in advance. Therefore, even if it is a parking target position where parking is difficult, parking assistance is not performed unless a marker is installed in advance.

  The present disclosure has been made in view of these circumstances, and an object thereof is to provide a technology for performing parking assistance using a marker in various environments.

  In order to solve the above problems, a parking assistance device according to an aspect of the present disclosure performs parking that uses a plurality of markers to assist parking of a vehicle after a plurality of portable markers are placed at a parking target position. The support device is an acquisition unit that acquires an image obtained by imaging the outside of the vehicle, a detection unit that detects a plurality of markers from the image acquired by the acquisition unit, and a positional relationship between the plurality of markers detected by the detection unit. And a determination unit that determines whether or not the arrangement of the plurality of markers is appropriate, and a generation unit that generates a parking route based on the positions of the plurality of markers when the determination unit determines that the arrangement is appropriate. And.

  Another aspect of the present disclosure is a vehicle. This vehicle is a vehicle, and after a plurality of portable markers are arranged at a parking target position, a parking assist device that assists parking of the vehicle using the plurality of markers and an image of the outside of the vehicle can be taken And an imaging device. The parking assist apparatus determines whether the arrangement of the plurality of markers is appropriate based on the positional relationship between the detection unit that detects the plurality of markers from the image captured by the imaging device and the plurality of markers detected by the detection unit. And a generation unit that generates a parking path based on the positions of a plurality of markers when it is determined that the determination unit is valid.

  In the present disclosure, any combination of the above components, the expression of the present disclosure converted between an apparatus, a system, a method, a program, a recording medium recording the program, a vehicle equipped with the present apparatus, etc. It is effective as an aspect.

  According to the present disclosure, parking assistance using markers can be performed in various environments.

It is a figure which shows the parking area which the vehicle which concerns on this Embodiment should park. It is a figure which shows the outline | summary of the parking assistance by the vehicle of FIG. It is a figure which shows the structure of the vehicle of FIG. It is a figure which shows the screen displayed on the display apparatus of FIG. 5a-5d are perspective views showing the structure of the marker of FIG. 6a-6b are top views showing another structure of the marker of FIG. It is a top view which shows arrangement | positioning of the imaging device of FIG. 8a to 8c are diagrams showing an outline of processing of the determination unit of FIG. 9a to 9c are diagrams showing an outline of another process of the determination unit of FIG. It is a figure which shows the data structure of the table memorize | stored in the notification control part of FIG. It is a figure which shows the other screen displayed on the display apparatus of FIG. It is a figure which shows the further another screen displayed on the display apparatus of FIG. It is a figure which shows the outline | summary of another process of the determination part of FIG. It is a flowchart which shows the parking assistance procedure by the parking assistance apparatus of FIG. It is a flowchart which shows the warning procedure by the parking assistance apparatus of FIG.

  Before specifically describing the present embodiment, an outline will be described. The present embodiment relates to a parking assistance device that assists parking of a vehicle. When performing automatic parking of a vehicle, the parking assistance device detects a parking target position, but when detecting a parking target position by sonar, the vehicle had to pass in front of the parking target position. However, when the parking target position faces the dead end, the vehicle can not pass the parking target position, and the parking assistance device can not detect the parking target position. On the other hand, the parking assistance apparatus may detect a parking target position using the image imaged with the imaging device. In this case, two markers are installed at the parking target position, and the parking assistance device specifies the parking target position based on the two markers detected from the image captured by the imaging device. The use of two markers facilitates identification of the parking target position. However, as described above, such a target parking position can not be specified in various environments where two markers need to be installed in advance, and two markers are not installed in advance.

  The two markers in the present embodiment have portability, and when parking assistance is started, the occupant gets off the vehicle and places the two markers at the parking target position. The occupant may be a driver or a passenger. This enables parking assistance using two markers even if two markers are not installed in advance. However, since two markers are arranged by the occupant, if the user selects an area as narrow as the vehicle can not park as the parking target position, the parking target position can be specified using the two markers. A situation occurs in which the vehicle can not be parked at the target position. In order to suppress the occurrence of such a situation, the parking assistance device according to the present embodiment determines whether the arrangement of the two markers is appropriate based on the distance between the two markers detected from the image. Determine The parking assistance apparatus identifies a parking target position from two markers, as appropriate, and generates a parking path. On the other hand, the parking assist device informs the occupant that the arrangement of the two markers is not appropriate when the parking assistance device is not appropriate, thereby causing the occupant to rearrange the two markers or select another parking target position. can do.

  Hereinafter, the present embodiment will be described in detail with reference to the drawings. In addition, each embodiment described below is an example, and this indication is not limited by these embodiments.

  FIG. 1 shows a parking area 200 where the vehicle 100 should park. A first parking area 200a to a ninth parking area 200i collectively referred to as a parking area 200 are arranged side by side facing the road 210 extending in the lateral direction. Each parking area 200 is an area in which the vehicle 100 can be parked in parallel, and has a vertically long rectangular shape formed by a short side extending in the lateral direction and a long side extending in the vertical direction. It is placed in contact with 200. The number of parking areas 200 is not limited to "9". Here, as an example, the first parked vehicle 110a to the eighth parked vehicle 110h exist in the first parking area 200a to the eighth parking area 200h. The first parked vehicle 110a to the eighth parked vehicle 110h are collectively referred to as a parked vehicle 110. The parking vehicle 110 directs the front side to the road 210, that is, the lower side. On the other hand, the ninth parking area 200i is empty.

  The vehicle 100 enters the road 210 from the lower side and tries to park. The vehicle 100 should park in the vacant ninth parking area 200i. This corresponds to setting the ninth parking area 200i at the parking target position. When the vehicle 100 detects a parking target position by sonar, the vehicle 100 has to travel on the road 210 in the right direction and pass the ninth parking area 200i. However, the road 210 is dead end and can not pass the ninth parking area 200i.

  FIG. 2 shows an outline of parking assistance by the vehicle 100. As shown in FIG. Vehicle 100, parked vehicle 110, parking area 200, and road 210 are shown as in FIG. The occupant of the vehicle 100 finds that the ninth parking area 200i is vacant, and selects the ninth parking area 200i as the parking target position 120. The vehicle 100 travels along A to the front of the ninth parking area 200i and stops there. Therefore, the occupant causes the parking assistance device to start the process, and gets off the vehicle 100 and arranges the first marker 80a and the second marker 80b in the ninth parking area 200i as shown by B. The first marker 80a and the second marker 80b are collectively referred to as the marker 80 and have portability.

  The parking assistance apparatus extracts two markers 80 from the image captured by the imaging device mounted on the vehicle 100, and determines whether the arrangement of the two markers 80 is appropriate based on the positional relationship between them. Do. The determination result is notified to the occupant outside the vehicle 100 by lighting and blinking of a headlight or the like. If the arrangement is not appropriate, the occupant rearranges the two markers 80 or selects another parking target position 120 and arranges the two markers 80 there. If the arrangement is appropriate, the parking assistance device generates a parking path from the current position to the parking target position 120. Here, the parking path is indicated by C, D, and E. The vehicle 100 moves along the parking path and parks at the parking target position 120. Specifically, the vehicle 100 moves backward along C, then moves forward along D, and moves backward along E.

  FIG. 3 shows the configuration of the vehicle 100, and in particular shows the configuration for parking assistance. The vehicle 100 includes an imaging device 10, an operation device 12, a display device 14, an information detection unit 16, a notification device 18, a movement control device 20, and a parking assistance device 30. The information detection unit 16 includes a sensor 24. The parking assistance device 30 includes a reception unit 40, a processing unit 42, a display control unit 44, an acquisition unit 46, a notification control unit 52, and a control unit 54. The processing unit 42 includes a detection unit 48, a determination unit 50, and a warning unit 60. The control unit 54 includes a generation unit 56 and a movement processing unit 58. In addition, the devices illustrated in FIG. 3 may be connected by a dedicated line or wired communication such as CAN (Controller Area Network). Alternatively, they may be connected by wired communication or wireless communication such as USB (Universal Serial Bus), Ethernet (registered trademark), Wi-Fi (registered trademark), Bluetooth (registered trademark), or the like.

  The operation device 12 is configured by a button, a dial, and the like, and receives an instruction from the driver. The operating device 12 may be a touch panel configured integrally with the display device 14. The operating device 12 includes, for example, a button for starting parking assistance (hereinafter referred to as “parking assistance start button”), and detects depression of the parking assistance start button by the driver. The reception unit 40 instructs the processing unit 42 to start the parking support when the depression of the parking support start button on the operation device 12 is received, that is, the instruction for the start of the parking support is received. When the processing unit 42 receives the instruction to start parking support, the processing unit 42 instructs the display control unit 44 to display a screen including the “message regarding marker arrangement”.

  When the display control unit 44 receives an instruction from the processing unit 42, the display control unit 44 causes the display device 14 to display a screen including the “message regarding arrangement of markers”. The screen shows the position where the two markers 80 should be placed, and a message for confirming that there is no obstacle at the parking target position 120. FIG. 4 shows a screen displayed on the display device 14. As shown, at the top, a message "Place marker as follows" and an image showing the placement of the marker 80 are displayed. This corresponds to the position where the two markers 80 should be arranged. At the bottom, a message "Please confirm that there are no obstacles near the marker" is displayed. This corresponds to a message for confirming that there is no obstacle at the parking target position 120. Return to FIG.

  Subsequently, as described above, the passenger places two portable markers 80 at the parking target position 120. Here, an example of the marker 80 will be described. 5a-5d are perspective views showing the structure of the marker 80. FIG. FIG. 5 a is a first example of the marker 80. The marker 80 has a rectangular parallelepiped shape elongated in one direction, and an arrow 82 extending along the long side is shown in one surface. Hereinafter, the direction indicated by the arrow 82 is referred to as “front side”, and the opposite direction of “front side” is referred to as “rear side”. FIG. 5 b is a second example of the marker 80. The marker 80 has a vehicle body shape elongated in one direction, one end side in the long direction is a first end 84 a, and the other end is a second end 84 b. The first end 84 a and the second end 84 b are collectively referred to as the end 84. Hereinafter, the first end 84 a side is referred to as “front side”, and the second end 84 b side is notified as “rear side”.

  FIG. 5 c is a third example of the marker 80. The marker 80 has a plate shape whose upper surface is in a substantially positive direction, and the first symbol 86 a to the third symbol 86 c are shown at three corners of the upper surface. Hereinafter, the first end 84 a side in the direction from the second symbol 86 b toward the first symbol 86 a is referred to as “front side”, and the second end 84 b side is referred to as “rear side”. FIG. 5 d is a fourth example of the marker 80. The marker 80 has a disk shape, and an arrow 82 is shown on the circular top surface. The arrow 82 is the same as that of FIG. 5a, so the description is omitted here. As described above, the marker 80 includes elements that can identify the front and back directions. Moreover, the marker 80 may have luminous property.

  6a-6b are top views showing another structure of the marker 80. FIG. The first marker 80a and the second marker 80b have the same shape as the marker 80 of FIG. 5d. The first marker 80 a and the second marker 80 b are connected by an expandable connection portion 88. These are configured like a tape measure. The connecting portion 88 may be foldable. FIG. 6a shows the case where the connecting portion 88 is contracted, and FIG. 6b shows the case where the connecting portion 88 is extended. In particular, two markers 80 in the state of FIG. 6 b, that is, in the state in which the connecting portion 86 is extended, are disposed at the parking target position 120. Return to FIG.

  The imaging device 10 can image the outside of the vehicle 100. Note that the video may be configured by arranging a plurality of images in time series. FIG. 7 shows the arrangement of the imaging device 10, which corresponds to a top view of the vehicle 100. The vehicle 100 includes a first imaging device 10a, a second imaging device 10b, a third imaging device 10c, and a fourth imaging device 10d which are collectively referred to as an imaging device 10. The first imaging device 10a, also referred to as a front camera, is installed on the front grille of the vehicle 100, and images a front area in a direction obliquely looking down with respect to the ground. The second imaging device 10b, also referred to as a left side camera, is installed on the left side mirror of the vehicle 100, and captures an image of the left side area in a direction looking down obliquely to the ground. The third imaging device 10c, also referred to as a rear camera, is installed near the trunk of the vehicle 100, and captures an image of the area behind the vehicle 100 in a direction obliquely looking down with respect to the ground. The fourth imaging device 10d, also referred to as a right side camera, is installed on the right side mirror of the vehicle 100, and images a region on the right side in a direction looking down obliquely to the ground. These imaging devices 10 are wide-angle cameras having an angle of view of, for example, about 180 °, and are arranged so that the entire circumference of the vehicle 100 can be imaged. The number of imaging devices 10 is not limited to "4". The imaging device 10 sequentially outputs the captured images to the parking assistance device 30. Return to FIG.

  The acquisition unit 46 sequentially receives an image captured by each imaging device 10, that is, an image captured of the outside of the vehicle 100. The acquisition unit 46 outputs the image to the processing unit 42 and the control unit 54. The detection unit 48 detects two markers 80 from the image acquired by the acquisition unit 46. For example, the detection unit 48 stores information on the appearance of the marker 80, and detects the two markers 80 by executing an image matching process between this and the image. Known techniques may be used for the image matching process, or pattern matching processes other than the image matching process may be used. In addition, when the marker 80 shakes the structure of FIGS. 6 a-6 b, the detection unit 48 detects the two markers 80 connected by the connection unit 86. The detection unit 48 outputs the image and the detection result of the two markers 80 to the determination unit 50.

  The determination unit 50 determines the direction in which the vehicle 100 is parked at the parking target position 120 specified by the two markers 80, and determines whether the arrangement of the two markers 80 is appropriate in the direction. In order to describe the process in the determination unit 50, FIGS. 8a to 8c will be described here. 8a to 8c show an outline of the processing of the determination unit 50. FIG. In FIG. 8a, the parking target position 120 is surrounded by two short sides and two long sides as before. One of the two short sides is the approach short side 150, and the vehicle 100 enters the parking target position 120 from the approach short side 150 side. The short side opposite to the approach short side 150 is the non-entry side short side 152. Further, the first long side 154 and the second long side 156 are arranged side by side so as to connect the non-entry side short side 152 and the entry side short side 150. FIG. 8a corresponds to the case where the vehicle 100 is parked in parallel. At this time, the occupant arranges the first marker 80a and the second marker 80b at both ends of the entry short side 150, and directs the front side of the first marker 80a and the second marker 80b to the entry short side 150. The front sides of the first marker 80 a and the second marker 80 b correspond to the direction in which the front side of the vehicle 100 faces when the vehicle 100 is parked at the parking target position 120.

  The determination unit 50 receives the detection result of the two markers 80 arranged as shown in FIG. 8A and the image from the detection unit 48, and as shown in FIG. 8b, the first facing the front side from the rear side of the first marker 80a. Detect vector v1. Here, the length of the first vector v1 is a length from the back side to the front side of the first marker 80a, and its value is known. A known technique may be used for this process, so the description is omitted here. In addition, the determination unit 50 detects a second vector v2 pointing from the rear side to the front side of the second marker 80b, similarly to the first vector v1. Further, the determination unit 50 derives a third vector v3 directed from the middle point of the first vector v1 to the middle point of the second vector v2. Following this, the determination unit 50 derives the angle θ between the first vector v1 and the third vector v3. If the angle θ is included in a predetermined range, for example, a range of 45 degrees to 135 degrees or a range of 225 degrees to 315 degrees, the determination unit 50 starts from the approach short side 150 as illustrated in FIG. It is determined that the vehicle 100 is to be parked by entering the parking target position 120. Below, such a state is called "the direction of parallel parking."

  When the determination unit 50 is in the parallel parking direction, the determination unit 50 compares the first threshold value, which is equal to or greater than the entire width of the vehicle 100 and shorter than the entire length of the vehicle 100, with the length of the third vector v3. If the length of the third vector v3 is longer than the first threshold, the determination unit 50 determines that the arrangement of the two markers 80 is appropriate. On the other hand, if the length of the third vector v3 is equal to or less than the first threshold value, the determination unit 50 determines that the arrangement of the two markers 80 is not appropriate.

  9a-9c show an overview of another process of the determination unit 50. FIG. In FIG. 9a, the target parking position 120 is surrounded by two short sides and two long sides as before. One of the two long sides is the approach long side 160, and the vehicle 100 enters the parking target position 120 from the approach long side 160 side. The long side opposite to the approach long side 160 is the non-entry side long side 162. Further, the first short side 164 and the second short side 166 are arranged side by side so as to connect the non-entry side long side 162 and the entry side long side 160. FIG. 9a corresponds to parallel parking of the vehicle 100. At this time, the occupant arranges the first marker 80 a and the second marker 80 b at both ends of the entry side long side 160 and directs the front side of the first marker 80 a and the second marker 80 b to the second short side 166. The front sides of the first marker 80a and the second marker 80b may be directed to the first short side 164, and the direction from the rear side to the front side of the first marker 80a and the second marker 80b is the entrance side long side 160. It should be along. As described above, the front sides of the first marker 80 a and the second marker 80 b correspond to the direction in which the front side of the vehicle 100 faces when the vehicle 100 is parked at the parking target position 120.

  The determination unit 50 receives the detection results of the two markers 80 arranged as shown in FIG. 9A and the image from the detection unit 48, and detects the first vector v1 and the second vector v2 as in the above. . These are shown in FIG. 9b. Further, the determination unit 50 derives a third vector v3 directed from the middle point of the first vector v1 to the middle point of the second vector v2. Following this, the determination unit 50 derives the angle θ between the first vector v1 and the third vector v3. If the angle θ is not included in a predetermined range, the determination unit 50 determines that the vehicle 100 is made to enter the parking target position 120 from the entrance long side 160 and parked as shown in FIG. 9a. Do. Below, such a state is called "direction of parallel parking".

  If the determination unit 50 is in the parallel parking direction, the determination unit 50 compares the second threshold that is equal to or longer than the entire length of the vehicle 100 with the length of the third vector v3. If the length of the third vector v3 is longer than the second threshold, the determination unit 50 determines that the arrangement of the two markers 80 is appropriate. On the other hand, when the length of the third vector v3 is equal to or less than the second threshold value, the determination unit 50 determines that the arrangement of the two markers 80 is not appropriate. Return to FIG.

  As described above, the determination unit 50 determines the direction of the vehicle 100 when the vehicle 100 is parked based on the relationship between the direction connecting the two markers 80 and the direction specified by the at least one marker 80. . Specifically, the direction in which the two markers 80 are connected corresponds to the direction of the third vector v3, and the direction specified from the at least one marker 80 corresponds to the direction of the first vector v1. Further, if the vehicle 100 can pass between the two markers 80, the determination unit 50 determines that the arrangement of the two markers 80 is appropriate, and the vehicle 100 can not pass between the two markers 80. If it is determined that the arrangement of the two markers 80 is not appropriate. 6a to 6b, it is determined whether the vehicle 100 can pass between the two markers 80 in a state in which the connecting portion 86 is extended. Thus, based on the positional relationship between the two markers 80 detected by the detection unit 48, the determination unit 50 determines whether the arrangement of the two markers 80 is valid. The determination unit 50 outputs the determination result to the notification control unit 52 and the control unit 54. In addition, when the determination result indicates that the determination result is valid, the determination unit 50 also outputs the detection results of the two markers 80 to the control unit 54.

  Notification control unit 52 receives the determination result from determination unit 50. The notification control unit 52 stores a table for determining an operation based on the determination result. FIG. 10 shows the data structure of a table stored in the notification control unit 52. As shown in the drawing, operations corresponding to each of the determination results "valid" and "not valid" are shown. Return to FIG. The notification control unit 52 determines the operation by referring to the table. The notification device 18 is, for example, a headlight or a taillight, and is lit or blinks under the control of the notification control unit 52. By means of such notification, the notification control unit 52 notifies that it is not valid when the determination unit 50 determines that it is not valid. The notification device 18 may be a horn, and in this case, the notification control unit 52 changes the manner in which the horn sounds depending on whether the arrangement is appropriate. Furthermore, the notification device 18 may be a communication unit with the outside of the vehicle, and in that case, the notification control unit 52 sends the content corresponding to whether or not the arrangement is appropriate to the portable terminal device in which the passenger has specified in advance. You may make it notify. At this time, the notification control unit 52 may notify the portable terminal device only when the arrangement is not appropriate.

  The information detection unit 16 detects the surrounding state and the traveling state of the vehicle 100. The information detection unit 16 detects, for example, position information of the vehicle 100, an obstacle around the vehicle 100, and a state of the vehicle 100. The information detection unit 16 outputs the detected various information (hereinafter, referred to as “detection information”) to the parking assistance device 30. The information detection unit 16 includes a sensor 24.

  The sensor 24 is a generic name of various sensors for detecting the condition outside the vehicle and the condition of the vehicle 100. For example, a camera, a millimeter wave radar, a light detection and ranging (LIDAR), a sonar, an air temperature sensor, an atmospheric pressure sensor, a humidity sensor, an illuminance sensor, and the like are mounted as sensors for detecting the situation outside the vehicle. The condition outside the vehicle includes the condition of the road on which the vehicle is traveling, the environment including the weather, the condition around the vehicle, and other vehicles in the vicinity. Any information outside the vehicle that can be detected by the sensor 24 may be used. Further, as a sensor 24 for detecting the state of the vehicle 100, for example, an acceleration sensor, a gyro sensor, a geomagnetic sensor, an inclination sensor, a steering angle sensor, a speed sensor, a shift sensor, and the like are mounted.

  The generation unit 56 receives the determination result from the determination unit 50. If the determination result indicates that the result is not valid, the generation unit 56 ends the process. On the other hand, if the determination result indicates that the determination result is valid, the generation unit 56 determines from the current position based on the image received from the acquisition unit 46, the detection information received from the information detection unit 16, and the positions of the two markers 80. A parking path toward the parking target position 120 is generated. In the parking path, the parking target position 120 is entered by passing between the two markers 80. A known technique may be used to generate a parking path, so the description is omitted here. The generation unit 56 outputs the parking path to the movement processing unit 58.

  The movement processing unit 58 receives the parking path from the generation unit 56. The movement processing unit 58 generates steering angle information for executing the movement along the parking path, based on the detection information received from the information detection unit 16. In addition, the movement processing unit 58 applies steering angle information or various information collected from various ECUs to an algorithm of automatic parking to calculate a control value for controlling an automatic control target of the steering angle of the vehicle 100. The movement processing unit 58 also calculates the movement speed of the vehicle 100 and includes the movement speed in the control value. The movement processing unit 58 outputs the control value to the movement control device 20.

  The movement control device 20 is an automatic driving controller implemented with an automatic driving control function, and determines an action of the vehicle 100 in automatic parking. In addition, the description regarding an automatic driving | operation is abbreviate | omitted below. The configuration of the mobility control device 20 can be realized by cooperation of hardware resources and software resources, or hardware resources only. A processor, ROM (Read Only Memory), RAM (Random Access Memory), and other LSIs can be used as hardware resources, and programs such as operating systems, applications, and firmware can be used as software resources. The movement control device 20 transmits the control value input from the parking assistance device 30 to each control target ECU or controller. In the present embodiment, the information is transmitted to the steering ECU, the engine ECU, and the brake ECU.

  After parking the vehicle 100 at the parking target position 120 along the parking path by executing the parking support described above, the occupant moves the vehicle 100 from the parking target position 120. Here, processing when moving the vehicle 100 will be described. The processing unit 42 recognizes the start of the movement of the vehicle 100 based on the detection information received from the information detection unit 16 when the engine is activated. For example, in response to this, the acquisition unit 46 acquires an image when the parked vehicle 100 moves, and the detection unit 48 receives the image. The detection unit 48 detects the two markers 80 from the image acquired by the acquisition unit 46 by executing the above-described processing. If the detection unit 48 does not detect two markers 80, the process ends. On the other hand, when the detection unit 48 detects two markers 80, the detection unit 48 notifies the warning unit 60 of detection of the two markers 80.

  When the warning unit 60 receives a notification from the detection unit 48, the warning unit 60 instructs the display control unit 44 to display a screen for warning of misplacement of the two markers 80. When the display control unit 44 receives an instruction from the warning unit 60, the display control unit 44 causes the display device 14 to display a screen including a “message for warning that the marker is forgotten”. FIG. 11 shows another screen displayed on the display device 14. As shown, the message "Forget markers" is displayed. Return to FIG.

  In the above description, two markers 80 are used, but only one marker 80 may be used. At that time, the determination of the validity of the arrangement of the markers 80 in the determination unit 50 is omitted. When there is one marker 80, it is determined in advance which end of the entrance short side 150 or the entrance long side 160 of the parking target position 120 the marker 80 should be disposed. In order to arrange the marker 80 at the end on the predetermined side, the screen including the “message regarding the arrangement of the marker” may be as shown in FIG. 12 instead of FIG. FIG. 12 shows still another screen displayed on the display device 14. Here, it is determined in advance that the marker 80 is disposed at the end on the inner wheel side of the vehicle 100 at the entrance short side 150 or the entrance long side 160 of the parking target position 120, and the marker 80 is An image for placement is shown. When it is determined in advance that marker 80 is to be arranged at the end on the outer ring side of vehicle 100 at entry side short side 150 or entry side long side 160 of parking target position 120, an image according to that is shown. May be Return to FIG.

  So far, two markers 80 have been used to signal the parking target position 120. The two markers 80 may be used in different applications. FIG. 13 shows an outline of still another process of the determination unit 50. As shown, the marker 80 is disposed so that the front side of the marker 80 is directed in the vertical direction. The detection unit 48 detects these markers 80, and the determination unit 50 determines that the front side of the markers 80 is directed in the vertical direction. In response to this determination, the generation unit 56 generates the virtual wall 70 in the portion sandwiched by the two markers 80 and generates a parking path so as not to pass the virtual wall 70. Two such markers 80 may be used with the two markers 80 described above. In this case, the former two markers 80 designate a part where passage is prohibited, and the latter two markers 80 indicate the parking target position 120.

  The operation of the parking assistance device 30 having the above configuration will be described. FIG. 14 is a flowchart showing a parking assistance procedure by the parking assistance device 30. The reception unit 40 receives an instruction to start parking assistance (S10). The display control unit 44 displays a message on the display device 14 (S12). If the detection unit 48 does not detect the marker 80 (N in S14), a voice such as "Please put the marker correctly" or "The marker is not detected" is reported to the outside of the vehicle (S15). Perform the detection operation again. When the detection unit 48 detects the marker 80 (Y in S14), the determination unit 50 determines the direction of the vehicle 100 (S16). If the placement of the marker 80 is not appropriate (N in S18), an audio such as "Please put the marker correctly" or "Please replace the marker correctly" is reported to the outside of the vehicle (S19) and return to step 14 . If the arrangement of the markers 80 is appropriate (Y in S18), the generation unit 56 generates a parking path (S20). The movement processing unit 58 executes parking (S22). It should be noted that N in S14 and N in S18 may be notified to a portable terminal device designated in advance by a passenger in addition to or in addition to notification by voice.

  FIG. 15 is a flowchart showing a warning procedure by the parking assistance device 30. The vehicle 100 is parked (S50). The processing unit 42 detects the movement of the vehicle 100 (S52). When the detection unit 48 detects the marker 80 (Y in S54), the warning unit 60 displays a warning (S56). If the detection unit 48 does not detect the marker 80 (N in S54), the process ends.

  According to the present embodiment, it is determined whether or not the arrangement of two markers is appropriate, so parking assistance is performed even when manually arranging a plurality of portable markers. it can. In addition, parking assistance is performed even if two markers are placed manually, so parking assistance using markers can be performed in various environments. Also, since only two markers are placed, the parking target position can be easily set. In addition, since the marker has luminous property, the marker can be detected even if the surrounding area is dark. In addition, since markers are detected even when the environment is dark, parking assistance using markers can be performed in various environments.

  Further, if the vehicle can not pass between the two markers, it is determined that the arrangement of the two markers is not appropriate, so that the determination accuracy can be improved. In addition, since notification is given that the arrangement of the two markers is not appropriate, it is possible to prompt the rearrangement of the markers. Moreover, since the marker is rearranged by this notification, parking assistance can be easily performed. In addition, since the direction of the vehicle when the vehicle is parked is determined, it is possible to support parking even in parallel parking or parallel parking.

  In addition, since a message for confirming that there is no obstacle at the parking target position is displayed, it is possible to make the occupant confirm that there is no obstacle. Further, since the position where the two markers are to be arranged is displayed, it is possible to inform the occupant of the position where the two markers are to be arranged. In addition, since the occupant is notified of the position where the two markers should be arranged, parking assistance using the markers can be performed smoothly. In addition, warning of misplacement of two markers can prevent misplacement of two markers. In addition, since two markers are manually placed, parking assistance can be performed even in a parking area without a white line such as a mechanical parking lot. In addition, since two markers can be placed at arbitrary positions, it is also possible to close the vehicle and park it.

  Although the embodiments according to the present disclosure have been described in detail with reference to the drawings, the functions of the above-described apparatus and each processing unit may be realized by a computer program. The computer that realizes the functions described above by a program includes input devices such as a keyboard, a mouse, and a touch pad, output devices such as a display and a speaker, a central processing unit (CPU), a ROM, a RAM, a hard disk drive, and a solid state drive (SSD). Etc., a reading device that reads information from a recording medium such as a DVD-ROM (Digital Versatile Disk Read Only Memory) or a USB memory, a network card that communicates via a network, etc. . Further, the reading device reads the program from the recording medium having the program recorded thereon and stores the program in the storage device. In addition, the function of each device is realized by the CPU copying the program stored in the storage device to the RAM, sequentially reading out the instructions included in the program from the RAM, and executing the instruction.

  The outline of one aspect of the present disclosure is as follows. A parking assistance device according to an aspect of the present disclosure is a parking assistance device that assists parking of a vehicle using a plurality of markers after a plurality of portable markers are arranged at a parking target position. An arrangement of a plurality of markers based on the positional relationship between an acquisition unit that acquires an image obtained by imaging the outside, a detection unit that detects a plurality of markers from the image acquired by the acquisition unit, and a plurality of markers detected by a detection unit And a generation unit that generates a parking path based on the positions of a plurality of markers when it is determined that the determination unit is appropriate.

  According to this aspect, since it is determined whether or not the arrangement of the plurality of markers is appropriate, even in the case of arranging the plurality of markers having portability, the parking target position is specified, so various environments Parking assistance using markers can be performed.

  The plurality of markers are luminous. In this case, since the plurality of markers have luminous properties, the plurality of markers can be detected even if the surrounding area is dark.

  The detection unit detects two markers, and the determination unit determines that the arrangement of the plurality of markers is not valid if the vehicle can not pass between the two markers detected by the detection unit. In this case, if the vehicle can not pass between the two markers, it is determined that the arrangement of the two markers is not appropriate, so the determination accuracy can be improved.

  The detection unit detects two markers connected by the expandable connection unit, and the determination unit is the two markers detected by the detection unit and between the two markers in a state in which the connection unit is extended. If the vehicle can not pass, it is determined that the arrangement of the plurality of markers is not appropriate. In this case, if the vehicle can not pass between the two markers, it is determined that the arrangement of the two markers is not appropriate, so the determination accuracy can be improved.

  The determination unit may further include a notification control unit that notifies that the determination is not appropriate. In this case, since notification is made that it is not appropriate, it is possible to prompt the repositioning of the marker.

  The plurality of markers may have an element whose direction can be identified. The determination unit determines the direction of the vehicle when the vehicle is parked based on the relationship between the direction connecting the plurality of markers and the direction specified by the at least one marker. In this case, since the direction of the vehicle at the time of parking the vehicle is determined, parking can be supported even in parallel parking or parallel parking.

  A reception control unit that receives an instruction to start parking assistance, and a display control unit that displays a message for confirming that there is no obstacle at the parking target position when the reception unit accepts a parking assistance start instruction; May further be provided. In this case, a message for confirming that there is no obstacle at the parking target position is displayed, so that the occupant can be confirmed that there is no obstacle.

  The display control unit displays the position where the plurality of markers are to be arranged. In this case, since the position where the plurality of markers should be arranged is displayed, it is possible to inform the occupant of the position where the plurality of markers should be arranged.

  The acquisition unit acquires an image when the vehicle parked along the parking path generated by the generation unit moves, and the parking assistance apparatus detects a plurality of markers from the image acquired by the acquisition unit by the detection unit. In the case, a warning unit may be further provided to warn of misplacement of a plurality of markers. In this case, since a warning is given to the misplacement of a plurality of markers, it is possible to prevent the misplacement of markers.

  Another aspect of the present disclosure is a vehicle. This vehicle is a vehicle, and after a plurality of portable markers are arranged at a parking target position, a parking assist device that assists parking of the vehicle using the plurality of markers and an image of the outside of the vehicle can be taken And an imaging device. The parking assist apparatus determines whether the arrangement of the plurality of markers is appropriate based on the positional relationship between the detection unit that detects the plurality of markers from the image captured by the imaging device and the plurality of markers detected by the detection unit. And a generation unit that generates a parking path based on the positions of a plurality of markers when it is determined that the determination unit is valid.

  According to this aspect, since it is determined whether or not the arrangement of the plurality of markers is appropriate, even in the case of arranging the plurality of markers having portability, the parking target position is specified, so various environments Parking assistance using markers can be performed.

  The present disclosure has been described above based on the embodiments. It is understood by those skilled in the art that these embodiments are exemplifications, and that various modifications can be made to the combinations of the respective components and processing processes, and such modifications are also within the scope of the present disclosure. It is a place.

  In the present embodiment, the determination unit 50 determines the direction of the vehicle 100 when parking at the parking target position 120. However, the determination unit 50 may omit the determination of the direction of the vehicle 100 when parking at the parking target position 120. At this time, whether the vehicle 100 parks in parallel or in parallel may be set by the occupant. Or, it may be determined to one in which either. According to this modification, processing can be simplified.

  In the present embodiment, the number of markers 80 is 2 or less. However, the number of markers 80 may be three or more. According to this modification, the flexibility of processing can be improved.

  According to the present disclosure, parking assistance using markers can be performed in various environments.

  DESCRIPTION OF SYMBOLS 10 imaging apparatus, 12 operation apparatus, 14 display apparatus, 16 information detection part, 18 notification apparatus, 20 movement control apparatus, 24 sensor, 30 parking assistance apparatus, 40 reception part, 42 processing part, 44 display control part, 46 acquisition part , 48 detection units, 50 judgment units, 52 notification control units, 54 control units, 56 generation units, 58 movement processing units, 60 warning units, 80 markers, 100 vehicles.

Claims (10)

A parking assist apparatus for assisting parking of a vehicle using a plurality of markers after the plurality of markers having portability are arranged at a parking target position,
An acquisition unit configured to acquire an image obtained by imaging the outside of the vehicle;
A detection unit that detects the plurality of markers from the image acquired by the acquisition unit;
A determination unit that determines whether the arrangement of the plurality of markers is appropriate based on the positional relationship of the plurality of markers detected by the detection unit;
A generation unit that generates a parking path based on the positions of the plurality of markers when it is determined that the determination unit is valid;
Parking assistance device provided with. The plurality of markers are luminous.
The parking assistance device according to claim 1. The detection unit detects two markers,
The determination unit determines that the arrangement of the plurality of markers is not valid if the vehicle can not pass between the two markers detected by the detection unit.
The parking assistance apparatus of Claim 1 or 2. The detection unit detects two markers connected by an expandable connection unit,
The determination unit is the two markers detected by the detection unit, and if the vehicle can not pass between the two markers in a state in which the connection portion is extended, the plurality of markers Determine that the placement is not valid,
The parking assistance apparatus of Claim 1 or 2. A notification control unit for notifying that the validity is not determined when the determination unit determines that the validity is not valid;
The parking assistance device according to any one of claims 1 to 4, further comprising: The plurality of markers have elements whose direction can be identified,
The determination unit determines the direction of the vehicle when the vehicle is parked based on a relationship between a direction connecting the plurality of markers and a direction specified by at least one marker.
The parking assistance apparatus of Claim 1 or 2. A reception unit that receives an instruction to start parking assistance;
A display control unit that displays a message for confirming that there is no obstacle at the parking target position when the reception unit receives an instruction to start parking assistance;
The parking assistance device according to any one of claims 1 to 6, further comprising: The display control unit displays a position at which the plurality of markers are to be arranged.
The parking assistance device according to claim 7. The acquisition unit acquires an image when the vehicle parked along the parking path generated by the generation unit moves.
The parking assistance device
A warning unit that warns of the misplacement of the plurality of markers when the detection unit detects the plurality of markers from the image acquired by the acquisition unit;
The parking assistance device according to claim 1, further comprising: A vehicle,
A parking assist device that assists parking of the vehicle using the plurality of markers after the plurality of portable markers are arranged at the parking target position;
And an imaging device capable of imaging the outside of the vehicle,
The parking assistance device
A detection unit that detects the plurality of markers from the image captured by the imaging device;
A determination unit that determines whether the arrangement of the plurality of markers is appropriate based on the positional relationship of the plurality of markers detected by the detection unit;
A generation unit that generates a parking path based on the positions of the plurality of markers when it is determined that the determination unit is valid;
Vehicles equipped with

Images