CN112748125A

CN112748125A - Vehicle appearance inspection system and method, vehicle, and parking position display

Info

Publication number: CN112748125A
Application number: CN201911036670.8A
Authority: CN
Inventors: 寺内章; 加藤奈保树; 深泽香代子; 友杉宏之; 饭岛贤大
Original assignee: Honda Motor Co Ltd
Current assignee: Honda Motor Co Ltd
Priority date: 2019-10-29
Filing date: 2019-10-29
Publication date: 2021-05-04

Abstract

The invention provides a vehicle appearance inspection system and a method thereof, a vehicle and a parking position display. An exterior abnormality such as a flaw or dirt on the exterior of a vehicle is detected quickly and at low cost without making a leak. The disclosed device is provided with: an on-vehicle image pickup device that picks up an image of an exterior of the vehicle; a mirror provided on a structure disposed at a parking position of the vehicle; and at least one processing device configured to capture an external view image of the vehicle reflected on the reflecting mirror by the image capturing device when the vehicle is parked at the parking position, determine whether there is an abnormality in the appearance of the vehicle based on the external view image, and generate information on the detected abnormality when the abnormality in the appearance of the vehicle is detected.

Description

Vehicle appearance inspection system and method, vehicle, and parking position display

Technical Field

The present invention relates to a vehicle appearance inspection system and a vehicle appearance inspection method for detecting whether or not there is an abnormality such as a flaw or dirt in the appearance of a vehicle in order to inspect the appearance of the vehicle.

Background

As a vehicle travels, abnormalities such as scars, pits, dirt, and the like may occur in the appearance of the vehicle due to contact with small stones, mud, insects flying from vegetation beside the road, and the like on the road. Such an abnormality in appearance can be detected by visual inspection by a user of the vehicle, i.e., a driver or a service person at a maintenance factory.

However, in the case of performing the appearance inspection by the user, i.e., the driver, a corresponding time is required, and omission is easily generated when abnormality is found. In addition, when a service person in a maintenance factory performs an appearance inspection, a considerable time often passes from the occurrence of an abnormality such as a flaw to the detection of the abnormality. Therefore, for example, when the vehicle is a rental vehicle, the appearance abnormality is not recognized for a long time, and it is not appropriate to continuously rent the vehicle having the flaw or the like to a plurality of users.

Conventionally, there have been known in-vehicle inspection devices and in-vehicle inspection methods for automatically inspecting whether or not an article left behind is present in a room of an automobile used for automobile sharing or the like (see patent document 1). However, since this device is provided with a dedicated camera for detecting a left behind article or the like in the vehicle interior, there is a possibility that the device will be costly for additional use and that it will not be possible to grasp the presence or absence of dirt on the exterior (appearance) of the vehicle.

Documents of the prior art

Patent document

Patent document 1: japanese patent laid-open publication No. 2013-191053

Disclosure of Invention

Problems to be solved by the invention

In light of the above background, there is a demand for a technique for detecting an abnormality in appearance such as a flaw or dirt on the exterior of a vehicle quickly and at low cost without making a vehicle overlook.

Means for solving the problems

One aspect of the present invention is a vehicle appearance inspection system including: an imaging device that is provided in a vehicle and that images the outside of the vehicle; a mirror provided on a structure disposed at a parking position of the vehicle; and at least one processing device configured to capture an external view image of the vehicle reflected on the mirror by the imaging device when the vehicle is parked at the parking position, determine whether or not there is an abnormality in the appearance of the vehicle based on the external view image, and generate information on the detected abnormality when the abnormality in the appearance of the vehicle is detected.

According to another aspect of the present invention, a mark indicating a position of the mirror is provided on the structure.

According to another aspect of the present invention, the processing device is configured to determine whether or not there is an abnormality in the appearance of the vehicle based on the external view image in response to detection of the mark.

According to another aspect of the present invention, the structure is a fixed or movable parking position display disposed in a parking space of the vehicle.

According to another aspect of the present invention, the structure is a parking pile, a parking cone, a parking obstacle, or a parking gate disposed in a traveling path up to a parking space or disposed in the parking space.

According to another aspect of the present invention, the structure is disposed outside in a vehicle width direction of the vehicle placed at the parking position.

According to another aspect of the present invention, the processing device is configured to repeatedly capture, by the imaging device, a surrounding image including a partial image showing a part of the vehicle reflected on the mirror at predetermined time intervals, and to construct the external view image of the vehicle from the repeatedly captured surrounding image.

According to another aspect of the present invention, the processing device is configured by one processing device mounted on the vehicle, and is configured to transmit the information on the detected abnormality to a server outside the vehicle.

According to another aspect of the present invention, the processing device includes a 1 st processing device mounted on the vehicle and a 2 nd processing device provided in a server disposed outside the vehicle, the 1 st processing device is configured to capture an external view image of the vehicle reflected on the mirror by the imaging device and transmit the external view image to the server through a communication device provided in the vehicle when the vehicle is parked at the parking position, and the 2 nd processing device is configured to determine whether or not there is an abnormality in the appearance of the vehicle based on the external view image received by the server from the 1 st processing device and to generate information on the detected abnormality when the abnormality in the appearance of the vehicle is detected.

According to another aspect of the present invention, the vehicle is a rental vehicle rented to a user, and the structure is a fixed or movable parking position display disposed in a parking space for rental of the vehicle.

Another aspect of the present invention is a vehicle that constitutes the vehicle appearance inspection system, the vehicle including the imaging device and the processing device.

Another aspect of the present invention is a parking position display that is movable, that indicates a parking space of a vehicle, and that includes a mirror and a mark indicating a position of the mirror.

Another aspect of the present invention is a vehicle appearance inspection method executed by at least one processing device, including: when a vehicle is parked at a parking position, capturing an external view image of the vehicle reflected on a mirror provided on a structure disposed at the parking position by an imaging device mounted on the vehicle; judging whether the appearance of the vehicle is abnormal or not according to the external scene image; and generating information on the detected abnormality when the abnormality of the appearance of the vehicle is detected.

ADVANTAGEOUS EFFECTS OF INVENTION

According to the present invention, it is possible to detect an abnormality in appearance such as a flaw or dirt on the exterior of a vehicle quickly and at low cost without making a leak.

Drawings

Fig. 1 is a diagram showing an example of the configuration of a vehicle appearance inspection system according to an embodiment of the present invention.

Fig. 2 is a diagram showing an example of a front outside view image captured by a front camera mounted on the vehicle in the vehicle appearance inspection system shown in fig. 1.

Fig. 3 is a diagram showing an example of a left outside view image captured by a left side view camera mounted on the vehicle in the vehicle appearance inspection system shown in fig. 1.

Fig. 4 is a diagram showing an example of a right side image captured by a right side angle-of-view camera mounted on the vehicle in the vehicle appearance inspection system shown in fig. 1.

Fig. 5 is a diagram showing an example of a right-side outside view image obtained by combining a plurality of right-side images captured by right-side view cameras in the vehicle appearance inspection system shown in fig. 1.

Fig. 6 is a diagram showing an example of the structure of the appearance inspection ECU mounted on the vehicle.

Fig. 7 is a diagram showing an example of the configuration of the management server.

Fig. 8 is a flowchart illustrating steps of an operation of the vehicle appearance inspection system shown in fig. 1.

Description of the reference symbols

100 … vehicle visual inspection system, 102 … vehicle, 104 … parking space, 106 … parking lot, 108 … parking rod, 110 … partition, 112, 114, 116 … mirror, 118, 120, 122, 124, 126, 128 … mark, 130 … front camera, 132 … left door mirror, 134 … right door mirror, 136 … left side view camera, 138 … right side view camera, 140 … visual inspection ECU, 142 … communication network, 144 … management server, 200 … front exterior view image, 202 … lot image, 204, 410 … background image, 206, 304, 404 … mirror image, 208, 210, 306, 308, 406, 408 … mark image, 300 … left side exterior view image, 302 … panel image, 310 … left side vehicle image, 400 right side image, 402, … rod image, 36412 right side image, 500 … exterior view image, … bus image, 36600 bus image, 602 … state detection ECU, 604 … image acquisition ECU, 606 … TCU, 608 … navigation device, 610 … GPS receiver, 612 … vehicle speed sensor, 614 … ignition switch, 616 … display device, 620, 700 … processing device, 622, 702 … storage device, 624, 704 … communication device, 630 … regression detection unit, 632 … image acquisition unit, 634 … abnormality detection unit, 636 … information output unit, 710 … utilization end detection unit, 712 … vehicle operation instruction unit

Detailed Description

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

Fig. 1 is a diagram showing an example of a vehicle appearance inspection system according to an embodiment of the present invention. The vehicle appearance inspection system 100 shown in fig. 1 includes a vehicle 102 to be subjected to appearance inspection, and a parking post 106 disposed in the periphery of a parking space 104 of the vehicle 102 and serving as a movable parking position display. Here, the parking position display means a fixed or movable structure indicating that the position is a parking space. The vehicle appearance inspection system 100 further includes a parking lever 108 including a parking mark (a mark denoted by "P" in the drawing) and a partition 110 for distinguishing the parking space 104 from other parking spaces, for example, as a fixed parking position display.

In particular, in the present embodiment, the parking peg 106, the parking lever 108, and the spacer 110 are provided with reflecting

mirrors

112, 114, and 116, respectively. In addition,

marks

118 and 120, 122 and 124, and 126 and 128 indicating the positions of the

mirrors

112, 114 and 116, respectively, are provided on the parking peg 106, the parking rod 108 and the spacer 110. The

indicia

118, 120, 122, 124, 126, 128 each include a predetermined recognizable pattern. The pattern may be a geometric pattern or a pattern constituting a bar code, for example. Hereinafter, the

mirrors

112, 114, 116 are also collectively referred to as the mirror 112, and the like, and the

marks

118, 120, 122, 124, 126, 128 are also collectively referred to as the marks 118, and the like.

The parking pile 106 is moved by the user of the vehicle 102 along the arrow of the dotted line after the vehicle 102 is parked, for example, and is disposed at the position of the dotted line so that the mirror 112 faces the direction of the vehicle 102. Here, the parking peg 106, the parking rod 108, and the spacer 110 correspond to a structure disposed at a parking position of the vehicle, and the

mirrors

112, 114, and 116 correspond to mirrors provided on the structure. The parking rod 108 and the bulkhead 110 correspond to a structure disposed outside the vehicle in the vehicle width direction of the vehicle placed at the parking position.

The vehicle 102 is provided with a front camera 130 that captures an image of the front of the vehicle, and a left-side angle-of-view camera 136 and a right-side angle-of-view camera 138 that are provided on a left door mirror 132 and a right door mirror 134, respectively, and capture the left side and the right side of the vehicle 102. The front camera 130 may be, for example, a wide-angle camera for safety check, which is used to photograph another vehicle approaching an intersection road toward the intersection when the vehicle 102 enters the intersection with poor visibility.

Similarly, the left-side view camera 136 and the right-side view camera 138 may be wide-angle cameras for safety check, for example, which cover blind spots of the driver during lane change on an expressway. Here, the front camera 130, the left side angle-of-view camera 136, and the right side angle-of-view camera 138 correspond to an image pickup device that is provided in a vehicle and picks up an image of the outside of the vehicle. Hereinafter, the front camera 130, the left-side angle-of-view camera 136, and the right-side angle-of-view camera 138 are also collectively referred to as the front camera 130 and the like.

The vehicle 102 is also provided with an appearance check ECU 140. The appearance check ECU 140 acquires an external view image of the front of the vehicle 102 reflected on the mirror 112 of the parking post 106 by using the front camera 130. The appearance check ECU 140 acquires external view images of the right and left sides of the vehicle 102 reflected on the mirror 114 of the parking lever 108 and the mirror 116 of the dash panel 110 by the right and left

viewing angle cameras

138 and 136.

The appearance check ECU 140 processes the external view images acquired from the front camera 130, the left-side view camera 136, and the right-side view camera 138 to detect an abnormality in the appearance of the vehicle 102. Here, the

mirrors

112 and 116 may be configured as convex mirrors, for example, so that the entire front and left side surfaces of the vehicle 102 are projected by the front camera 130 and the left side view camera 136. The mirror 114 provided on the parking lever 108 may be a two-dimensional convex mirror along the outer surface of the cylindrical parking lever 108, or a flat mirror extending in the longitudinal direction of the parking lever 108.

The vehicle appearance inspection system 100 may further include a management server 144, and the management server 144 may be communicatively connected to the vehicle 102 (more specifically, the appearance inspection ECU 140 of the vehicle 102) via a communication network 142 such as the internet. For example, in the present embodiment, the vehicle 102 is a rental vehicle rented to a plurality of users, and the management server 144 is a server that manages the usage status of the vehicle 102 as the rental vehicle.

Fig. 2 is a diagram showing an example of an external view image (front external view image) 200 in front of the vehicle 102 acquired from the front camera 130 by the appearance inspection ECU 140 when the parking post 106 is disposed in front of the vehicle 102 (for example, at the position indicated by the dotted line shown in fig. 1).

The front outside view image 200 includes, for example, an image (pile image) 202 of the upper half of the parking pile 106, an image (mirror image) 206 of the mirror 112, and a front vehicle image 212, which is an appearance image of the front of the vehicle 102 and is reflected inside the mirror image 206. The front outside view image 200 also includes images (mark images) 208 and 210 of the

marks

118 and 120 indicating the positions of the mirrors 112, and a background image 204 that is the view of the surroundings on the side opposite to the vehicle 102 with the parking post 106 interposed therebetween.

The appearance check ECU 140 determines whether or not the

marker images

208 and 210 are included in the front outside view image 200, for example, and extracts and stores the front vehicle image 212 from the images in the section sandwiched between the

marker images

208 and 210 in the left-right direction in the figure when the marker images are included. The appearance check ECU 140 compares the preceding vehicle image 212 acquired and stored as described above with the preceding vehicle image 212 acquired this time, and determines whether or not there is an abnormality in appearance such as a flaw or dirt in the front surface of the vehicle 102. Then, the appearance inspection ECU generates and stores information about the detected abnormality (for example, information on the position, size, and the like of the abnormality) when the abnormality is detected.

Fig. 3 is a diagram showing an example of an external view image (left external view image) 300 of the left side of the vehicle 102 acquired by the appearance inspection ECU 140 from the left side view camera 136.

The left outside scene image 300 includes, for example, an image (panel image) 302 of the diaphragm 110, an image (mirror image) 304 of the mirror 116 included in the panel image 302, and images (marker images) 306 and 308 of the

markers

126 and 128. Further, the mirror image 304 includes a left side vehicle image 310 which is an external appearance image of the left side of the vehicle 102.

The appearance check ECU 140 processes the left outside view image 300 and determines whether or not the

marker images

306 and 308 are included in the left outside view image 300. When these marker images are included, the left side vehicle image 310 is extracted from the images in the section sandwiched between the

marker images

306 and 308 in the left-right direction in the figure and stored. The appearance check ECU 140 compares the left side vehicle image 310 acquired and stored as described above with the left side vehicle image 310 acquired this time, and determines whether or not there is an appearance abnormality such as a flaw or dirt on the left side of the vehicle 102. Then, the appearance inspection ECU 140 generates and stores information about the detected abnormality when the abnormality is detected.

Fig. 4 is a diagram showing an example of a right-side image (right-side image) 400 that is an image of the periphery of the vehicle 102 and is acquired by the appearance inspection ECU 140 from the right-side viewing angle camera 138. The right side image 400 of the vehicle 102 taken by the camera 138 from the right side view angle includes, for example, an image (pole image) 402 of the parking pole 108, an image (mirror image) 404 of the mirror 114 that is a part of the pole image 402, images (marker images) 406 and 408 of the markers 122 and 124, and a background image 410 on the back of the parking pole 108. In addition, a partial image 412 of the right side surface of the vehicle 102 is included in the mirror image 404.

As shown in fig. 4, since the entire right side surface of the vehicle 102 is not reflected on the mirror 114, the appearance check ECU 140 repeatedly captures a right side image 400, which is a surrounding image including a partial image 412 of the vehicle 102, at predetermined time intervals as the vehicle 102 moves (for example, as the vehicle 102 moves backward toward the parking space 104). Then, the appearance check ECU 140 extracts, for example, the bar images 402 including the partial image 412 of the vehicle 102 from the respective right side images 400, and arranges the bar images 402 in time series to obtain the right side outside view image of the vehicle 102.

Fig. 5 is a diagram showing an example of the right side outside view image configured as described above. The right outside view image 500 shown in fig. 5 is formed by arranging 20 bar images 402 including the partial image 412 of the vehicle 102, that is, the bar images 402-1, 402-2, … …, 402-20, in chronological order (that is, in the order of capturing) from right to left, which are repeatedly captured as the vehicle 102 moves. The appearance check ECU 140 virtually arranges and synthesizes the plurality of bar images 402 as shown in fig. 5, thereby constructing a right-side outside view image 500 including a right-side vehicle image 502 representing the appearance of the right side surface of the vehicle 102.

Then, the appearance inspection ECU 140 extracts and stores a right side vehicle image 502 from the constructed right side outside view image 500. The appearance check ECU 140 compares the right side vehicle image 502 acquired and stored as described above with the right side vehicle image 502 acquired this time, and determines whether or not there is an appearance abnormality such as a flaw or dirt on the right side of the vehicle 102. Then, the appearance inspection ECU 140 generates and stores information about the detected abnormality when the abnormality is detected.

Hereinafter, the front external view image, the left external view image, and the right external view image, such as the front external view image 200, the left external view image 300, and the right external view image 500, are collectively referred to as external view images, and the front vehicle image, such as the front vehicle image 212, the left vehicle image 310, and the right vehicle image 502, the left vehicle image, and the right vehicle image, are collectively referred to as vehicle images.

The processing operation of the appearance abnormality detection based on the whole image of the vehicle 102 shown in fig. 2 and 3 reflected in one mirror is referred to as whole image mode processing, and the processing of the appearance abnormality detection based on the vehicle image constructed using the partial image of the vehicle 102 shown in fig. 5 reflected in one mirror is referred to as partial image mode processing. Depending on whether the shape of the mirror 112 or the like is to be subjected to the whole image mode processing or the partial image mode processing, it can be indicated by information (for example, a graphic, a barcode, or the like) indicated by a mark 118 or the like provided in the vicinity of the mirror.

For example, when the mark 118 or the like detected in one acquired image is a mark indicating partial image pattern processing, the appearance inspection ECU 140 performs the above-described processing in relation to fig. 5, and can combine the external view images of the vehicle 102 from a plurality of surrounding images and extract the vehicle image from the combined external view image. For example, in the embodiment shown in fig. 1, the

marks

118, 120, 126, and 128 can display information indicating the whole image mode processing, and the marks 122 and 124 can display information indicating the partial image mode processing.

Fig. 6 is a diagram showing an example of the structure of the appearance inspection ECU 140. Appearance inspection ECU 140 is communicably connected to state detection ECU 602, image acquisition ECU 604, TCU (Telematics Control Unit) 606, navigation device 608, and GPS receiver 610 via network bus 600 constituting an in-vehicle network mounted on vehicle 102. Here, the Network bus 600 is, for example, a CAN (Control Area Network) bus for performing communication conforming to CAN communication standards.

The state detection ECU 602 controls a sensor that detects the operating state of the vehicle 102, and acquires information from the sensor. In the present embodiment, the state detection ECU 602 detects, for example, the on/off state of the ignition switch 614, and obtains the vehicle speed of the vehicle 102 using the vehicle speed sensor 612.

The image acquisition ECU 604 controls the front camera 130, the left-side view camera 136, and the right-side view camera 138, and acquires images from these cameras.

The TCU 606 includes a wireless device, and communicates with the management server 144, for example, via the communication network 142. The GPS receiver 610 receives GPS radio waves from GPS satellites and outputs current position information of the vehicle 102. The navigation device 608 acquires current position information from the GPS receiver 610, and displays the acquired current position together with map information on a display unit 616 provided in the navigation device 608. The navigation device 608 searches for a route to the destination input by the current user of the vehicle 102 and displays the route on the display device 616.

Appearance inspection ECU 140 includes processing device 620, storage device 622, and communication device 624. The storage device 622 is configured by, for example, a volatile and/or nonvolatile semiconductor memory, a hard disk device, or the like. The storage device 622 stores data necessary for the processing by the processing device 620 in advance or stores necessary data along with the processing. The communication device 624 is, for example, a CAN transceiver for communicating with another device such as the image acquisition ECU 604 via the network bus 600 as a CAN bus.

The Processing device 620 is a computer including a processor such as a CPU (Central Processing Unit). The processing device 620 may have a configuration including a ROM (Read Only Memory) in which a program is written, a RAM (Random Access Memory) for temporarily storing data, and the like. The processing device 620 includes a regression detection unit 630, an image acquisition unit 632, an abnormality detection unit 634, and an information output unit 636 as functional elements or functional units.

These functional elements included in the processing device 620 are realized by executing a program by the processing device 620 as a computer, for example. The computer program can be stored in advance in any computer-readable storage medium. Alternatively, all or a part of the functional elements included in the processing device 620 may be configured by hardware including one or more electronic circuit components.

Fig. 8 is a flowchart showing steps of the operation of the vehicle appearance inspection system 100 including the appearance inspection ECU 140. The operation of each functional element included in the processing device 620 will be described below with reference to a flowchart shown in fig. 8.

In fig. 8, the vehicle appearance inspection system 100 first determines whether or not a start condition for starting the process of appearance inspection is satisfied (S100). In the present embodiment, the start condition is that the vehicle 102 reaches the vicinity (for example, within 10 m) of the predetermined parking space 104. Hereinafter, the arrival in the vicinity of the parking space 104 is also referred to as "return".

Specifically, the regression detection unit 630 of the processing device 620 determines whether or not the vehicle 102 has returned to the predetermined parking space 104. The return detection unit 630 acquires the current position of the vehicle 102 from the navigation device 608 via the network bus 600, and determines whether or not the current position is in the vicinity of the predetermined parking space 104. Here, the information on the position of the parking space 104 can be stored in the storage device 622 in advance.

When the start condition is not satisfied (no in S100), the vehicle appearance inspection system 100 returns to step S100 to repeat the process and waits for the start condition to be satisfied. That is, the return detection unit 630 repeatedly determines whether the vehicle 102 returns to the parking space 104.

On the other hand, when the start condition is satisfied (yes in S100), that is, when the return of the vehicle 102 to the parking space 104 is detected, the return detection unit 630 notifies the image acquisition unit 632 of this fact. This starts the process of appearance inspection from step S102.

Upon receiving a notification that the vehicle 102 has returned to the parking space 104, the image acquisition unit 632 first acquires an image of the periphery of the vehicle 102 (S102). Specifically, the image acquisition unit 632 instructs the image acquisition ECU 604 via the network bus to acquire images of the front, left, and right sides of the vehicle 102 using the front camera 130, the left-side view camera 136, and the right-side view camera 138.

Next, the image acquisition unit 632 determines whether any one of the

markers

118, 120, 122, 124, 126, and 128 (hereinafter, also referred to as the marker 118, etc.) is detected in the acquired image (S104). This determination can be made by determining whether or not a specific pattern of the mark 118 or the like is detected in the acquired image, or whether or not a barcode included in the mark 118 or the like is detected.

Next, when the image acquisition unit 632 does not detect any flag 118 or the like (no in S104), it inquires of the state detection ECU 602 to determine whether or not the ignition switch 614 of the vehicle 102 is turned OFF (OFF) (S106). When ignition switch 614 is turned off (yes in S106), image acquisition unit 632 determines that the mirror necessary for capturing the appearance of vehicle 102 cannot be found, and ends the process. Accordingly, processing device 620 performs the processing of step S102 and subsequent steps except for step S106 in response to detection of marker 118 or the like, and determines the presence or absence of an abnormality in the appearance of vehicle 102 from the outside view image.

On the other hand, if the ignition switch 614 is not turned off (no in S106), the image acquisition unit 632 returns to step S102 to repeat the processing.

On the other hand, when any of the markers 118 and the like is detected in step S104 (yes in S104), the image acquisition unit 632 determines whether or not the detected marker 118 and the like is a marker for designating the partial image mode processing (S108). When there are a plurality of images in which any one of the markers 118 and the like is detected, the vehicle appearance inspection system 100 (more specifically, the image acquisition unit 632 and the abnormality detection unit 634) executes the processing of step S108 to step S128 in parallel for each image and each camera corresponding to the image.

Next, when the detected flag or the like is not a flag indicating the partial image mode processing (no in S108), that is, when the detected flag or the like is a flag indicating the whole image mode processing, the image acquisition unit 632 inquires of the state detection ECU 602 to determine whether or not the ignition switch 614 of the vehicle 102 is turned OFF (OFF) (S110). Then, if the ignition switch 614 is not turned off (no in S110), the image acquisition unit 632 returns to step S110 to wait for turning off. When ignition switch 614 is turned off (yes in S110), image acquisition unit 632 instructs image acquisition ECU 604 to acquire an external view image of vehicle 102 reflected on mirror 112 or the like using the camera corresponding to the image in which the marker is detected (S112).

Through the above-described processing, for example, when detecting the image of the

mark

118 or 120 of the parking pile 106 or the

mark

126 or 128 of the bulkhead 110 from the image acquired by the front camera 130 or the left-side angle-of-view camera 136, the image acquisition unit 632 recognizes that these marks are marks indicating the entire image mode processing (corresponding to step S108). Then, when the vehicle 102 is stopped and the ignition switch 614 is turned off (corresponding to step S110), the image obtaining unit 632 obtains the front exterior view image 200 or the left exterior view image 300 shown in fig. 2 and 3 by using the front camera 130 or the left viewing angle camera 136 (corresponding to step S112).

On the other hand, if the flag detected in step S108 is a flag indicating that the partial image pattern processing is specified (yes in S108), the image acquisition unit 632 instructs the image acquisition ECU 604 to start repeatedly acquiring the surrounding image of the vehicle 102 in the field of view of the camera using the camera corresponding to the image in which the flag is detected (S116). As described in connection with fig. 5, the repetition of the partial image can be performed, for example, as the vehicle 102 moves.

Next, the image acquisition unit 632 inquires of the state detection ECU 602 whether or not the ignition switch 614 of the vehicle 102 is turned OFF (OFF) (S118), and if not turned OFF (S118, no), returns to step S118 and waits for turning OFF.

On the other hand, when the ignition switch 614 is turned off in step S118 (yes in S118), the image acquisition unit 632 ends the repeated acquisition of the surrounding images (S120), and synthesizes a plurality of repeatedly acquired surrounding images to acquire an external view image (S122).

Through the above-described processing, when detecting an image of the mark 122 or 124 applied to the parking lever 108 from the image acquired by the right-side view camera 138, for example, the image acquisition unit 632 recognizes that the mark is a mark indicating the partial image mode processing (corresponding to step S108). Then, the image acquisition unit 632 repeatedly acquires the surrounding image (right side image 400) shown in fig. 4 at predetermined time intervals as the vehicle 102 moves, using the right side view camera 138 (corresponding to step S116). When the vehicle 102 is stopped and the ignition switch 614 is turned off (corresponding to step S118), the image obtaining unit 632 obtains the right outside view image 500 as the composite image shown in fig. 5 from the obtained right image 400 (corresponding to step S122).

The time interval between the repeated acquisition of the surrounding image can be set as appropriate in accordance with the speed of the vehicle 102 acquired from the vehicle speed sensor 612.

Next, the image obtaining unit 632 stores the external view image of the vehicle 102 obtained in step S112 and/or step S122 in the storage device 622 (S114).

Next, the abnormality detection unit 634 refers to the external view image stored in step S114, extracts an image of the vehicle 102 (vehicle image) from the external view image (S124), and stores the image in the storage device 622 (S126). When extracting the vehicle image, the abnormality detection unit 634 finally determines whether or not the mark is included in the external view image, and can extract the vehicle image by performing image processing on the external view image for a section defined by 2 marks in accordance with the detection of the 2 marks, for example. This makes it possible to limit the image range to be processed, reduce the processing load, and prevent deterioration in the accuracy of vehicle image extraction due to the unnecessary image portion being processed.

In addition, the abnormality detection unit 634 can extract the vehicle image in which the distortion is corrected in the appearance image, for example, by referring to the typical image of the vehicle 102 based on the information on the vehicle type of the vehicle 102 when extracting the vehicle image. The correspondence between the typical images and/or vehicle types and the typical images can be stored in the storage device 622 in advance.

Next, the abnormality detection unit 634 compares each vehicle image stored this time (hereinafter, this vehicle image) with each corresponding vehicle image (hereinafter, the previous vehicle image) previously extracted from the captured images and stored in the storage device 622 (S128). Then, the abnormality detection unit 634 extracts a portion different from the corresponding previous vehicle image for each current vehicle image, and determines whether or not there is an abnormality in the vehicle appearance in any one of the current vehicle images based on the different portion (S130).

Then, when there is an abnormality in the vehicle appearance in any one of the vehicle images of this time (yes in S130), the abnormality detection unit 634 generates information about the abnormality for each of the vehicle images in which the abnormality exists (S132). After that, the information output unit 636 transmits and outputs the generated information on the abnormality to the management server 144 via the TCU 606, for example (S134), and the process is terminated.

Further, after receiving the information on the abnormality, the management server 144 may transmit an instruction to display information to the user to the vehicle 102, for example. The information display can include notification of the presence of a flaw or the like to the next user of the vehicle 102 as a rental vehicle, and/or a recommendation of car washing and a proposal of a reward (a discount on the use fee of the vehicle or the like) at the time of car washing. In response to receiving the instruction to display the information, the appearance check ECU 140 can display the information, for example, to the navigation device 608 and output the information to the display device 616 when the next use of the vehicle 102 is started.

On the other hand, in step S130, when no abnormality in the vehicle appearance is found in any of the current vehicle images (no in S130), the abnormality detection unit 634 ends the process.

When determining whether or not there is an abnormality in the vehicle appearance in step S130, the abnormality detection unit 634 can determine, for example, a different portion having a size larger than a predetermined size, of the different portions between the current vehicle image and the previous vehicle image, as an abnormality in the appearance. The abnormality detection unit 634 may regard, of the different portions, a portion whose shape and size are the same within a predetermined error range as the previous vehicle image and the current vehicle image and whose position in the appearance of the vehicle 102 is different only, as being due to dirt or the like adhering to the mirror 112 or the like, and may not determine that the vehicle 102 is abnormal in the appearance.

Next, the configuration of the management server 144 will be described. Fig. 7 is a diagram showing an example of the configuration of the management server 144. Management server 144 in the present embodiment includes processing device 700, storage device 702, and communication device 704.

The storage device 702 is configured by, for example, a volatile and/or nonvolatile semiconductor memory, a hard disk device, or the like. The storage device 702 stores data necessary for the processing of the processing device 700 in advance or stores data necessary for the processing. The communication device 704 is a transceiver for performing wired communication and/or wireless communication with the vehicle 102 or a portable terminal (not shown) held by a user of the vehicle 102 via the communication network 142.

The processing device 700 is a computer provided with a processor such as a CPU, for example. The processing device 700 may have a configuration including a ROM into which a program is written, a RAM for temporarily storing data, and the like. The processing device 700 includes a use end detection unit 710 and a vehicle operation instruction unit 712 as functional elements or functional means.

These functional elements included in the processing device 700 are realized by executing a program by the processing device 700 as a computer, for example. The computer program can be stored in advance in any computer-readable storage medium. Alternatively, all or a part of the functional elements included in the processing device 700 may be configured by hardware including one or more electronic circuit components.

When the user of the vehicle 102 as a rental vehicle finishes the use of the vehicle 102, the use end detection unit 710 receives a use end notification transmitted from, for example, a portable terminal (not shown) of the user, and detects the end of the use of the vehicle 102.

For example, when the use end detection unit 710 detects the end of use of the vehicle 102, the vehicle operation instruction unit 712 transmits a door lock signal to the vehicle 102 to lock the door of the vehicle 102. The door locking operation when the door locking signal is received in the vehicle 102 can be performed by an appropriate electronic control device provided in the vehicle 102 according to the conventional technique. Further, according to the related art, processing device 700 can perform management of rental reservation of the rental car, transmission of a door unlock instruction to vehicle 102 at the start of rental, and transmission of an instruction such as permission of engine start.

In the vehicle appearance inspection system 100, the

mirrors

112, 114, and 116 are disposed on the parking pile 106, the parking rod 108, and the spacer 110, which are fixed or movable structures disposed in the parking space 104. Then, an image of the vehicle 102 reflected on the mirror 112 or the like is acquired by the front camera 130, the left-side view camera 136, and the right-side view camera 138 provided for safety check of the vehicle 102, and an appearance abnormality such as a flaw or dirt of the vehicle 102 is detected based on the acquired image.

Therefore, the vehicle appearance inspection system 100 can detect an abnormality in appearance such as a flaw or dirt on the exterior of the vehicle 102 quickly and at low cost without making a leak.

The present invention is not limited to the configurations of the above-described embodiments, and can be implemented in various embodiments without departing from the scope of the present invention.

For example, in the above-described embodiment, as the structures arranged at the parking position of the vehicle, the parking pile 106 is arranged in front of the vehicle 102 when the vehicle is parked in the parking space 104, the parking rod 108 is arranged on the right side, the partition 110 is arranged on the left side, and the

mirrors

112, 114, and 116 are provided to these structures, respectively, but the present invention is not limited to this.

For example, any one of the parking post 106, the parking lever 108, and the spacer 110 may be disposed to detect an abnormality in the appearance of any one of the front, left, and right sides of the vehicle 102. The arrangement of these structures can be arranged at any position as long as at least a part of the appearance of the vehicle 102 reflected by the reflecting mirror provided in the structure can be imaged by the onboard camera such as the front camera 130 when the vehicle is parked in the parking space 104.

Further, a rear camera for photographing the rear of the vehicle 102 may be provided in the rear of the vehicle 102, and a structure provided with a mirror may be further disposed at a position behind the vehicle 102 parked in the parking space 104. This also enables detection of an appearance abnormality of the rear surface of the vehicle 102.

As a structure that can include a mirror for reflecting an external view image of a vehicle, in addition to the parking pile 106, parking rod 108, and partition 110 shown in the above embodiment, a movable parking cone having a triangular cone shape, a parking obstacle that stands up from the ground in front of the vehicle after the vehicle is parked at a parking position, or a parking gate that is provided in a path leading to the parking position and includes a lifting/lowering crossbar may be used. In these cases, the reflecting mirror may be provided on the parking cone itself or the parking obstacle itself, and may be disposed on the crossbar when the parking shutter is employed.

In the above embodiment, the one processing device 620 provided in the appearance inspection ECU 140 mounted on the vehicle 102 acquires the external view image of the vehicle 102 reflected on the mirror 112 or the like, and extracts the vehicle image from the external view image to detect the appearance abnormality, but the present invention is not limited thereto. These processes can be executed in a distributed manner by a plurality of processing devices provided in the vehicle appearance inspection system 100. In this sense, the vehicle appearance inspection system 100 includes at least one processing device that can take an image of an external view of the vehicle reflected on the mirror by the image pickup device, determine whether or not there is an appearance abnormality of the vehicle based on the external view image, and generate information on the detected abnormality.

For example, the abnormality detector 634 may be provided in the processing device 700 of the management server 144 instead of the processing device 620 provided in the appearance inspection ECU 140 of the vehicle 102. In this case, for example, the processing device 620 of the vehicle 102 functions as the 1 st treatment device, and can transmit the external view image of the vehicle 102 acquired by the image acquisition unit 632 to the management server 144 via the TCU 606. The processing device 700 of the management server 144 functions as the 2 nd processing device, and is capable of receiving the external view image and storing it in the storage device 702, and extracting the vehicle image from the received external view image by the abnormality detection unit 634 to detect an appearance abnormality.

In the above-described embodiment, the condition that the vehicle 102 returns to the predetermined parking space 104 (that is, reaches the vicinity thereof) is used as the start condition for starting the process of the appearance inspection, and the return detection unit 630 of the appearance inspection ECU 140 provided in the vehicle 102 determines whether or not the start condition is satisfied based on the current position of the vehicle 102, but the present invention is not limited thereto.

For example, appearance check ECU 140 may acquire the current position of vehicle 102 from GPS receiver 610 at predetermined time intervals and transmit the current position to management server 144, and processing device 700 of management server 144 may determine whether or not the start condition is satisfied based on the received current position and map information. In this case, the processing device 700 of the management server 144 that determines that the start condition is satisfied can transmit an instruction to start the appearance inspection process to the vehicle 102 via the communication device 704 by the vehicle operation instructing unit 712. In response to receiving the start instruction via the TCU 606, the appearance check ECU 140 of the vehicle 102 can execute the appearance check processing after step S102 shown in fig. 8.

In the above-described embodiment, as the start condition for starting the process of the appearance inspection, the condition that the vehicle 102 returns to the predetermined parking space 104 (that is, reaches the vicinity thereof) is used, but the present invention is not limited thereto. For example, the start condition may be a condition in which the management server 144 receives a use end notification that the user of the vehicle 102 as a rental car has parked the vehicle 102 in the parking space 104 as a predetermined rental parking position and then has transmitted the user's portable terminal to the management server 144.

In this case, when the use end detection unit 710 of the processing device 700 detects the end of use of the vehicle 102 based on the received use end notification, the management server 144 can determine that the start condition is satisfied, and can transmit an instruction to start the appearance check process to the vehicle 102 via the communication device 704 by the vehicle operation instruction unit 712. In response to receiving the start instruction via the TCU 606, the appearance check ECU 140 of the vehicle 102 can perform the processes of steps S112, S114, and S124 and thereafter shown in fig. 8. In this case, since the external view image is acquired after the vehicle 102 is parked, only the processing in the entire image mode can be performed.

As described above, the vehicle appearance inspection system 100 of the above embodiment includes: a front camera 130 and the like which are provided in the vehicle 102 and which are imaging devices that image the outside of the vehicle 102; a mirror 112 and the like provided on a parking pile 106, the parking pile 106 being a structure disposed in a parking space 104 which is a parking position of the vehicle 102; and at least one processing device 620. The processing device 620 is configured to capture an external view image of the vehicle 102 reflected on the mirror 112 or the like by the front camera 130 or the like when the vehicle 102 is parked in the parking space 104 by the image acquisition unit 632. The processing device 620 is configured to determine whether there is an abnormality in the appearance of the vehicle 102 from the external view image by the abnormality detection unit 634, and to generate information about the detected abnormality when the abnormality in the appearance is detected.

With this configuration, it is possible to detect an abnormality in appearance such as a flaw or dirt on the exterior of the vehicle 102 quickly and at low cost without making a leak.

In the vehicle appearance inspection system 100, a mark 118 or the like indicating the position of the reflecting mirror 112 or the like is provided on the structure such as the parking pin 106. With this configuration, an abnormality in appearance can be detected only for the portion indicated by the mark 118 or the like in the external view image, and thus the processing load can be reduced.

In the vehicle appearance inspection system 100, the processing device 620 is configured to acquire an external view image by the image acquisition unit 632 in response to detection of the mark 118 or the like, and to determine whether or not there is an abnormality in the appearance of the vehicle 102 from the external view image by the abnormality detection unit 634.

According to this configuration, by checking the presence of the mark 118 or the like, it is possible to reliably acquire an external view image including an image in which the mirror 112 or the like of the vehicle 102 is reflected, and it is possible to detect an appearance abnormality of the vehicle 102, and therefore, it is possible to prevent occurrence of unnecessary image processing.

In the vehicle appearance inspection system 100, the above-described structure such as the parking peg 106 is a fixed or movable parking position display disposed in the parking space 104 of the vehicle 102. With this configuration, since a structure that can be normally disposed in the parking space 104 can be used without disposing a special dedicated structure, the vehicle appearance inspection system 100 can be configured simply and inexpensively.

In the vehicle appearance inspection system 100, the structure may be a parking pile, a parking cone, a parking obstacle, or a parking gate disposed in a travel path up to or in a parking space. With this configuration, the vehicle appearance inspection system 100 can be easily and inexpensively configured using various structures that can be generally disposed in the parking space 104.

In the vehicle appearance inspection system 100, for example, the parking rod 108 and the partition 110, which are the above-described structures, are disposed on the outside in the vehicle width direction of the vehicle 102 placed in the parking space 104. According to this configuration, even with a structure that can be normally disposed in the parking space 104, an abnormality in appearance can be detected with respect to the side surface of the vehicle 102.

In the vehicle appearance inspection system 100, the processing device 620 is configured to repeatedly capture a right side image 400, which is a surrounding image including a partial image showing a part of the vehicle 102 reflected on the mirror 114, at predetermined time intervals by using the right side view camera 138, for example, and to construct the right side outside view image 500 of the vehicle 102 from the repeatedly captured right side image 400.

According to this configuration, even if the single mirror is not a mirror that can cover the entire one surface of the vehicle 102 at the same time, the appearance abnormality can be detected for the entire one surface of the vehicle 102 by using a mirror having a smaller size that can cover at least a part thereof. Therefore, the restriction on the arrangement space of the mirror can be reduced, and the vehicle appearance inspection system 100 can be configured simply and inexpensively.

In vehicle appearance inspection system 100, processing device 620 is a processing device mounted on vehicle 102, and is configured to transmit information on an abnormality detected with respect to the appearance of vehicle 102 to management server 144, which is a server outside the vehicle. With this configuration, even if there is a communication abnormality with a server or the like outside the vehicle, the vehicle 102 can detect an appearance abnormality alone.

In vehicle appearance inspection system 100, the processing device can be configured by processing device 620 as the 1 st processing device mounted on the vehicle and processing device 700 as the 2 nd processing device provided in the management server disposed outside vehicle 102. The processing device 620 as the 1 st processing device includes the image acquisition unit 632, and may be configured to capture an external view image of the vehicle 102 reflected on the mirror 112 or the like by an imaging device such as the front camera 130 when the vehicle 102 is parked in the parking space 104, and to transmit the external view image to the management server 144 via the TCU 606 as a communication device included in the vehicle 102. The processing device 700 as the 2 nd processing device may be configured to include the abnormality detection unit 634, determine whether there is an abnormality in the appearance of the vehicle 102 from the external view image received by the management server 144 from the processing device 620 as the 1 st processing device, and generate information on the detected abnormality when the abnormality in the appearance of the vehicle 102 is detected.

According to this configuration, since the processing for detecting the appearance abnormality of vehicle 102 can be performed in a distributed manner among 2 processing devices, processing with a large load of image processing is performed by processing device 700 of management server 144, for example, so that the processing load of processing device 620 mounted on vehicle 102 can be reduced, and processing device 620 can be configured at a low cost.

In the vehicle appearance inspection system 100, the vehicle 102 is a rental vehicle rented to a user, and the structure in which the mirror is disposed is a fixed or movable parking position display disposed in a parking space for rental of the vehicle 102. According to this configuration, in a rental car that can be used by a plurality of users, the occurrence of an abnormality in appearance can be detected quickly and at low cost without overlooking. Therefore, the generated abnormality can be promptly dealt with, and a decrease in the commodity value of the vehicle 102 as a rental car can be promptly prevented.

In addition, the vehicle appearance inspection method is performed in the vehicle appearance inspection system 100. The inspection method is comprised of steps performed by at least one processing device (e.g., processing device 620). This step includes a step (S112 and/or S122) of capturing an external view image (for example, a front external view image 200) of the vehicle 102 reflected on a mirror 112 or the like provided on the parking pile 106 or the like as a structure arranged in the parking space 104 by using a front camera 130 or the like as an imaging device mounted on the vehicle 102 when the vehicle 102 is parked in the parking space 104. Further, the steps executed by the processing device include: the method includes a step (S130) of determining whether or not there is an abnormality in the appearance of the vehicle 102 from the external view image, and a step (S132) of generating information on the detected abnormality when an abnormality in the appearance of the vehicle 102 is detected.

Claims

1. A vehicle appearance inspection system includes:

an imaging device that is provided in a vehicle and that images the outside of the vehicle;

a mirror provided on a structure disposed at a parking position of the vehicle; and

at least one processing device is arranged on the device,

the processing device is configured to:

capturing an external view image of the vehicle reflected on the reflecting mirror by the image capturing device when the vehicle is parked at the parking position,

judging whether the appearance of the vehicle is abnormal or not according to the external scene image,

upon detection of an abnormality in the appearance of the vehicle, information about the detected abnormality is generated.

2. The vehicle appearance inspection system according to claim 1,

the structure is provided with a mark indicating a position of the mirror.

3. The vehicle appearance inspection system according to claim 2,

the processing device is configured to determine whether or not there is an abnormality in the appearance of the vehicle based on the external view image in accordance with the detection of the mark.

4. The vehicle appearance inspection system according to any one of claims 1 to 3,

the structure is a fixed or movable parking position display disposed in a parking space of the vehicle.

5. The vehicle appearance inspection system according to claim 4,

the structure is a parking pile, a parking cone, a parking obstacle or a parking gate arranged in a driving path up to a parking space or in the parking space.

6. The vehicle appearance inspection system according to any one of claims 1 to 5,

the structure is disposed outside in a vehicle width direction of the vehicle placed at the parking position.

7. The vehicle appearance inspection system according to any one of claims 1 to 6,

the processing device is configured to:

repeatedly capturing, by the imaging device, a surrounding image at a predetermined time interval, the surrounding image including a partial image showing a part of the vehicle reflected on the mirror,

the external view image of the vehicle is constructed from the repeatedly captured surrounding images.

8. The vehicle appearance inspection system according to any one of claims 1 to 7,

the processing device is constituted by one processing device mounted on the vehicle,

the processing device is configured to transmit the information about the detected abnormality to a server outside the vehicle.

9. The vehicle appearance inspection system according to any one of claims 1 to 7,

the processing device is composed of a 1 st processing device mounted on the vehicle and a 2 nd processing device provided in a server disposed outside the vehicle,

the 1 st processing device is configured to capture an external view image of the vehicle reflected on the mirror by the imaging device when the vehicle is parked at the parking position, and to transmit the external view image to the server through a communication device provided in the vehicle,

the 2 nd processing device is configured to determine whether or not there is an abnormality in the appearance of the vehicle based on the external view image received by the server from the 1 st processing device, and when an abnormality in the appearance of the vehicle is detected, generate information about the detected abnormality.

10. The vehicle appearance inspection system according to any one of claims 1 to 9,

the vehicle is a rental vehicle rented to the user,

the structure is a stationary or movable parking position display disposed in a parking space for rental of the vehicle.

11. A vehicle, wherein,

the vehicle constitutes the vehicle appearance inspection system according to claim 1, and the vehicle is provided with the imaging device and the processing device.

12. A parking position display in which, when the parking position is displayed,

the parking position display is movable, represents a parking space of the vehicle, and includes a mirror and a mark indicating a position of the mirror.

13. A vehicle appearance inspection method, performed by at least one processing device,

the vehicle appearance inspection method comprises the following steps:

when a vehicle is parked at a parking position, capturing an external view image of the vehicle reflected on a mirror provided on a structure disposed at the parking position by an imaging device mounted on the vehicle;

judging whether the appearance of the vehicle is abnormal or not according to the external scene image; and