JP6926917B2 - Offline verification system - Google Patents

Description

The present disclosure relates to an offline verification system.

Conventionally, there is an object recognition device that detects an object by a sensor and determines whether or not the detected object is dangerous (see, for example, Patent Document 1). The object recognition device of Patent Document 1 is mounted on a vehicle, irradiates a laser to detect an object in front of the vehicle, and detects an object that may collide. In this object recognition device, an identification mark is attached to an erroneously detected object.

Japanese Unexamined Patent Publication No. 2010-181246

The technique described in Patent Document 1 described above is used while a vehicle equipped with an object recognition device is running. Patent Document 1 does not disclose that the information acquired during traveling is used offline. The present disclosure provides an offline verification system capable of offlinely verifying the performance of a system using information acquired in advance, and reflecting the verification result to suppress false detection of an object. The purpose is.

The offline verification system of the present disclosure includes a route information storage unit that stores linear information about a route on which a moving object travels, and target image information that has been acquired in advance, and includes image information of an object candidate and image information of the route. An image information storage unit that stores image information, a suppression display image generation unit that suppresses the display of objects that do not need to be identified among object candidates, and a route display that generates a route display image that simulates a route. It includes an image generation unit, a display control unit that superimposes and displays a suppression display image and a route display image on an image based on the target image information, and a suppression display editing unit that edits the suppression display image.

This offline verification system includes an image information storage unit, stores the target image information acquired in advance in the image information storage unit, and can use the saved target image information later. The offline verification system includes a suppression display image generation unit, and can suppress the display of an object that does not require identification among the object candidates included in the object image information. As a result, the object that does not need to be identified can be made inconspicuous, and the object to be identified can be made relatively conspicuous. Since the offline verification system is provided with a route display image generation unit and can generate a route display image simulating the route, the position, shape, and the like of the route can be clearly shown. Since the offline verification system is provided with a display control unit and can superimpose and display the suppression display image and the route display image, the user can visually confirm whether or not the suppression display image and the route display image are correctly displayed. .. In addition, the offline verification system is equipped with a suppression display editing unit and can edit the suppression display image, so the position and size of the suppression display image can be changed, and unnecessary suppression display images can be deleted. , A suppressed display image can be added as appropriate.

The offline verification system may be configured to further include a route display editing unit that edits the route display image. This offline verification system is equipped with a route display editing unit and can edit the route display image, so the position and size of the route display image can be changed, unnecessary route display images can be deleted, and so on. A route display image can be added as appropriate.

The offline verification system of the present disclosure includes a route information storage unit that stores linear information about a route on which a moving object moves, target image information acquired in advance, and includes image information of an object candidate and image information of the route. An image information storage unit that stores target image information, a suppressed display image generation unit that suppresses the display of objects that do not need to be identified among target candidates, and a route that generates a route display image that simulates a route. It includes a display image generation unit, a display control unit that superimposes and displays a suppression display image and a route display image on an image based on the target image information, and a suppression display editing unit that edits the route display image.

This offline verification system includes an image information storage unit, stores the target image information acquired in advance in the image information storage unit, and can use the saved target image information later. The offline verification system includes a suppression display image generation unit, and can suppress the display of an object that does not require identification among the object candidates included in the object image information. As a result, the object that does not need to be identified can be made inconspicuous, and the object to be identified can be made relatively conspicuous. Since the offline verification system is provided with a route display image generation unit and can generate a route display image simulating the route, the position, shape, and the like of the route can be clearly shown. Since the offline verification system is provided with a display control unit and can superimpose and display the suppression display image and the route display image, the user can visually confirm whether or not the suppression display image and the route display image are correctly displayed. .. In addition, the offline verification system is equipped with a route display editing unit and can edit the route display image, so the position and size of the route display image can be changed, and unnecessary route display images can be deleted. , A route display image can be added as appropriate.

According to the present disclosure, it is possible to verify the performance of the system offline by using the information acquired in advance, and it is possible to suppress the false detection of the object by reflecting the verification result.

It is a perspective view which shows the vehicle equipped with the obstacle detection system, and the road on which the vehicle travels. It is a block block diagram which shows the offline verification system. FIG. 3A is a diagram showing an image acquired in advance. FIG. 3B is a diagram showing an object included in the detection result information acquired in advance. FIG. 3C is a diagram showing a mask display image. FIG. 3D is a diagram showing a route display image. It is a figure which shows the image which superposed and displayed the mask display image and the route display image on the image acquired in advance. It is a flowchart which shows the processing procedure executed in an offline verification system. It is a flowchart which shows the processing procedure of the data edit executed in the offline verification system. It is a figure which shows the mask display edit screen.

Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the drawings. In each figure, the same parts or corresponding parts are designated by the same reference numerals, and duplicate description will be omitted.

As shown in FIG. 2, the offline verification system 1 shown in FIG. 1 uses, for example, image information acquired in advance by a vehicle (moving body) M equipped with a camera or the like while traveling on a route (road C) to provide information. It is a system that performs processing. In the offline verification system 1, for example, the past image information (moving image) stored in the image information storage unit 12 is reproduced, and other information (mask display, route display, etc., which will be described later) associated with the image information is edited. Includes the ability to do.

The data processed by the offline verification system 1 can be used by the obstacle detection system 2 mounted on the vehicle M. Further, the obstacle detection system 2 has a function of detecting an object candidate (for example, a falling object B) in front of the vehicle. The obstacle detection system 2 has a function of determining whether or not the target object candidate is an obstacle. The obstacle detection system 2 includes, for example, a camera (not shown), a wheel speed sensor, an obstacle detection unit, and the like. The obstacle detection system 2 tracks the detected object candidates, and determines the tracked object candidates that satisfy the determination conditions as obstacles. The "obstacle" is an object that may interfere with the running of the vehicle M, and it is necessary to change the running or stop and remove the object so as to avoid the object. be.

The offline verification system 1 includes an offline verification unit 3, a display unit 4, and an operation unit 5. The offline verification unit 3 is a computer composed of hardware such as a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory), and software such as a program stored in the ROM. .. The offline verification unit 3 includes an input signal circuit, an output signal circuit, a power supply circuit, and the like. The offline verification unit 3 includes an image information acquisition unit 6, a detection result information acquisition unit 7, an image processing unit 8, a linear calculation unit 9, a coordinate conversion unit 10, and an obstacle determination unit 11. Further, the offline verification unit 3 includes an image information storage unit 12, a detection result information storage unit 13, a route information storage unit 14, and a camera parameter storage unit 15.

The image information acquisition unit 6 is a receiving unit that acquires image information acquired in advance by the obstacle detection system 2. The image information is image information (target image information) captured by the camera of the obstacle detection system 2, and is generated by photographing the front of the vehicle while the vehicle M is traveling. The image information includes the image information of the road C and the image information of the object candidate. The "object candidate" includes an object that may be detected as an object, an object that is determined to be an obstacle, an object that is not determined to be an obstacle, and an object that is later excluded from the object candidates. .. The image information received by the image information acquisition unit 6 is stored in the image information storage unit 12. The image information acquisition unit 6 receives the traveling information of the vehicle M as the information associated with the image information. The traveling information of the vehicle M is information on the traveling of the vehicle M when the image information is acquired in advance, and includes, for example, data on the vehicle speed of the vehicle M, data on the position of the vehicle M, and the like. The acquired traveling information is stored in the image information storage unit 12.

The detection result information acquisition unit 7 is a reception unit that acquires the detection result information acquired in advance by the obstacle detection system 2. The detection result information is information on the obstacle detected by the obstacle detection system 2, and includes information on the position of the obstacle, information on the shape of the obstacle, and the like. The detection result information may include other information that can identify obstacles. Further, the information regarding the obstacle may include, for example, information regarding the actual position and shape, information regarding the position on the screen, and information regarding the number of pixels. The detection result information received by the detection result information acquisition unit 7 is stored in the detection result information storage unit 13. The offline verification unit 3 can acquire other information such as camera data in addition to the image information, the traveling information, and the detection result information.

The image processing unit 8 performs image processing on the image information received by the image information acquisition unit 6. The image processing unit 8 performs, for example, binarization processing as image processing, and detects information on an object and information on a white line on a road.

The linear calculation unit 9 obtains the own vehicle position information indicating the position from the starting point of the vehicle M based on the traveling information stored in the storage unit and the linear information of the route stored in the route information storage unit 14. calculate. The route information storage unit 14 is a storage unit that stores linear information of the route. "Linear information of the route" is information on the three-dimensional shape of the route corresponding to the distance from the starting point of the route, information on the horizontal curvature of the curve on the road, and information on the vertical gradient on the slope. Etc. are included.

The coordinate conversion unit 10 converts the path coordinates (three-dimensional coordinates) and the image coordinates (two-dimensional coordinates). The coordinate conversion unit 10 can convert, for example, path coordinates and image coordinates based on the image information of an object whose position and size are known among the objects in the image. For example, a conversion formula between the route coordinates and the image coordinates can be set using known values such as the length and width of the white lines of the road and the distance between the white lines.

The coordinate conversion unit 10 mutually converts the route coordinates and the image coordinates by using the linear information of the route stored in the route information storage unit 14 and the information regarding the camera parameters stored in the camera parameter storage unit 15. .. The camera parameter storage unit 15 is a storage unit that stores information related to camera parameters. The "information about the camera parameters" includes, for example, the focal length of the camera when the image information is acquired by the vehicle M, the element size of the image sensor of the camera, the installation position of the camera, the installation angle of the camera, and the like. The coordinate conversion unit 10 can grasp the actual position of the object candidate (for example, the falling object B) on the image by converting the image coordinates into the path coordinates. This makes it possible to calculate the distance from the reference point to the object candidate. The reference point is, for example, an arbitrary position of the vehicle M, and can be the position of the lens of the camera.

The obstacle determination unit 11 is a determination unit that determines whether or not the object candidate is an obstacle with respect to the image information of the object candidate. For example, after detecting the object candidate, the object that is grasped as the object candidate even after the vehicle M has traveled for a predetermined time can be determined as an obstacle. The obstacle determination unit 11 can determine that an object candidate satisfying other determination conditions is an obstacle. The obstacle determination unit 11 may determine whether or not the obstacle is an obstacle, for example, according to the position in the road width direction and the position in the height direction with respect to the road of the object candidate.

Further, the display unit 4 and the operation unit 5 are electrically connected to the offline verification unit 3. The display unit 4 is, for example, a liquid crystal display device, and displays an image based on the image information stored in the image information storage unit 12. The operation unit 5 includes, for example, a keyboard, a mouse, a touch panel, and the like. The user can operate the operation unit 5 to edit various information stored in the storage unit of the offline verification unit 3.

Here, the offline verification unit 3 includes a mask display image generation unit (suppression display image generation unit) 21, a route display image generation unit (path display image generation unit) 22, a display control unit 23, and a mask display editing unit (suppression display editing). Unit) 24, a route display editing unit 25, a route display information storage unit 26, and a mask display information storage unit 27.

The mask display image generation unit 21 generates a mask display image that suppresses and displays an object that is not an obstacle. An object that is not an obstacle is an object that is determined not to be an obstacle among the detected object candidates and is an object that does not need to be identified as an obstacle (object that does not need to be identified). Examples of objects that are determined not to be obstacles include road signs, traffic lights, and the like. The mask display includes a display that covers the object so that it cannot be seen by the user, a display of a frame surrounding the object, and the like. Further, the mask display may be a display that reduces the visibility of the object by covering it with a semi-transparent image. FIG. 3C shows an image 33 including a mask display image E. The mask display image E is, for example, a frame body surrounding a sign, indicated by a broken line, and has a rectangular shape. The mask display image E may be shown by, for example, a solid line or a long-dotted chain line. The information regarding the mask display image E includes, for example, the position of the object suppressed and displayed by the mask display (distance from the starting point of the route), the display position on the mask display screen (x, y), and the mask display. Width, height of mask display (length in the vertical direction), etc. are included. Information about these mask display images E is stored in the mask display information storage unit 27.

The route display image generation unit 22 generates route display images F1 and F2 that simulate the route. As shown in FIG. 3D, the route display image is, for example, along the white line G1 at the center of the road C in the road width direction (route display image F1) or along the white line G2 outside the road width direction of the road C. Includes lines (route display image F2) and the like. The route display images F1 and F2 simulating the route can indicate the direction in which the road C extends, the road width of the road C, and the like included in the captured image information (the target image 31 shown in FIG. 3A). .. Further, the information regarding the route display images F1 and F2 includes the display positions (x, y) of the route display images F1 and F2 on the screen. Information about the route display images F1 and F2 is stored in the route display information storage unit 26.

The display control unit 23 superimposes and displays the mask display image E and the route display images F1 and F2 when reproducing an image based on the image information (target image information) stored in the image information storage unit 12. 3A to 3D show an image 31 based on the target image information, an image 32 based on the detection result, an image 33 including the mask display image E, and an image including the route display images F1 and F2. There is. As shown in FIG. 4, the display control unit 23 superimposes and displays the image 32 based on the detection result, the mask display image E, and the route display images F1 and F2 on the image 31 based on the target image information. As a result, on the screen of the display unit 4, the sign D is suppressed and displayed by the mask display image E. Further, on the screen of the display unit 4, the route display images F1 and F2 clearly indicate the position of the road C, the width of the road C, the direction in which the road C extends, and the like. As a result, the visibility is improved and the road C is easily recognized, so that the user can easily recognize the positional relationship between the object candidate and the road C.

The mask display editing unit 24 edits information related to the mask display image stored in the mask display information storage unit 27. The mask display editing unit 24 causes the display unit 4 to display the mask display image editing screen 41 as shown in FIG. 7, based on, for example, an operation input to the operation unit 5. The mask display editing unit 24 acquires the input information and edits the information related to the mask display image. In the mask display image editing screen 41, for example, editable items are displayed on the left side of the screen, and an area 42 for displaying the current data and an area 43 for inputting the edited data are provided on the right side thereof.

Further, the mask display image editing screen 41 is provided with an add button 44, a change button 45, a delete button 46, and a cancel button 47 on the right side of the screen. The user can operate the add button 44 to add information about the new mask display image input to the area 43. The user can operate the change button 45 to change the information displayed in the area 42 to the information input in the area 43. The user can operate the delete button 46 to delete the information displayed in the area 42. The user can operate the cancel button 47 to cancel the editing work using the mask display image editing screen 41.

Next, the processing procedure executed by the offline verification system 1 will be described with reference to the flowchart shown in FIG. The image information acquisition unit 6 of the offline verification unit 3 acquires the image information acquired in advance (step S1). The image information acquisition unit 6 reads image information stored in a storage medium such as a USB memory, for example. The read image information is stored in the image information storage unit 12. Further, the image information acquisition unit 6 may receive the image information transmitted using the communication line. The image information transmitted using the communication line is, for example, image information transmitted from the obstacle detection system 2 existing at a place away from the offline verification system 1. Alternatively, the image information acquisition unit 6 may read the image information already stored in the image information storage unit 12.

The detection result information acquisition unit 7 acquires the detection result information acquired in advance (step S2). The detection result information acquisition unit 7 reads the detection result information stored in a storage medium such as a USB memory. The read detection result information is stored in the detection result information storage unit 13. Similar to step S1 described above, the detection result information may be received from the obstacle detection system 2 using the communication line, or the detection result information already stored in the detection result information storage unit 13 may be read.

Next, the route display image generation unit 22 generates route display images F1 and F2 (step S3). The route display image generation unit 22 generates route display images F1 and F2 along the white lines G1 and G2 of the road C, for example, based on the information about the route included in the image information.

The mask display image generation unit 21 generates the mask display image E (step S4). The mask display image generation unit 21 generates the mask display image E based on the detection result information.

Next, the display control unit 23 superimposes and displays the route display images F1 and F2 and the mask display image E on the target image 31 (step S5). The superimposed and displayed image 35 is displayed on the display unit 4.

Next, the route display information storage unit 26 stores the route display images F1 and F2 generated in step S3 in the route display information storage unit 26 (step S6). Further, the mask display information storage unit 27 stores the mask display image E generated in step S4 in the mask display information storage unit 27 (step S7). This ends the processing here.

Next, the data editing processing procedure executed by the offline verification system 1 will be described with reference to the flowchart shown in FIG. The mask display editing unit 24 displays the mask display image editing screen 41 on the display unit 4 in response to the operation input of the user (step S11). The user can edit the data using the operation unit 5 and add, change, or delete the mask display image information. The mask display editing unit 24 acquires the edited mask display image information input through the mask display image editing screen 41 (step S12). The mask display information storage unit 27 stores the edited mask display image information acquired in step S12 (step S13).

The route display editing unit 25 displays the route display image editing screen on the display unit 4 in response to the operation input of the user (step S14). Similar to the mask display image editing screen 41, the route display image editing screen includes data editable items, information before editing, an area for inputting information after editing, an add button, a change button, a delete button, and a cancel button. Buttons etc. are displayed. The user can edit the data using the operation unit 5 to add, change, or delete the route display image information. The route display editing unit 25 acquires the edited route display image information input through the route display image editing screen (step S15). The route display information storage unit 26 stores the edited route display image information acquired in step S15. The edited information stored in the route display information storage unit 26 and the mask display information storage unit 27 can be stored in the storage medium as log output and used in the obstacle detection system 2. In addition, the edited information is transmitted through the communication line and can be used in the obstacle detection system.

The offline verification system 1 includes an image information storage unit 12, stores the target image information acquired in advance in the image information storage unit 12, and can use the saved target image information later. The offline verification system 1 includes a mask display image generation unit 21, and can make an object that is not an obstacle inconspicuous by mask display and make the object to be identified relatively conspicuous. Further, since the offline verification system 1 is provided with the route display image generation unit 22 and can generate route display images F1 and F2 simulating the route, the position, shape, and the like of the road C can be clearly shown.

Since the offline verification system 1 is provided with a display control unit 23 and can superimpose and display the mask display image E and the route display images F1 and F2, is the mask display image E and the route display images F1 and F2 correctly displayed? Whether or not it can be visually confirmed by the user. Further, since the offline verification system 1 includes a mask display editing unit 24 and can edit the mask display image E, the position and size of the mask display image E to be displayed can be changed, and an unnecessary mask display image can be changed. E can be deleted, and a mask display image E can be added as appropriate. As a result, the performance of the obstacle detection system can be verified offline using the image information acquired in advance, and the verification result can be reflected to suppress erroneous detection of the object.

Since the offline verification system 1 includes a route display editing unit 25 and can edit the route display images F1 and F2, the position and size of the route display images F1 and F2 are changed or unnecessary. The route display images F1 and F2 can be deleted, and the route display images F1 and F2 can be added as appropriate.

For example, an obstacle detection system before shipment in a factory can be applied as an offline verification system 1. This makes it possible to confirm and verify the performance of the obstacle detection system before shipment by using the image information acquired in advance by another obstacle detection system that is already in use. Further, for example, an obstacle detection system shipped as a product and actually used in the field can be applied as an offline verification system. This makes it possible to confirm and verify the performance of the obstacle detection system that has been used for a certain period of time. By optimizing the mask display image and the route display image by using the offline verification system 1, it is possible to suppress erroneous detection of an object in the obstacle detection system and improve the detection accuracy of the obstacle.

The present disclosure is not limited to the above-described embodiment, and various modifications as described below are possible without departing from the gist of the present disclosure. In the above embodiment, the mask display editing unit 24 and the route display editing unit 25 are provided, but the offline verification system 1 may be configured to include the mask display editing unit 24 or the route display editing unit 25. Further, the object is not limited to a stationary object, but may be a moving object, a person, or an animal. Further, the obstacle is not limited to a dangerous object, but may be an obstacle object or an object that meets specific conditions. Further, the mask display image is not limited to a rectangular image, and may be a circular image or another shape. Further, the route display image may be a simulation of a white line on a road, or may be a simulation of another guardrail or the like extending along the road.

1 Offline verification system 3 Offline verification unit 12 Image information storage unit 14 Route information storage unit 21 Mask display image generation unit (suppressed display image generation unit)
22 Route display image generation unit 23 Display control unit 24 Mask display editing unit (suppressed display editing unit)
25 Route display editorial unit 31 Target image (image based on target image information)
B Falling objects (object candidates, objects, obstacles)
C road (route)
D sign (object candidate, object, object that does not require identification)
E Mask display image (suppressed display image)
F1, F2 route display image M vehicle (moving body)

Claims (3)

A route information storage unit that stores linear information about the route on which the moving object travels,
An image information storage unit that stores the target image information acquired in advance and stores the target image information including the image information of the target candidate and the image information of the route.
A suppression display image generation unit that generates a suppression display image that suppresses the display of an object that does not require identification among the object candidates.
A route display image generation unit that generates a route display image that simulates the route, and
A display control unit that superimposes and displays the suppressed display image and the route display image on the image based on the target image information.
An offline verification system including a suppression display editing unit that edits the suppression display image. The offline verification system according to claim 1, further comprising a route display editing unit for editing the route display image. A route information storage unit that stores linear information about the route on which the moving object travels,
An image information storage unit that stores the target image information acquired in advance and stores the target image information including the image information of the target candidate and the image information of the route.
A suppression display image generation unit that generates a suppression display image that suppresses the display of an object that does not require identification among the object candidates.
A route display image generation unit that generates a route display image that simulates the route, and
A display control unit that superimposes and displays the suppressed display image and the route display image on the image based on the target image information.
An offline verification system including a suppression display editing unit that edits the route display image.

Images