JP7411771B1 - Vehicle speed detection device, vehicle speed detection method and program - Google Patents

Abstract

[Problem] To accurately grasp the situation when a vehicle accident occurs. [Solution] A vehicle speed detection device includes an image acquisition unit that acquires an image for one frame from moving image data including a plurality of frames, and a position and size of a first moving object that is a moving object included in the acquired image. a bounding box detection unit that detects a bounding box for specifying the bounding box; a tracking target point detection unit that detects the position coordinates of a tracking target point, which is a point based on the position and size of the detected bounding box; Based on changes in the positional coordinates of the tracking target point detected in each image, the relative speed of the second moving body, which is a moving body in which an imaging device for capturing the moving image data is installed, and the first moving body is determined. and a relative velocity estimator for estimating. [Selection diagram] Figure 12

Description

The present invention relates to a vehicle speed detection device, a vehicle speed detection method, and a program.

BACKGROUND ART Conventionally, in order to accurately grasp the situation when a vehicle accident occurs, a camera such as a drive recorder is installed in a vehicle to capture moving images of the surrounding area in the direction in which the vehicle is traveling. When a vehicle accident occurs between one's own vehicle and another vehicle, the speed of the other vehicle from before the accident to the time of the accident becomes an important factor in accident analysis. Therefore, many techniques for estimating the speed of the other vehicle from moving images captured by drive recorders have been studied. As one method of estimating the speed of an opponent vehicle, a method of two-dimensionally analyzing a moving image captured by a drive recorder has been proposed (see, for example, Patent Document 1). According to such a technique, by using optical flow, it is possible to calculate the coordinates of a moving object in a moving image and estimate the speed of the other vehicle using the calculated coordinates.

JP2021-189531A

According to the technique described above, the grounding point between the tire and the road surface is specified for each frame, and the relative speed between the own vehicle and the other vehicle is estimated based on the transition of the specified grounding point for each frame. Therefore, if the grounding point specified for each frame deviates from the actual position, this will directly lead to a deviation in speed. As described above, the conventional technology has a problem in that speed changes for each frame cannot be accurately detected. If the speed of the other vehicle cannot be accurately calculated, it is difficult to accurately grasp the situation when a vehicle accident occurs.

Therefore, an object of the present invention is to provide a technology that can accurately grasp the situation when a vehicle accident occurs.

[1] In order to solve the above problems, one aspect of the present invention is an image acquisition unit that acquires an image for one frame from video data including a plurality of frames, and a moving object included in the acquired image. A bounding box detection unit that detects a bounding box that specifies the position and size of the first moving object, and a tracking target inspection that detects the position coordinates of a tracked point that is a point based on the position and size of the detected bounding box. a second moving body, which is a moving body equipped with an imaging device that captures the moving image data, based on a change in the position coordinates of the tracking target point detected in each of a plurality of frames of images; a relative speed estimating section that estimates a relative speed with respect to a first moving object, and the relative speed estimating section includes information indicating a relative position between the second moving object and the first moving object for each frame of images. This is a vehicle speed detection device that outputs a change in relative position and estimates the relative speed based on a change in the output relative position .

[2] Further, one aspect of the present invention is that in the vehicle speed detection device according to [1] above, the tracking target point detection unit detects a point based on one or more vertices of the bounding box as the tracking target point. It is detected as a point.

[3] Moreover, one aspect of the present invention is that in the vehicle speed detection device according to [2] above, the tracking target point detection unit detects the center of one vertex or two or more vertices of the bounding box. The point is detected as the tracking target point.

[4] Further, one aspect of the present invention is the vehicle speed detection device according to any one of [1] to [3] above, in which the bounding box is a three-dimensional bounding box having a depth direction.

[5] Moreover, one aspect of the present invention is the vehicle speed detection device according to [4] above, in which the tracking target point detection unit detects a point included in a bottom surface of the detected three-dimensional bounding box. This is detected as a tracking target point.

[6] Further, one aspect of the present invention provides the vehicle speed detection device according to any one of [1] to [5] above, including a speed information acquisition unit that acquires the traveling speed of the second moving body; The vehicle further includes an absolute speed estimating section that estimates the traveling speed of the first moving body based on the relative velocity determined by the vehicle and the traveling speed of the second moving body.

[7] Further, one aspect of the present invention is the vehicle speed detection device according to any one of [1] to [6] above, further comprising a distortion correction unit that corrects distortion of the image acquired by the image acquisition unit. Furthermore, the bounding box detection section detects the bounding box based on the image subjected to distortion correction by the distortion correction section.

[8] In addition, in the vehicle speed detection device according to [7] above, one aspect of the present invention is that the position coordinates of the detected bounding box are determined by the position coordinates of the detected bounding box in the coordinate system of the image acquired by the image acquisition unit. The apparatus further includes a detection result shaping section that converts the detection result into coordinates.

[9] Further, one aspect of the present invention is the vehicle speed detection device according to any one of [1] to [8] above, in which the bounding box detection unit has a predetermined input image and a frame corresponding to the input image. The bounding box is detected using a machine learning model learned based on teacher data that is associated with the position of the bounding box.

[10] Further, one aspect of the present invention is that in the vehicle speed detection device according to [9] above, the training data used for learning the machine learning model is a day/night conversion model based on images captured during the day. As a result of image processing using , images converted to images taken at night are included.

[11] Further, one aspect of the present invention provides an image acquisition step of acquiring an image for one frame from moving image data including a plurality of frames, and a position of a first moving object that is a moving object included in the acquired image. a bounding box detection step that detects a bounding box whose size is specified; a tracking point detection step that detects the position coordinates of a tracking point that is a point based on the position and size of the detected bounding box; Based on changes in the position coordinates of the tracking target point detected in each of the images, the relative relationship between the second moving body, which is a moving body in which an imaging device for capturing the moving image data is installed, and the first moving body is determined. a relative speed estimating step of estimating a speed , and the relative speed estimating step outputs information indicating the relative position of the second moving object and the first moving object for each frame of images; This vehicle speed detection method estimates the relative speed based on a change in relative position .

[12] Further, one aspect of the present invention includes an image acquisition step of acquiring an image for one frame from moving image data including a plurality of frames, and a first moving object that is a moving object included in the acquired image. a bounding box detection step for detecting a bounding box that specifies the position and size of the body; a tracking target point detection step for detecting the position coordinates of a tracking target point, which is a point based on the position and size of the detected bounding box; , a second mobile body that is a mobile body equipped with an imaging device that captures the moving image data based on changes in the position coordinates of the tracking target point detected in each of a plurality of frames of images; and the first mobile body. and a relative speed estimating step of estimating a relative speed between the second moving object and the first moving object, and the relative speed estimating step outputs information indicating the relative position of the second moving object and the first moving object for each frame of images. The program estimates the relative velocity based on the output change in relative position .

According to the present invention, it is possible to accurately grasp the situation when a vehicle accident occurs.

FIG. 1 is a block diagram for explaining an overview of a system according to an embodiment. FIG. 2 is a sequence diagram for explaining an overview of processing performed by the system according to the embodiment. It is a flowchart for explaining details of vehicle recognition processing performed by a system concerning an embodiment. FIG. 2 is a first diagram for explaining distortion correction processing performed by the system according to the embodiment. FIG. 2 is a second diagram for explaining distortion correction processing performed by the system according to the embodiment. FIG. 3 is a diagram for explaining bounding box detection processing performed by the system according to the embodiment. FIG. 2 is a first diagram for explaining detection result shaping processing performed by the system according to the embodiment. FIG. 7 is a second diagram for explaining detection result shaping processing performed by the system according to the embodiment. It is a flowchart for explaining the details of speed estimation processing performed by the system concerning an embodiment. FIG. 3 is a diagram for explaining an example of a three-dimensional bounding box and a tracking target point according to an embodiment. FIG. 3 is a diagram for explaining an example of tracking processing performed by the system according to the embodiment. FIG. 1 is a block diagram showing an example of a functional configuration of a vehicle speed detection device according to an embodiment. 2 is a flowchart illustrating an example of processing of a user interface included in the system according to the embodiment. FIG. 2 is a first diagram illustrating an example of a display screen displayed by a user interface included in the system according to the embodiment. FIG. 7 is a second diagram showing an example of a display screen displayed by a user interface included in the system according to the embodiment. FIG. 2 is a first diagram showing an example of a form output by the system according to the embodiment.

[Embodiment]
DESCRIPTION OF THE PREFERRED EMBODIMENTS A vehicle speed detection device, a vehicle speed detection method, and a program according to aspects of the present invention will be described in detail below with reference to preferred embodiments and the accompanying drawings. Note that aspects of the present invention are not limited to these embodiments, but also include those with various changes or improvements. That is, the components described below include those that can be easily assumed by those skilled in the art and are substantially the same, and the components described below can be combined as appropriate. Moreover, various omissions, substitutions, or changes of the constituent elements can be made without departing from the gist of the present invention. Further, in the following drawings, in order to make each structure easier to understand, the scale, number, etc. of each structure may be different from the scale, number, etc. of the actual structure.

FIG. 1 is a block diagram for explaining an overview of a system according to an embodiment. An overview of the system 1 will be explained with reference to the same figure. System 1 grasps the situation when a vehicle accident occurs. Understanding the situation when a vehicle accident occurs includes at least specifying the relative position between the own vehicle and the other vehicle. Also, grasping the situation when a vehicle accident occurs may include estimating the relative speed between the own vehicle and the other vehicle, or may include estimating the absolute speed of the other vehicle. In addition, grasping the situation when a vehicle accident occurs may broadly include calculation of the percentage of fault.

The system 1 includes a UI server device 10, an AI server device 20, and a file server device 30. The UI server device 10, the AI server device 20, and the file server device 30 are connected to each other via a predetermined communication network NW. The communication network NW is a communication network configured using wired or wireless lines. The predetermined communication network NW may be an open network, and devices connected to the communication network NW can communicate with each other using a predetermined protocol. An example of the predetermined communication network NW is the Internet. The UI server device 10, the AI server device 20, and the file server device 30 communicate information with each other via the communication network NW.

Note that the assignment of the functional configuration of the system 1 as shown in the figure is an example, and the functions of the system 1 may be realized by one device, for example.

The UI server device 10 has a user interface function. Examples of the functions of the user interface include acquiring moving image data captured by a drive recorder and receiving instructions regarding understanding the situation when an accident occurs. The UI server device 10 outputs information to the AI server device 20 and the file server device 30 via the communication network NW based on the acquired moving image data and received instructions. The UI server device 10 includes a frame division section 11 and a request section 12.

The frame dividing unit 11 divides the acquired moving image data into frame images that are still images. In other words, the moving image data captured by the drive recorder includes a plurality of frame images. The frame dividing unit 11 outputs the divided frame images to the file server device 30 and stores them. The divided frame images may be associated with information for identifying the source moving image data, information regarding the order of frames, frame rate, and the like.

The requesting unit 12 issues a request to the AI server device 20 to start processing based on the received instruction regarding understanding the situation when an accident occurs. Instructions regarding situational awareness at the time of occurrence of an accident may be given through an operation by the user of the system 1. Examples of instructions related to understanding the situation when an accident occurs include vehicle recognition processing and speed estimation processing. The information output by the requesting unit 12 may include information for identifying the corresponding moving image data (or frame image) and moving image data.

The AI server device 20 has a function for performing processing related to grasping the situation when an accident occurs. As an example of the process related to grasping the situation when an accident occurs, vehicle recognition process, speed estimation process, etc. can be exemplified. The AI server device 20 includes a vehicle recognition processing section 21 and a speed estimation processing section 22.

The vehicle recognition processing unit 21 acquires a request for starting vehicle recognition processing from the UI server device 10. The request includes information (or frame image) for identifying the corresponding frame image. The vehicle recognition processing unit 21 acquires a corresponding frame image from the file server device 30, and performs vehicle recognition processing based on the acquired frame image. Vehicle recognition processing is processing for identifying the position of a vehicle in a frame image. Specifically, the vehicle recognition process may be a process of identifying the size and position coordinates of a bounding box surrounding the vehicle. A machine learning algorithm may be used for the processing. The vehicle recognition processing unit 21 outputs the vehicle recognition result obtained as a result of the processing to the file server device 30 and stores it.

The speed estimation processing unit 22 acquires a request for starting speed estimation processing from the UI server device 10. The request includes information for specifying the corresponding frame image (or vehicle recognition result). Further, the request may include the corresponding frame image (or vehicle recognition result) itself. The speed estimation processing unit 22 acquires vehicle recognition results corresponding to a plurality of frame images. The speed estimation processing unit 22 obtains a plurality of corresponding vehicle recognition results from the file server device 30, and performs a speed estimation process based on the obtained plurality of vehicle recognition results. The speed estimation process may be a process for estimating the relative speed between the host vehicle and the other vehicle, or may be a process for estimating the absolute speed of the other vehicle. Specifically, speed estimation processing is to estimate speed based on changes in the position (or coordinates) of a specific point (for example, the center point of a bounding box) for each frame based on the recognized vehicle position. You can. The speed estimation processing unit 22 outputs the speed estimation result obtained as a result of the processing to the file server device 30 and stores it.

The file server device 30 acquires a frame image from the UI server device 10 and stores it as a frame image 31. Further, the file server device 30 acquires the vehicle recognition result from the AI server device 20 and stores it as a vehicle recognition result 32. Further, the file server device 30 acquires the speed estimation result from the AI server device 20 and stores it as the speed estimation result 33. The frame image 31, the vehicle recognition result 32, and the speed estimation result 33 may be managed so that they can be associated with each other using, for example, information that identifies a drive recorder, or an accident number that is a number that identifies an accident case. good.

FIG. 2 is a sequence diagram for explaining an overview of processing performed by the system according to the embodiment. An overview of the processing performed by the system 1 will be explained with reference to the same figure. As a premise of the processing performed by the system 1 as shown in the figure, the UI server device 10 acquires moving image data from a drive recorder, divides the acquired moving image data into frame images, and sends the divided frame images to the file server device 30. It is assumed that it has been memorized.

First, vehicle recognition processing performed by the system 1 will be explained. The UI server device 10 outputs a vehicle recognition request to the AI server device 20 (step S110). Upon acquiring the vehicle recognition request, the AI server device 20 outputs a frame image request to the file server device 30 (step S120). In the frame image request, the frame image to be acquired is specified using the identification information (for example, the identification information of the drive recorder or the identification information of the moving image data) included in the vehicle recognition request. The file server device 30 outputs the frame image specified based on the frame image request to the AI server device 20 (step S130). Note that the number of frame images output from the file server device 30 to the AI server device 20 may be one or more, or may be plural. For example, all frame images identified by the identification information may be output. The AI server device 20 performs vehicle recognition processing based on the acquired frame image (step S140). Details of the vehicle recognition process will be described later. The AI server device 20 outputs the result of vehicle recognition to the file server device 30 (step S150). The file server device 30 stores vehicle recognition results in association with, for example, identification information of a drive recorder, identification information of moving image data, and the like.

Next, the speed estimation process performed by the system 1 will be explained. The UI server device 10 outputs a speed estimation request to the AI server device 20 (step S210). Upon acquiring the speed estimation request, the AI server device 20 outputs a vehicle recognition result request to the file server device 30 (step S220). In the vehicle recognition result request, the vehicle recognition result to be obtained is specified using identification information (for example, drive recorder identification information or video data identification information) included in the speed estimation request. The file server device 30 outputs the vehicle recognition result specified based on the vehicle recognition result request to the AI server device 20 (step S230). The AI server device 20 performs speed estimation processing based on the acquired vehicle recognition result (step S240). Details of the speed estimation will be described later. The AI server device 20 outputs the speed estimation result to the file server device 30 (step S250). The file server device 30 stores the speed estimation in association with, for example, identification information of a drive recorder, identification information of moving image data, and the like.

Next, details of the vehicle recognition process will be described with reference to FIGS. 3 to 8.

FIG. 3 is a flowchart for explaining details of vehicle recognition processing performed by the system according to the embodiment. The details of the vehicle recognition process will be explained with reference to the figure.

First, the vehicle recognition processing unit 21 of the AI server device 20 acquires a frame image from the file server device 30 based on a request from the requesting unit 12 (step S310). Here, the frame image stored in the file server device 30 corresponds to one frame of a moving image captured by a drive recorder. Images captured by a drive recorder have not been subjected to distortion correction, so when estimating a vehicle position using the images, the accuracy may decrease due to the effects of distortion. An image that has not been subjected to distortion correction can also be said to be an image based on a radial distortion model. In order to reduce the influence of distortion, the vehicle recognition processing unit 21 performs distortion correction on the acquired frame image (step S320). Details of the distortion correction processing performed by the vehicle recognition processing section 21 will be described with reference to FIGS. 4 and 5.

FIG. 4 is a first diagram for explaining distortion correction processing performed by the system according to the embodiment. FIG. 4A shows an example of an image that has not been subjected to distortion correction (that is, a frame image stored in the file server device 30). The figure shows an image captured in the traveling direction of the host vehicle. Furthermore, in the example shown in the figure, the other vehicle (oncoming vehicle) in the vehicle accident is reflected within the angle of view. FIG. 4(A) can also be said to be an image based on a Radial distortion model.

FIG. 4(B) shows an example of an image after the first correction is performed by the vehicle recognition processing unit 21. Specifically, FIG. 4(B) is an image based on the Pinhole camera model. FIG. 4C shows an example of an image after the second correction is performed by the vehicle recognition processing unit 21. Specifically, FIG. 4(C) is an image based on the FOV distortion model. As shown in the figure, the vehicle recognition processing unit 21 first converts into a pinhole camera model, and then converts into an FOV distortion model. The vehicle recognition processing unit 21 performs vehicle recognition processing based on the image that has been converted twice.

Here, if the process of first converting to the Pinhole camera model and then converting to the FOV distortion model is performed for each frame, a problem may arise in that the time required for the entire process becomes long. Therefore, by multiplying the matrix for converting to the Pinhole camera model and the matrix for converting to the FOV distortion model in advance, you can prepare the matrix for converting from the Pinhole camera model to the FOV distortion model. It is preferable to leave the In this case, by multiplying each frame by a matrix prepared in advance, it becomes possible to convert from a pinhole camera model to an FOV distortion model in one operation, reducing the time required for the entire processing. become able to.

FIG. 5 is a second diagram for explaining the distortion correction processing performed by the system according to the embodiment. The Pinhole camera model and FOV distortion model will be explained with reference to the same figure. The Pinhole camera model is a model in which light passes through the center of the lens surface and travels straight, forming an image on the imaging surface. The path of light in the pinhole camera model is shown by the dashed line in the figure. The plane indicated by the vertical solid line on the right side of the figure indicates the imaging plane, and corresponds to the image sensor in the camera. Further, the lens surface shown in the figure is a surface indicated by a vertical solid line passing through the center of a circle, and corresponds to a surface abstractly showing a lens in a camera. The circles shown in solid lines are drawn conceptually for illustrative purposes and do not represent any actual surface. According to the pinhole camera model, a straight object has the characteristic that it appears as a straight line. Furthermore, according to the pinhole camera model, a subject that forms a large angle with the optical axis (for example, a subject that is present at the edge of the image) has a characteristic that it appears greatly elongated.

The FOV distortion model is a model in which the distance from the center of the image plane is proportional to the angle that the subject makes with the optical axis. FOV The FOV of the distortion model is Field-of-view. The path of light in the FOV distortion model is shown by solid line segments connecting white circles. The line segment can be said to conceptually indicate the position where light passing through the center of the lens forms an image on the conceptual imaging plane in the FOV distortion model and on the actual imaging plane. According to the FOV distortion model, points that are the same distance from the center of the image plane also have the same angle with the optical axis. That is, according to the FOV distortion model, even a subject that forms a large angle with the optical axis (for example, a subject located at the edge of the image) will not be significantly stretched. Therefore, by performing vehicle recognition using the FOV distortion model, the vehicle recognition processing unit 21 can improve the recognition accuracy of vehicles present at the edges of the image.

Returning to FIG. 3, the vehicle recognition processing unit 21 detects a bounding box based on the image after the above-described distortion correction (step S330). The bounding box specifies the location of a moving object (for example, a vehicle such as a car, a motorcycle, or a two-wheeled vehicle, a pedestrian, etc.) that may be involved in an accident.

Here, it is preferable that the bounding box according to this embodiment is a three-dimensional bounding box (3D bounding box). A three-dimensional bounding box specifies the three-dimensional position coordinates of an object included in an image, which is two-dimensional information. That is, a three-dimensional bounding box differs from a two-dimensional bounding box in that it further includes depth information. For example, in the case of a vehicle, the coordinates of eight points that three-dimensionally surround the vehicle may be used to specify a three-dimensional bounding box that surrounds the vehicle. Note that the bounding box according to the present embodiment is not limited to an example of a three-dimensional bounding box, and a two-dimensional bounding box using only one of the six sides constituting the three-dimensional bounding box may be used. .

FIG. 6 is a diagram for explaining bounding box detection processing performed by the system according to the embodiment. The details of the bounding box detection process will be explained with reference to the figure. FIG. 6A shows an example of a three-dimensional bounding box specified by the vehicle recognition processing unit 21. The bounding box may be obtained, for example, as a result of inference using a trained model. In the illustrated example, three moving objects (specifically, vehicles) are reflected in the moving image captured by the drive recorder, so three bounding boxes are detected. FIG. 6(B) is a bird's eye view generated from the position coordinate information of the bounding box. The bird's-eye view shows the position coordinates where the moving object, identified from the position coordinates of the three-dimensional bounding box, is present. The bird's eye view has information about the distance to the position of the own vehicle, the direction in which the moving body exists as seen from the position of the own vehicle, and the orientation (progressing direction) of the moving body. The position of the host vehicle in the figure is indicated by an upward arrow at the bottom of the figure.

Returning to FIG. 3, the vehicle recognition processing unit 21 shapes the detection result obtained in step S330 described above (step S340). Here, the bounding box detection process in step S330 is performed using an FOV distortion model. Therefore, in step S340, the true coordinates in the camera coordinate system are calculated by shaping the detection results.

FIG. 7 is a first diagram for explaining detection result shaping processing performed by the system according to the embodiment. The details of the detection result shaping process will be explained with reference to the same figure. First, line L71 indicates the optical path at the time when a three-dimensional bounding box is detected using the FOV distortion model. Further, the coordinates at that point in time are shown as point P1. Since point P1 cannot be directly applied to the camera coordinate system (because an error will occur), the coordinates in the camera coordinate system are calculated by detection result shaping processing. A line L72 indicates the optical path after being converted from the FOV distortion model to the Pinhole camera model. Further, the coordinates at that point in time are shown as point P2. Point P2 is the coordinate obtained as a result of the detection result shaping process.

Note that, as a specific example of mutually converting the coordinates of a pixel on the image plane and the coordinates on the camera coordinate system, the following equation (1) can be exemplified. The 3×4 matrix in equation (1) is a camera parameter matrix.

cx and cy indicate the distance on the imaging plane from the optical axis, and fx and fy indicate the distance on the optical axis from the lens surface to the imaging plane. cx and cy are determined by setting them to half the width and height of the Pinhole camera model image. Further, fx and fy are determined by fx=cx/tanθ and fy=cy/tanθ (the angle of view θ is the angle formed by the optical axis and the optical path) using known cx and cy.

FIG. 8 is a second diagram for explaining the detection result shaping process performed by the system according to the embodiment. With reference to the figure, changes in the bird's-eye view before and after the detection result shaping process will be described. FIG. 8(A) shows a bird's-eye view before the detection result shaping process. FIG. 8(B) shows a bird's-eye view after the detection result shaping process. As shown in the figure, it can be seen that as a result of the detection result shaping process, the position of the moving object has moved closer to the host vehicle and has been converted to a position in the actual camera coordinate system.

Returning to FIG. 3, the vehicle recognition processing unit 21 outputs the vehicle recognition result to the file server device 30 (step S350). Note that the vehicle recognition result includes at least the coordinates of the bounding box. The vehicle recognition result may include an image including a three-dimensional bounding box as shown in FIG. 6(A), or a bird's-eye view as shown in FIG. 7(B).

Next, details of the speed estimation process will be described with reference to FIGS. 9 to 11.

FIG. 9 is a flowchart for explaining details of the speed estimation process performed by the system according to the embodiment. The details of the speed estimation process will be explained with reference to the figure. First, the speed estimation processing unit 22 of the AI server device 20 obtains a vehicle recognition result from the file server device 30 based on a request from the requesting unit 12 (step S410). The vehicle recognition result stored in the file server device 30 corresponds to one frame of a moving image captured by a drive recorder. The speed estimation processing unit 22 acquires vehicle recognition results for multiple frames. The speed estimation processing unit 22 tracks the target vehicle (the moving object that is the target of the speed estimation process) based on the acquired vehicle recognition results for a plurality of frames (step S420). Specifically, the tracking process is a process of identifying a bounding to be tracked among one or more bounding boxes in each frame (that is, identifying boundings surrounding the same vehicle in each frame).

A specific example of tracking processing will be described. First, among a plurality of frames to be subjected to speed estimation processing, the first and last two frames are identified. Furthermore, in each of the two specified frames, a bounding box surrounding the vehicle to be subjected to speed estimation processing is specified. The vehicle surrounded by the bounding box specified in the first frame (start frame) and the vehicle surrounded by the bounding box specified in the last frame (end frame) are the same vehicle. Further, it is preferable that the task of specifying two frames and the task of specifying a bounding box in the two specified frames are performed by the user's selection. Next, for a frame located between two frames, (1) the bounding box position calculated by linear interpolation from the bounding box position in the start frame and the bounding box position in the end frame, and (2) the bounding box position. Based on the position of the bounding box detected by the box detection unit 45, a bounding box surrounding the vehicle to be subjected to speed estimation processing is specified. The specific work may be, for example, to specify the bounding box (2) that has the largest area overlapping with the bounding box (1). In the case of a three-dimensional bounding box, the area comparison may be performed based on specific planes that make up the three-dimensional bounding box. In this way, in the tracking process, the bounding box to be tracked in each frame is specified based on the bounding box specified by the user. In addition, when the start frame is the 1st frame and the end frame is the nth frame (n is a natural number of 1 or more), this process is performed on the 1+1st frame, the n-1st frame, the 1+2nd frame, and the nth frame. -2nd frame, . . . , etc. may be performed in this order. In this case, the process of identifying the position of the bounding box in (1) by linear interpolation is based on the bounding boxes that have already been determined such as 1+1 frame, n-1 frame, 1+2 frame, n-2 frame, etc. It may be done based on.

The speed estimation processing unit 22 derives the relative speed between the own vehicle and the other vehicle (step S430). The position of the bounding box specified by the tracking process in step S420 and characteristics related to the drive recorder (for example, frame rate, etc.) are used to derive the relative speed. The characteristics related to the drive recorder may be stored in advance by the speed estimation processing unit 22, or may be included in a request from the UI server device 10, a frame image, a vehicle recognition result, or the like. In order to measure the amount of movement of the bounding box, a vertex of the bounding box may be used, or a point specified by a plurality of vertices (for example, a midpoint or center point) may be used.

FIG. 10 is a diagram for explaining an example of a three-dimensional bounding box and a tracking target point according to the embodiment. An example of points used in the velocity calculation process of a moving object will be explained with reference to the same figure. The figure shows a schematic diagram of a bounding box. Points A to H are the vertices of a bounding box surrounding the moving object. A rectangle formed by points A, B, C, and D corresponds to the front side of the vehicle, for example. A rectangle formed by points E, F, G, and H corresponds to, for example, the rear side of the vehicle. A rectangle formed by points C, D, G, and H corresponds to the lower side of the vehicle, for example. A rectangle formed by points A, B, E, and F corresponds to the upper side of the vehicle, for example. Point I is the center point of the lower rectangle of the vehicle. The speed estimation processing unit 22 may specify, for example, the center point of the lower rectangle as a tracking target point, and may track the tracking target point. Note that the center point on the front side or rear side may be used as the tracking target point, or the center point of a three-dimensional bounding box may be used.

Note that if the bounding box is a two-dimensional bounding, the center point of the two-dimensional bounding box (rectangle) may be used as the tracking target point, or the midpoint of the lower line segment may be used as the tracking target point. Good too.

FIG. 11 is a diagram for explaining an example of tracking processing performed by the system according to the embodiment. An example of tracking processing will be described with reference to the same figure. FIG. 11(A) is a bird's-eye view showing the position of the vehicle extracted from the first frame image, and FIG. 11(B) is a bird's-eye view showing the position of the vehicle extracted from the second frame image. The first frame image and the second frame image may be consecutive frames. When the frame rate is 10 [fps (frames per second)], the time from the situation shown in FIG. 11(A) to the situation shown in FIG. 11(B) is 0.1 [second]. In this case, for example, if the amount of movement of the vehicle per frame is 3 [m (meter)], the relative speed will be 3 [m] x 36000 = 108 [km/hour].

Returning to FIG. 9, the speed estimation processing unit 22 outputs the speed estimation result to the file server device 30 (step S440). Note that the speed estimation result includes at least information indicating relative speed. In addition, the speed estimation result may include information regarding the result estimated by the speed estimation processing section 22 (for example, absolute speed, etc.).

FIG. 12 is a block diagram showing an example of the functional configuration of the vehicle speed detection device according to the embodiment. An example of the functional configuration of the vehicle speed detection device 40 will be described with reference to the same figure. The vehicle speed detection device 40 has a functional configuration when the above-described vehicle speed detection method is attempted to be implemented by a single device. The functional configuration of the vehicle speed detection device 40 may be distributed among a plurality of devices, for example, as shown in FIG.

The vehicle speed detection device 40 includes a video acquisition section 41, a frame division section 42, an image acquisition section 43, a distortion correction section 44, a bounding box detection section 45, a tracking target point detection section 46, and a detection result shaping section. 47, a relative speed estimation section 48, a speed information acquisition section 49, and an absolute speed estimation section 411. Each of these functional units is realized using, for example, an electronic circuit. Furthermore, each functional unit may be provided with storage means such as a semiconductor memory or a magnetic hard disk device, if necessary. Further, each functional unit may be realized by a computer and software.

The video acquisition unit 41 acquires video data captured by a drive recorder. The moving image data captured by the drive recorder is stored in the moving image information storage section 51, for example. The video information storage section 51 may be included in the drive recorder, or may be a storage section that exists on the cloud.

The frame dividing section 42 divides the moving image data acquired by the moving image acquiring section 41 into frames to form frame images. The frame images divided by the frame dividing section 42 may be temporarily stored in a predetermined storage section (not shown). The storage unit may be included in the vehicle speed detection device 40 or may exist on the cloud.

The image acquisition unit 43 acquires the frame images divided by the frame division unit 42. In other words, the image acquisition unit 43 can also acquire one frame's worth of images from moving image data including a plurality of frames.

The distortion correction unit 44 performs distortion correction on the frame image acquired by the image acquisition unit 43. Distortion correction processing is performed to correct distortion of moving image data captured by a drive recorder. A specific example of the distortion correction process may be converting from a Radial distortion model to a Pinhole camera model, and further converting from a Pinhole camera model to an FOV distortion model.

The bounding box detection unit 45 detects a bounding box that specifies the position and size of the opponent vehicle included in the frame image. The frame image to be subjected to bounding box detection is an image acquired by the image acquisition unit 43. Preferably, the frame image to be subjected to bounding box detection is an image subjected to distortion correction by the distortion correction unit 44. Here, the other vehicle broadly includes vehicles such as automobiles, motorcycles, and two-wheeled vehicles. In the following description, a vehicle or the like detected by the bounding box detection unit 45 may be referred to as a first moving object. The first moving object is not limited to an example of a vehicle, and may include a wide variety of pedestrians, wild animals, and the like. Further, the own vehicle (the vehicle in which the drive recorder is installed) may be referred to as a second moving body. The second moving body may be other than a car, and may be a vehicle such as a motorcycle or a two-wheeled vehicle.

As described above, the bounding box detected by the bounding box detection unit 45 may be a two-dimensional bounding box, but preferably, it may be a three-dimensional bounding box (3D bounding box) having a depth direction. good. The three-dimensional bounding box three-dimensionally specifies the position and size of the opponent vehicle included in the frame image.

Further, the bounding box detection unit 45 may detect bounding boxes using a machine learning model learned in advance. The machine learning model is trained based on teacher data in which a predetermined input image and the position of a bounding box corresponding to the input image are associated with each other. When detecting bounding boxes (especially detecting three-dimensional bounding boxes) by machine learning, known techniques such as "Monocular Quasi-Dense 3D Object Tracking (hereinafter referred to as QD-3DT)" may be used. . A known data set may be used for learning a machine learning model using QD-3DT. In order to improve the bounding box detection accuracy, a predetermined number of images included in a known dataset are converted to nighttime images using a day/night conversion model, and then QD-3DT is trained. Good too. In other words, the training data used for learning the machine learning model includes images that have been converted to images taken at night as a result of image processing using a day/night conversion model based on images taken during the day. It can also be said that

Note that the position of the bounding box detected by the bounding box detection unit 45 is determined as a result of the tracking process performed in step S420 of FIG. The tracking process may be performed by a tracking unit (not shown) after the start frame and end frame are specified by the user.

The detection result shaping unit 47 converts the position coordinates of the bounding box detected by the bounding box detection unit 45 into position coordinates in the coordinate system of the image acquired by the image acquisition unit 43. Here, the bounding box detection unit 45 performs learning based on images captured at a predetermined angle of view (for example, about 67 degrees). Therefore, the detection result shaping process performed by the detection result shaping unit 47 is a process of correcting the angle of view at the time of learning of the bounding box detection unit 45 to the same angle of view as the actual video.

The tracking target point detection unit 46 detects the position coordinates of the tracking target point based on the bounding box. The bounding box used by the tracking target point detection unit 46 is one detected by the bounding box detection unit 45, and preferably the position coordinates may be converted by the detection result shaping unit 47. . Here, the tracking target point is a point used to specify the position of the vehicle for each frame. The tracked point is used to calculate speed. The tracking target point is a point based on the position and size of the detected bounding box, and it is preferable that how to specify the position of the tracking target point is determined in advance. The tracking target point detection unit 46 detects, for example, a point based on one vertex or two or more vertices included in a bounding box as a tracking target point. In this case, point C, point D, or the midpoint of a line segment consisting of point C and point D in FIG. 10 may be set as the tracking target point. Further, the tracking target point detection unit 46 detects, for example, the center point of two or more vertices included in the bounding box as the tracking target point. In this case, point I, which is the center point of a rectangle whose vertices are point C, point D, point G, and point H in FIG. 10, may be the tracking target point. In other words, the tracking target point detection unit 46 may detect a point included in the bottom surface of the detected three-dimensional bounding box as the tracking target point. The point included in the bottom surface may be, for example, a point where the tire contacts the ground. Further, the tracking target point detection unit 46 may detect the center of gravity of a rectangular parallelepiped, which is a three-dimensional bounding box, as the tracking target point. The tracking target point detection unit 46 performs tracking target point detection processing according to the position of the bounding box detected by the bounding box detection unit 45 and determined by the tracking process. I do.

The relative velocity estimating unit 48 detects a moving body (i.e., a moving body) in which an imaging device (i.e., a drive recorder) that captures moving image data is installed, based on changes in the position coordinates of the tracking target point detected in each of a plurality of frames of images. , the second moving body) and the moving body (that is, the first moving body) specified by the bounding box detected by the bounding box detection unit 45 is estimated. Specifically, the relative velocity estimating unit 48 may calculate the relative velocity by performing calculations based on the moving distance of the tracking target point for each frame and the frame rate.

The speed information acquisition unit 49 acquires the traveling speed of the host vehicle (ie, the second moving body). When the second moving body includes the vehicle speed sensor 61, the speed information acquisition unit 49 acquires speed information including information regarding the traveling speed from the vehicle speed sensor 61. The vehicle speed sensor 61 may detect the traveling speed based on the rotational speed of the shaft, for example, or may be calculated based on radio waves received from an artificial satellite such as GPS (Global Positioning System), and the speed information may be detected by the ECU. (Engine Control Unit or Electronic Control Unit). Furthermore, when the second moving object does not include the vehicle speed sensor 61, the vehicle speed may be estimated by image processing based on moving image data by providing a vehicle speed estimation section (not shown). For example, existing techniques such as optical flow may be used for the image processing.

The absolute speed estimating section 411 acquires information regarding relative speed from the relative speed estimating section 48 and acquires the traveling speed of the host vehicle from the speed information acquiring section 49. The absolute speed estimator 411 determines the moving object (i.e., the 1) Estimate the traveling speed of the moving object. The absolute speed estimated by the absolute speed estimation unit 411 may be output to a predetermined information processing device together with the relative speed estimated by the relative speed estimation unit 48.

Next, with reference to FIGS. 13 to 17, processing performed by the user interface of the system 1 according to the present embodiment, an example of a display screen displayed by the user interface, etc. will be described.

FIG. 13 is a flowchart illustrating an example of processing of a user interface included in the system according to the embodiment. A series of user interface processes will be described with reference to the same figure.

(Step S510) First, basic information is input by the user. The user is a person who uses the system 1, and may be an owner of a drive recorder, a staff member of an insurance company, or the like. The basic information input by the user includes information such as the date and time of the accident and the location. Furthermore, the basic information input by the user may include moving image data that is a target of understanding the accident situation.

(Step S520) Next, the user confirms and edits the video data. The user confirms the video data that is the target of understanding the accident situation. If the moving image data is different, the user replaces it with the correct one. Furthermore, in this step, the user may perform appropriate editing work on the moving image data. Here, depending on the length of the video data, videos that are not directly related to the accident situation may be included. Therefore, in this step, the user adjusts the moving image data to an appropriate length. The user identifies the moment of collision by editing the video data to a predetermined length.

(Steps S530 and S540) In this step, the user specifies the start position and end position of the video data.The start position may be, for example, a position where the other vehicle begins to be seen, and the end position is , for example, immediately after contact with the other vehicle occurs. In this step, a specific bounding box among one or more bounding boxes included in the frame at the start point (start frame) is specified as a tracking target, and one or more bounding boxes included in the frame at the end point (end frame) Among them, a specific bounding box is identified as a tracking target. The specific work is performed according to the user's selection.

(Step S550) When the start position and end position of the moving image data are determined, the system 1 performs AI determination. The AI determination includes vehicle recognition processing, speed estimation processing, etc. as described above. The AI processing may also include other processing necessary for understanding the accident situation. The system 1 transmits information to the user by outputting the results of the AI determination using a display screen as will be described later with reference to FIG.

(Step S560) The user confirms the results output in step S550 and confirms the accident situation. The user also inputs detailed information about the circumstances at the time of the accident.

(Step S570) Finally, the system 1 outputs the estimation result. The estimation result may include the relative speed between the own vehicle and the other vehicle, or may include the absolute speed of the other vehicle. A form in a predetermined format may be used to output the estimation results. An example of the form will be described later with reference to FIGS. 16 and 17.

FIG. 14 is a first diagram showing an example of a display screen displayed by a user interface included in the system according to the embodiment. A display screen D100, which is an example of a display screen displayed by the user interface, will be described with reference to the same figure. The display screen D100 is an example of a screen displayed when specifying the start in step S530 and specifying the end in step S540 described above. In the illustrated example, the components of the display screen displayed when specifying the end are shown, but the display screen displayed when specifying the start may also have similar components. The display screen D100 includes the symbols D111, D113, D121, D123, D131, D133, D141, D143, D145, D147, D151, and D151. D153, code D161, and code D163 are provided as screen components.

The code D111 indicates an accident number. The accident number is a number assigned to each accident case, and uniquely identifies the accident case. In the illustrated example, the accident number is "jikobango".

Reference numeral D113 indicates all steps of processing performed by the user interface. This step corresponds to the series of processes described with reference to FIG. The code D113 may indicate which step is currently displayed on the display screen. Specifically, it may be shown which step the current display screen is displaying by making the display color of the element indicating the current step different from the display color of the elements indicating other steps. In the illustrated example, the display color of the "end designation" element is different from the display color of elements indicating other steps, indicating that processing for "end designation" is being performed.

Reference numeral D121 indicates moving image data. An auxiliary line may be provided at the symbol D121 to be superimposed on the moving image data. The auxiliary line is for assisting in specifying the position of the other vehicle, and the user can operate the auxiliary line (or operate the frame specified by the auxiliary line).

Reference numeral D123 indicates a bird's eye view based on the moving image data. The bird's eye view is obtained as a result of AI processing by the system 1. The bird's-eye view shows the position of the other vehicle and the direction in which the other vehicle is facing.

Reference numeral D131 indicates the file name of the moving image data. Reference numeral D133 indicates the entire length of the moving image data, and reference numeral D121 indicates the point in time of the entire video being displayed. In the illustrated example, the entire length of the moving image data is 5.00 [seconds], and the video at the time of 2.3 [seconds] is displayed at D121.

Reference numerals D141 to D147 are buttons for selecting the frame to be displayed at reference numeral D121. By operating this button, the displayed frame can be moved in units of 1 frame or 10 frames. A frame is a frame. Specifically, when the frame rate is 10 [fps], one frame is 0.1 [second]. Therefore, the user can move the displayed frame in units of 0.1 [seconds] or 1.0 [seconds] by operating the button. Reference numeral D141 is a button for moving back 10 frames, D143 is a button for moving back 1 frame, D145 is a button for moving forward 1 frame, and D147 is a button for moving 10 frames forward.

Reference numerals D151 and D153 are buttons for reproducing and temporarily stopping moving image data. Reference numeral D151 is a button for playing, and reference D153 is a button for pausing. When the playback button is pressed, the moving image data is played back by switching frame images in units of 0.1 [seconds] at reference numeral D121.

Reference numerals D161 and D163 are buttons for performing step movement indicated by reference numeral D113. Reference numeral D161 is a button for going back a step, and D163 is a button for advancing a step. According to code D113, the current step is "end specification", so the step to which to return by pressing code D161 is "start specification", and the step to which to proceed by pressing code D163 is "AI judgment result" ( That is, the start of estimating the accident situation).

FIG. 15 is a second diagram showing an example of a display screen displayed by a user interface included in the system according to the embodiment. With reference to the figure, a display screen D200, which is an example of a display screen displayed by the user interface, will be described. The display screen D200 is an example of a screen displayed when outputting the AI determination result in step S550 described above. The display screen D200 includes a code D111, a code D113, a code D220, a code D231, a code D233, a code D241, a code D251, a code D265, and a code D267 as screen components. In the description of the display screen D200, the same components as those of the display screen D100 are given the same reference numerals and the description will be omitted.

Reference numeral D220 is a display section showing the result of element extraction by AI. The display section includes components D221 to D226. The code D221 indicates the form of the intersection. In the illustrated example, the shape of the intersection is a "crossroads." Reference numeral D222 indicates the direction in which the vehicle is traveling. In the illustrated example, the vehicle's traveling direction is "straight ahead". Reference numeral D223 indicates the initial position of the other vehicle. In the illustrated example, the initial position of the other vehicle is "to the left." Reference numeral D224 indicates the traveling direction of the other vehicle. In the illustrated example, the traveling direction of the other vehicle is "straight ahead". The code D225 indicates the presence or absence of a facing traffic light. In the illustrated example, the presence or absence of a facing traffic signal is "signal present". The code D226 indicates the color of the facing traffic light. In the illustrated example, the color of the facing traffic light is "blue." The information indicated by symbols D221 to D226 may be automatically determined by the AI server device 20 (or vehicle speed detection device 40). For this determination, a known object detection technique based on machine learning or a known pattern matching technique may be used. Further, as another embodiment, any of the codes D221 to D226 may be updated or newly input by the user.

Reference numeral D231 indicates a frame image of moving image data at the start point specified in step S530, the speed of the own vehicle, and the speed of the other vehicle. In the illustrated example, the speed of the own vehicle is "37.1 [km/h]" and the speed of the other vehicle is "50.4 [km/h]".

Reference numeral D233 indicates the frame image of the moving image data at the end point designated in step S540, the speed of the own vehicle, and the speed of the other vehicle. In the illustrated example, the speed of the own vehicle is "32.6 [km/h]" and the speed of the other vehicle is "20.9 [km/h]".

Reference numeral D241 indicates an accident situation diagram. The accident situation diagram is drawn based on the result of element extraction by AI indicated by reference numeral D220. In the illustrated example, the host vehicle is traveling straight at an intersection that is a crossroads, and the other vehicle is entering from the left side of the intersection. A traffic light is installed at the intersection, indicating that the traffic light is green.

Reference numeral D251 is a display section in which "accident type", "basic ratio", etc. are written. In the illustrated example, the "accident type" is "an accident between two vehicles traveling straight at an intersection with traffic lights". Furthermore, it is shown that the "basic ratio" is "[own vehicle] 0:[other vehicle] 100."

Reference numeral D265 is an accident video download button. The user can download the video data used for analysis by pressing the code D265, which is an accident video download button.

Reference numeral D267 is a still image download button from start designation to end designation (hereinafter simply referred to as still image download button). The user can download each frame image from the start point to the end point by pressing the still image download button D267. Alternatively, the user may be configured to be able to download all frame images by checking a predetermined checkbox and pressing the button.

FIG. 16 is a diagram showing an example of a form output by the system according to the embodiment. A form D300, which is an example of a form output by the system 1, will be described with reference to FIG. 16. The form D300 may be one form with columns, or may be two separate forms. Form D300 is an example of a screen displayed when outputting the estimation result in step S570 described above.

Form D300 shows "customer's vehicle speed analysis results." The form D300 includes a code D310 and a code D320 as constituent elements.

At D310, a scene-by-scene drive recorder image and the own vehicle speed corresponding to the image are displayed. In the illustrated example, situations in three scenes are displayed: "first scene of video (10 seconds before collision)," "collision scene," and "last scene of video (5 seconds before collision)." Also, the speed in the “first scene of the video (10 seconds before the collision)” is “50 [km/h]”, the speed in the “collision scene” is “40 [km/h]”, and the speed in “the video The speed in the last scene (5 seconds before the collision) is 10 [km/h].

A speed transition graph is displayed at D320. The speed change graph graph shows speed changes of the own vehicle and the other vehicle over time, with the vertical axis representing speed and the horizontal axis representing time. The speed change is a combination of speed information calculated for each frame.

[Summary of embodiments]
According to the embodiment described above, the vehicle speed detection device 40 includes the image acquisition unit 43 to acquire one frame of image from video data including a plurality of frames, and includes the bounding box detection unit 45. A bounding box that is detected by detecting a bounding box that specifies the position and size of a first moving object (for example, an opponent vehicle) that is a moving object included in an image acquired by the tracking target point detection unit 46 By detecting the position coordinates of the tracking target point, which is a point based on the position and size of The relative speed between the first moving body and a second moving body (for example, the own vehicle), which is a moving body equipped with an imaging device that captures image data, is estimated. That is, according to the vehicle speed detection device 40, the bounding box surrounding the opponent vehicle is first detected, and the speed is detected based on a change in the position of the bounding box. According to the present embodiment, the position of the moving body is specified based on the bounding box, so the accuracy of position specification can be improved. Therefore, according to this embodiment, the accuracy of speed calculation can be improved.

Further, according to the vehicle speed detection device 40 described above, the tracking target point detection unit 46 detects a point based on two or more vertices of the bounding box as a tracking target point. Here, according to the prior art, since the grounding point between the tire and the road surface was specified, the detected position differed from frame to frame. According to the present embodiment, a point based on two or more vertices of a bounding box is detected as a tracking target point, so the accuracy of position specification can be improved. Therefore, according to this embodiment, the accuracy of speed calculation can be improved.

Further, according to the vehicle speed detection device 40 described above, the tracking target point detection unit 46 detects the center point of two or more vertices of the bounding box as the tracking target point. Therefore, even if the position of one vertex shifts, the shift of the tracking target point can be suppressed. That is, according to this embodiment, the accuracy of position specification can be improved. Therefore, according to this embodiment, the accuracy of speed calculation can be improved.

Further, according to the vehicle speed detection device 40 described above, the bounding box is a three-dimensional bounding box (3D bounding box) having a depth direction. Therefore, according to this embodiment, the position of the moving body can be accurately specified. Therefore, according to the embodiment, the accuracy of positioning can be improved. Therefore, according to this embodiment, the accuracy of speed calculation can be improved.

Further, according to the vehicle speed detection device 40 described above, the tracking target point detection unit 46 detects a point included in the bottom surface of the detected three-dimensional bounding box as a tracking target point. That is, the tracking target point detection unit 46 specifies the position of the moving object based on the four points forming the bottom surface. Therefore, according to the embodiment, the accuracy of position specification can be further improved compared to the case where the position is specified based on two points. Therefore, according to this embodiment, the accuracy of speed calculation can be improved.

Further, according to the vehicle speed detection device 40 described above, the speed information acquisition unit 49 is provided to acquire the running speed of the second moving object (for example, the own vehicle), and the absolute speed estimation unit 411 is provided to obtain the traveling speed of the second moving object (for example, the host vehicle). The running speed (absolute speed) of the first moving body (for example, the other vehicle) is estimated based on the determined relative speed and the running speed of the second moving body. Therefore, according to this embodiment, it is possible to easily understand the accident situation. Further, according to the present embodiment, it is possible to estimate the traveling speed (absolute speed) of the first moving object (for example, the other vehicle) with high accuracy.

Further, according to the vehicle speed detection device 40 described above, by further including the distortion correction section 44, distortion correction of the image acquired by the image acquisition section 43 is performed. Further, the bounding box detection unit 45 detects a bounding box based on the image subjected to distortion correction by the distortion correction unit 44. Therefore, according to this embodiment, a bounding box can be detected with high accuracy in an image whose distortion has been corrected. Therefore, according to this embodiment, the accuracy of position specification can be improved. Therefore, according to this embodiment, the accuracy of speed calculation can be improved.

Further, according to the vehicle speed detection device 40 described above, by including the detection result shaping unit 47, the position coordinates of the detected bounding box are converted into position coordinates in the coordinate system of the image acquired by the image acquisition unit 43. do. Therefore, according to this embodiment, after the bounding box is detected in the coordinate system in which the distortion has been corrected, the position of the bounding box in the camera coordinate system can be specified.

Further, according to the vehicle speed detection device 40 described above, the bounding box detection unit 45 is a machine trained based on teacher data in which a predetermined input image and the position of a bounding box corresponding to the input image are associated with each other. Detect bounding boxes using a learning model. That is, the bounding box detection unit 45 detects the position of the bounding box by machine learning. Therefore, according to this embodiment, the position of the bounding box can be detected easily and accurately.

Further, according to the vehicle speed detection device 40 described above, the training data used for learning the machine learning model is obtained by performing image processing using a day/night conversion model based on images captured during the day. A converted image is included in the captured image. That is, the machine learning model used by the bounding box detection unit 45 is learned based on nighttime images. Therefore, the bounding box detection unit 45 can detect the position of the bounding box even in a nighttime image. Therefore, according to the present embodiment, since it is possible to deal with images taken at night, it is possible to accurately grasp the accident situation even if the accident occurs at night.

In addition, all or part of the functions of each part included in the system 1 in the embodiment described above can be achieved by recording a program for realizing these functions on a computer-readable recording medium, and then using the program recorded on this recording medium. This may be realized by loading and executing into a computer system. Note that the "computer system" herein includes hardware such as an OS and peripheral devices.
Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, magneto-optical disks, ROMs, and CD-ROMs, and storage units such as hard disks built into computer systems. Furthermore, a "computer-readable recording medium" refers to a storage medium that dynamically stores a program for a short period of time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. It may also include a device that retains a program for a certain period of time, such as a volatile memory inside a computer system that is a server or client. Further, the above-mentioned program may be one for realizing a part of the above-mentioned functions, or may be one that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

DESCRIPTION OF SYMBOLS 1... System, 10... UI server device, 11... Frame division part, 12... Request part, 20... AI server device, 21... Vehicle recognition processing part, 22... Speed estimation processing part, 30... File server device, 31... Frame Image, 32... Vehicle recognition result, 33... Speed estimation result, 40... Vehicle speed detection device, 41... Video acquisition section, 42... Frame division section, 43... Image acquisition section, 44... Distortion correction section, 45... Bounding box detection Section, 46... Tracking target point detection section, 47... Detection result shaping section, 48... Relative speed estimation section, 49... Speed information acquisition section, 411... Absolute speed estimation section, 51... Video information storage section, 61... Vehicle speed sensor

Claims (12)

an image acquisition unit that acquires an image for one frame from video data including multiple frames;
a bounding box detection unit that detects a bounding box that specifies the position and size of a first moving object that is a moving object included in the acquired image;
a tracking target point detection unit that detects the position coordinates of a tracking target point, which is a point based on the position and size of the detected bounding box;
Based on changes in the position coordinates of the tracking target point detected in each of a plurality of frames of images, a second moving body that is a moving body in which an imaging device that captures the moving image data is installed and the first moving body and a relative speed estimator for estimating the relative speed of
The relative speed estimation unit outputs information indicating the relative position between the second moving body and the first moving body for each frame of images, and calculates the relative speed based on a change in the output relative position. presume
Vehicle speed detection device. The vehicle speed detection device according to claim 1, wherein the tracking target point detection unit detects a point based on one or more vertices of the bounding box as the tracking target point. The vehicle speed detection device according to claim 2, wherein the tracking target point detection unit detects one vertex or a center point of two or more vertices of the bounding box as the tracking target point. The vehicle speed detection device according to claim 1 or 2, wherein the bounding box is a three-dimensional bounding box having a depth direction. The vehicle speed detection device according to claim 4, wherein the tracking target point detection unit detects a point included in a bottom surface of the detected three-dimensional bounding box as the tracking target point. a speed information acquisition unit that acquires the running speed of the second moving body;
The vehicle according to claim 1 or 2, further comprising: an absolute speed estimation unit that estimates the running speed of the first moving body based on the estimated relative speed and the running speed of the second moving body. Speed detection device. further comprising a distortion correction unit that corrects distortion of the image acquired by the image acquisition unit,
The vehicle speed detection device according to claim 1 or 2, wherein the bounding box detection section detects the bounding box based on an image subjected to distortion correction by the distortion correction section. The vehicle speed detection device according to claim 7 , further comprising a detection result shaping unit that converts the detected position coordinates of the bounding box into position coordinates in the coordinate system of the image acquired by the image acquisition unit. The bounding box detection unit detects the bounding box using a machine learning model learned based on teacher data in which a predetermined input image and a position of a bounding box corresponding to the input image are associated with each other. 3. The vehicle speed detection device according to claim 1 or claim 2. The training data used for learning the machine learning model includes images that are converted into images taken at night as a result of image processing using a day/night conversion model based on images taken during the day. The vehicle speed detection device according to claim 9. an image acquisition step of acquiring an image for one frame from video data including multiple frames;
a bounding box detection step of detecting a bounding box that specifies the position and size of a first moving object that is a moving object included in the acquired image;
a tracking target point detection step of detecting the position coordinates of the tracking target point, which is a point based on the position and size of the detected bounding box;
Based on changes in the position coordinates of the tracking target point detected in each of a plurality of frames of images, a second moving body that is a moving body in which an imaging device that captures the moving image data is installed and the first moving body and a relative speed estimation step of estimating the relative speed of
The relative velocity estimation step outputs information indicating the relative position of the second moving body and the first moving body for each frame of images, and calculates the relative velocity based on a change in the output relative position. presume
Vehicle speed detection method. to the computer,
an image acquisition step of acquiring an image for one frame from video data including multiple frames;
a bounding box detection step of detecting a bounding box that specifies the position and size of a first moving object that is a moving object included in the acquired image;
a tracking target point detection step of detecting the position coordinates of the tracking target point, which is a point based on the position and size of the detected bounding box;
Based on changes in the position coordinates of the tracking target point detected in each of a plurality of frames of images, a second moving body that is a moving body in which an imaging device that captures the moving image data is installed and the first moving body A relative speed estimation step of estimating the relative speed of
The relative speed estimating step outputs information indicating the relative position between the second moving body and the first moving body for each frame of images, and calculates the relative speed based on a change in the output relative position. presume
program.

Images