JP6611315B2 - Video reference point determination system, video reference point determination method, and video reference point determination program - Google Patents

Description

  The present invention relates to a video reference point determination system, a video reference point determination method, and a video for determining a specific reference point on the video used when recognizing a video shot by an in-vehicle camera installed on a vehicle. It relates to a reference point determination program.

  Conventionally, various situations have been detected in a vehicle by automatically recognizing an image captured by an in-vehicle camera. For example, in the meandering operation detection device for a vehicle disclosed in Patent Document 1, a lateral position detection unit that detects a lateral positional relationship between a lane and the vehicle on the vehicle and a meandering operation based on a change in the positional relationship. An abnormality detecting means for identifying the presence or absence of abnormality is provided.

  As shown in Patent Document 1, when recognizing an image obtained by photographing a landscape such as a road with an in-vehicle camera on a traveling vehicle, the position of each subject in the image is accompanied by the movement of the own vehicle. Move. Further, when processing such an image, if a vector representing the movement of each pixel in the image, that is, an optical flow is detected, an extension line of all the optical flows is made one point for a non-moving subject. Converge. This point is FOE (Focus of Expansion), that is, the vanishing point of motion.

  For example, in an image taken with an in-vehicle camera on a vehicle running straight in a direction along the road on a straight road, the earliest infinity point of the road in the image is the FOE. become. Each pixel in the video moves so as to spread radially from the FOE position. Also, by registering such FOE as a fixed reference point in advance, various recognition processes can be performed.

  Due to the difference in lateral position between the intersection of the extension lines of the lane markings (white lines) on the left and right roads of the host vehicle and the registered FOE reference point, the difference between the direction in which the road extends and the direction of travel of the host vehicle Can be detected. Further, if the reference point of the FOE is known, the direction in which each lane line extends can be specified even if each lane line in the image is not a continuous solid line, so that the lane line can be easily recognized.

  The reference point as described above needs to be individually adjusted and determined for each vehicle so as to be in an optimum position. For example, calibrate the reference point so that errors due to effects such as displacement of the mounting position of the in-vehicle camera, displacement of the shooting direction, difference in angle of view, difference in vehicle height, difference in vehicle tilt in the front-rear direction, etc. do not occur. There is a need.

  In the infinity point determination apparatus of Patent Document 2, an apparatus that can be used for calibrating the reference point of the infinity point (FOE) is shown. In the case of calibrating the reference point of the infinity point (FOE), as shown in FIG. 3 of Patent Document 2, conventionally, in the work process of adjusting each vehicle, the vehicle is stopped in front of the stopped vehicle. A calibration target is arranged, the position of the target is detected from an image taken by the in-vehicle camera 130, and the reference point is determined based on this position.

  Moreover, in the infinity point determination apparatus shown by patent document 3, it determined so that the bonnet of the own vehicle might be included in the imaging | photography range of a vehicle-mounted camera, and a reference point was set based on the position of the vertex of the bonnet in an image. Has been decided. Therefore, it is not necessary to install the target during calibration.

  Moreover, in the vehicle inclination measuring apparatus of patent document 4, the 1st vanishing point detected in the state which the vehicle is an empty state and has stopped on the flat ground, and the 2nd vanishing point detected in a driving state Based on the above, the inclination angle of the vehicle is calculated. When the second vanishing point is detected, when the vehicle is actually traveling on the road and the vehicle speed exceeds a predetermined minimum speed, the image is processed and the corresponding point is calculated from the optical flow or white line. ing.

JP 2011-227551 A JP 2006-322855 A JP 2006-327495 A JP 2014-224774 A

  As described above, in order to determine the reference point of FOE, in order to reduce individual differences and errors, a dedicated infinite number as shown in Patent Document 2 and Patent Document 3 is provided for each vehicle on which an in-vehicle camera is installed. It was necessary to carry out calibration work individually while bringing a far-point determination device, a general-purpose personal computer (PC), etc. onto the vehicle and viewing an image taken by the worker. Further, when performing such a calibration operation, a target (target) as shown in Patent Document 2 is disposed in front of the target vehicle, and the target vehicle and the target are disposed on a flat place. Because it is necessary, a sufficiently large work space is required.

  On the other hand, in the actual driving state of the vehicle, there is a possibility that it may be affected by the loading of luggage, the replacement of tires, the change in the mounting state of the in-vehicle camera, etc. It is assumed that the position of the calibrated reference point will be in an inappropriate state because of deviation from the work environment.

Note that, as shown in Patent Document 4, it is also possible to calculate an appropriate point from an optical flow or a white line by processing an image while the vehicle is actually traveling on a road. However, in a situation where the vehicle is traveling on an actual road, the following changes appear in the video to be taken.
(1) Changes in the left-right direction due to the influence of a curved road shape.
(2) Changes in the vertical direction due to the influence of unevenness on the road surface.
(3) Changes in the vertical direction due to the influence of the slope of the road surface when traveling on a slope.
(4) A change in the left-right direction due to the influence of the deviation between the actual traveling direction of the host vehicle and the road direction.

  Therefore, there is a high possibility that the FOE point detected when the vehicle is actually traveling on the road cannot be used as the reference point.

  The present invention has been made in view of the above-described circumstances, and an object of the present invention is to provide a video reference point determination system, a video reference point determination method, and a video reference point determination program that can easily determine an FOE reference point appropriately. It is to provide.

In order to achieve the above-described object, a video reference point determination system, a video reference point determination method, and a video reference point determination program according to the present invention are characterized by the following (1) to (6).
(1) A video reference comprising: a vehicle-mounted camera installed on a vehicle; and a management device for determining a specific reference point on the video used when recognizing a video captured by the vehicle-mounted camera. A point determination system,
The management device reads recorded image data of a video obtained by photographing with the in-vehicle camera while the vehicle is running, and extracts processing target recorded image data satisfying a predetermined condition from the recorded image data An extractor to perform,
For each of a plurality of recording sections included in the processing target record image data, performs predetermined image recognition processing, the left than the vehicle appearing in a direction along the road on the road on which the vehicle is traveling A first line, a second line on the right side of the vehicle, and a third line corresponding to the horizon, respectively.
Among the plurality of recording sections, the data of the recording section in which the three lines of the first line, the second line, and the third line do not intersect at a common point is excluded from the calculation target, and the three lines A reference point determination unit that calculates an intersection of the three lines as a reference point for each of the recording sections that intersect at a common point, and determines a definite value of the reference point based on the plurality of calculated reference points; It is characterized by having.

  According to the video reference point determination system of (1) above, since the management device determines the reference point based on the recorded image data of the video captured by the vehicle-mounted device while the vehicle is running, There is no need to prepare a special flat space or target. In addition, compared with the case where calibration is performed in a stopped state, errors due to the effects of loading / unloading, tire replacement, changes in the mounting state of the in-vehicle camera, and the like do not occur, so that more appropriate calibration is realized. In addition, the recorded image data of the video shot while the vehicle is traveling is likely to include a scene that is not suitable for specifying the reference point. However, the predetermined condition in the recorded image data is satisfied. Since the processing target recording image data is extracted and processed, the reference point can be determined at an appropriate position. Further, since the processing target recording image data is extracted and processed, the amount of data to be processed can be reduced and the time required for processing can be shortened.

(2) A video reference point determination method for determining a specific reference point on the video used when recognizing a video shot by an in-vehicle camera installed on a vehicle,
Read the recorded image data of the video obtained by photographing with the in-vehicle camera during the traveling of the vehicle, extract the processing target recorded image data satisfying a predetermined condition in the recorded image data,
A predetermined image recognition process is performed on each of the plurality of recording sections included in the processing target recording image data, and the left side of the vehicle appearing in the direction along the road on the road on which the vehicle is traveling Detecting the first line, the second line on the right side of the vehicle, and the third line corresponding to the horizon,
Among the plurality of recording sections, the data of the recording section in which the three lines of the first line, the second line, and the third line do not intersect at a common point is excluded from the calculation target, and the three lines For each of the recording sections intersecting at a common point, the intersection of the three lines is calculated as a reference point, and a fixed value of the reference point is determined based on the plurality of calculated reference points .
It is characterized by that.

  According to the video reference point determination method of (2) above, since the reference point is determined based on the recorded image data of the video taken while the vehicle is traveling, a special flat space or target for calibration is required. There is no need to prepare anything. In addition, compared with the case where calibration is performed in a stopped state, errors due to the effects of loading / unloading, tire replacement, changes in the mounting state of the in-vehicle camera, and the like do not occur, so that more appropriate calibration is realized. In addition, the recorded image data of the video shot while the vehicle is traveling is likely to include a scene that is not suitable for specifying the reference point. However, the predetermined condition in the recorded image data is satisfied. Since the processing target recording image data is extracted and processed, the reference point can be determined at an appropriate position. Further, since the processing target recording image data is extracted and processed, the amount of data to be processed can be reduced and the time required for processing can be shortened.

(3) As the processing target recording image data, extract data of a section in which the vehicle is traveling on an expressway or an automobile exclusive road.
The video reference point determination method according to (2) above, characterized in that:

  According to the video reference point determination method of (3) above, since the data of the section where the vehicle is traveling on the expressway or the automobile exclusive road is used as the processing target recording image data, the reference point is determined at an appropriate position. Can do. In other words, on expressways and motorway roads, there are few places that are curved compared to ordinary roads, and there are few irregularities on the road surface, so there is one extension line for the first, second, and third lines. There is a high possibility of crossing at a point, and the probability that this point is at an appropriate position as a reference point for FOE is also high.

(4) A result of performing statistical processing by collecting information of the plurality of calculated reference points is output as a fixed value of the reference points.
The video reference point determination method according to (2) above, characterized in that:

  According to the video reference point determination method of (4) above, when a large number of reference point candidates each including some errors are obtained as reference points, the positions of the large number of reference point candidates are averaged by statistical processing, for example. By doing so, a result with few errors can be output as a definite value of the reference point.

(5) When new data is added as the recorded image data, the determined value is updated using the new reference point detected based on the added data.
The video reference point determination method according to (4) above, wherein

  According to the video reference point determination method of (5) above, when new data is added to the recorded image data, the final value is updated to reflect the content of this data, so the final value is optimized. Can do. For example, when the optimal position of the reference point in the image changes due to changes in the loading status of the luggage, tire changes, changes in the mounting state of the in-vehicle camera, etc., the fixed value is set so as to follow the result. It can be corrected.

(6) A video reference point determination program that causes a computer to execute the procedure of the video reference point determination method according to any one of (2) to (5) above.

  The reference point can be determined at an appropriate position by reading and executing the video reference point determination program of (6) above with a management computer or a computer in the vehicle-mounted device. Further, since the reference point is determined based on the recorded image data of the video taken while the vehicle is traveling, it is not necessary to prepare a special flat space or target for calibration. In addition, compared with the case where calibration is performed in a stopped state, errors due to the effects of loading / unloading, tire replacement, changes in the mounting state of the in-vehicle camera, and the like do not occur, so that more appropriate calibration is realized. In addition, the recorded image data of the video shot while the vehicle is traveling is likely to include a scene that is not suitable for specifying the reference point. However, the predetermined condition in the recorded image data is satisfied. Since the processing target recording image data is extracted and processed, the reference point can be determined at an appropriate position. Further, since the processing target recording image data is extracted and processed, the amount of data to be processed can be reduced and the time required for processing can be shortened.

  The video reference point determination method according to the present invention may be as described in the following (7) to (9).

(7) The video reference point determination method according to (2) above,
If the first line, the second line, and the third line do not intersect at a common point, the corresponding data is excluded from the calculation target.
It is characterized by that.

  According to the video reference point determination method of (7) above, even when an image of an inappropriate scene is included in the processing target recording image data, when the data is processed, the corresponding data is used as the reference point. Can be excluded from the calculation target. For example, when any of the first line, the second line, and the third line is erroneously recognized due to the influence of some noise included in the image, the result of the erroneous recognition is reflected in the position of the reference point. Can be prevented.

(8) The video reference point determination method according to (3) above,
If the recorded image data includes position information representing the current position of the vehicle traveling,
The road corresponding to the processing target data is identified by comparing predetermined road map data with the position information in the recorded image data, and whether the vehicle is running on an expressway or an automobile-only road Identify whether or not
It is characterized by that.

  According to the video reference point determination method of (8) above, it is possible to easily identify whether or not the content of each section of the recorded image data is data of a section in which the vehicle is traveling on an expressway or an automobile exclusive road. Therefore, the processing target recording image data can be easily extracted.

(9) The video reference point determination method according to (3) above,
If the recorded image data includes road type information indicating the type of road on which the vehicle is running,
With reference to the road type information in the recorded image data, it is identified whether or not the vehicle is data of a section running on an expressway or an automobile exclusive road,
It is characterized by that.

  According to the video reference point determination method of (9) above, it is possible to easily identify whether or not the content of each section of the recorded image data is data of a section in which the vehicle is traveling on an expressway or an automobile exclusive road. For example, when using recorded image data recorded by an operation recorder such as a digital tachograph, the recorded image data includes road type information that distinguishes expressways, dedicated roads, and ordinary roads. It is easy to distinguish between expressways, exclusive roads and ordinary roads. Therefore, the processing target recording image data can be easily extracted.

  According to the video reference point determination method and the video reference point determination program of the present invention, it is easy to appropriately determine the FOE reference point. That is, it is not necessary to carry out a calibration work by bringing a PC for calibration work into each vehicle, and an error is less likely to occur than in the case of calibration in a stopped state, thereby realizing more appropriate calibration.

  The present invention has been briefly described above. Further, the details of the present invention will be further clarified by reading through a mode for carrying out the invention described below (hereinafter referred to as “embodiment”) with reference to the accompanying drawings. .

FIG. 1 is a flowchart showing a main processing procedure when the video reference point determination method of the embodiment of the present invention is applied to the operation of the office PC. FIG. 2 is a block diagram showing a configuration example of a system to which the present invention can be applied. FIG. 3 is a flowchart showing an operation example of the vehicle-mounted device. FIG. 4 is a front view showing a specific example (1) of the screen when the recorded image frame is reproduced and displayed on the screen. FIG. 5 is a front view showing a specific example (2) of the screen when the recorded image frame is reproduced and displayed on the screen. FIG. 6 is a side view showing a specific example of the appearance of the host vehicle equipped with the vehicle-mounted device.

  Specific embodiments relating to the present invention will be described below with reference to the drawings.

First, an outline of a system to which the present invention can be applied will be described.
A configuration example of a system to which the present invention is applicable is shown in FIG. Moreover, the specific example of the external appearance of the own vehicle carrying the onboard equipment 10 is shown in FIG. FIG. 4 shows a specific example (1) of the screen when the recorded image frame 50 is reproduced and displayed on the screen.

  As shown in FIG. 6, the vehicle-mounted device 10 is mounted on the own vehicle 56 to be managed. The vehicle-mounted device 10 has a function of a digital tachograph, that is, a vehicle operation recorder. Further, a high-quality vehicle-mounted camera 31 is connected to the video input of the vehicle-mounted device 10. The vehicle-mounted device 10 can automatically record the corresponding recorded image data by processing the video taken by the high-quality vehicle-mounted camera 31.

  The high image quality in-vehicle camera 31 is installed in the passenger compartment of the host vehicle 56 and is fixed in a direction in which a landscape including the road surface of the road ahead in the traveling direction can be photographed. By processing the video signal output from the high-quality vehicle-mounted camera 31, for example, a recorded image frame 50 as shown in FIG. 4 can be generated.

  That is, various images of the subject such as the traveling lane 51 of the own vehicle, the preceding vehicle 52, the lane left lane marking 53, the lane right lane marking 54, and the horizon 55 appear in the video captured by the high-quality in-vehicle camera 31. . Each of the lane left side lane line 53 and the lane right side lane line 54 is a visible line extending in a direction parallel to the direction in which the road extends, like a white line marked on the road surface of the road. The horizon 55 is a visible horizontal line corresponding to the boundary between the ground-side region and the sky region.

  For example, in the example of the recorded image frame 50 shown in FIG. 4, the lane left lane marking 53, the lane right lane marking 54, and the extended line of the horizon 55 intersect at one point. This point is a motion vanishing point FOE (Focus of Expansion), which corresponds to an infinite point. That is, when the host vehicle 56 travels along the travel lane 51, the high-quality vehicle-mounted camera 31 and each subject move relatively, so that each image in the recorded image frame 50 flows radially from the FOE point. To move.

  When processing the video of the high-quality vehicle-mounted camera 31 while the host vehicle 56 is traveling, if a vector representing the movement of each pixel included in the recorded image frame, that is, the optical flow is detected, The extension line of the optical flow converges to one point. This point is FOE, that is, the vanishing point of motion. In the example of the recorded image frame 50 shown in FIG. 4, the extending direction of each of the lane left partition line 53 and the lane right partition line 54 coincides with the optical flow direction. The extension of the line 54 and the horizon 55 intersects at the FOE position.

  Such FOE represents a position on the image frame corresponding to the point at infinity in the direction in which the host vehicle 56 is heading, so that the high-quality in-vehicle camera 31 is fixed to the vehicle body of the host vehicle 56. There is little change, and the FOE can be used as a reference point on the image frame.

  However, for example, when the vehicle height or inclination changes due to the condition of the luggage loaded on the vehicle body, the influence of acceleration while driving, the influence of unevenness of the road surface, etc., the vehicle body direction is parallel to the road surface. In such a case, the FOE position on the image frame may be shifted in the vertical direction.

  For example, when the intersection of the extension line of the lane left lane line 53 and the lane right lane line 54 is detected as FOE, the traveling direction of the host vehicle 56 is changed, for example, when the host vehicle 56 is meandering. When the traveling lane 51 of the host vehicle does not coincide with the extending direction, the position of the intersection and the FOE do not coincide. Further, when the traveling lane 51 of the own vehicle is curved instead of a straight line, a positional deviation occurs. Further, when the road has unevenness, when traveling on a slope, or when the host vehicle 56 is inclined in the front-rear direction, the position of the horizon 55 in the recorded image frame 50 fluctuates up and down. .

  Such FOE can be used as a reference position in various applications. For example, when the host vehicle 56 is traveling straight along the road, each lane line on the road extends in a direction toward the FOE, so that the FOE can be used as a reference position for recognizing each lane line. . Further, by comparing the intersection point on the extension line of the left and right lane markings of the own vehicle 56 with the FOE, it is possible to detect a deviation between the direction of the traveling lane 51 and the actual traveling direction of the own vehicle 56.

  Therefore, in an application for performing image recognition of the recorded image frame 50 as shown in FIG. 4, the FOE reference position in the image frame is registered in advance as fixed data. For this registration, in general, with the host vehicle 56 stopped in a flat and wide place, a predetermined target is placed in front of the vehicle 56 and calibrated by visual inspection by an operator or the like. Has been identified.

  On the other hand, when the video reference point determination method or the video reference point determination program of the present invention is used, the reference position can be automatically specified as will be described later, so that the calibration work can be performed with a very simple operation. it can. That is, in the video reference point determination method or the video reference point determination program of the present invention, for example, the office PC 40 reads and analyzes the recorded image data recorded by the vehicle-mounted device 10 in advance, so that the reference position is automatically determined. Can be identified. The recorded image data is a result of the vehicle-mounted device 10 recording a video actually captured by the high-quality vehicle-mounted camera 31 while the host vehicle 56 is traveling.

Next, a configuration example of the vehicle-mounted device 10 will be described.
2 includes a microcomputer (CPU) 11, an image processing unit 12, a G (acceleration) sensor 13, a speaker 14, a GPS interface (I / F) 15, a speed interface 16, and a flash memory (FROM). 17, a card interface 18, a data processing unit 19, a flash memory 20, a temporary storage memory (DDR) 21, and a power interface 22.

  The video signal output from the above-described high-quality vehicle-mounted camera 31 is input to the image processing unit 12. The image processing unit 12 converts the input video signal into digital image data for each frame. Further, the image processing unit 12 executes image recognition of the digital image data, and recognizes, for example, the preceding vehicle 52, the lane left lane line 53, the lane right lane line 54, and the horizon 55 shown in FIG.

  The microcomputer 11 performs control for realizing various functions required for the vehicle-mounted device 10 by executing a program incorporated in advance. Since the vehicle-mounted device 10 of the present embodiment is a digital tachograph, the microcomputer 11 automatically acquires various information representing the state of the vehicle and the operation state as a basic operation and stores the data on the memory card 32. Record. In addition, digital image data corresponding to the video of the high-quality in-vehicle camera 31 is also recorded on the memory card 32.

  The G sensor 13 can detect the magnitude of acceleration applied in the front-rear direction and the left-right direction of the host vehicle. The speaker 14 can convert a voice electric signal into sound and output a voice message, instruction, alarm, or the like.

  The GPS interface 15 performs signal processing for connecting the GPS receiver 33 to the vehicle-mounted device 10. The GPS receiver 33 receives radio waves from a plurality of GPS (Global Positioning System) satellites, and can calculate latitude and longitude information representing the current position of the host vehicle based on the reception time.

  The speed interface 16 inputs a speed pulse signal output from a vehicle speed sensor on the vehicle side, and converts it into a signal suitable for processing by the microcomputer 11. The microcomputer 11 can calculate the current vehicle speed (km / h) based on the pulse period of the speed pulse signal and the number of pulses within a certain time. The travel distance can also be calculated from the number of pulses of the speed pulse signal.

  The flash memories 17 and 20 are nonvolatile read-only memories that can rewrite data. The flash memory 17 holds predetermined backup data. The flash memory 20 holds programs executed by the microcomputer 11 and data.

  The card interface 18 has a predetermined card slot capable of detachably holding the memory card 32 and performs signal processing for connecting the memory card 32 to the vehicle-mounted device 10.

  The data processing unit 19 accesses the memory card 32 attached to the card interface 18 and performs data processing necessary for reading and writing data. For example, processing such as access permission control, data encryption / decryption control, and data compression / decompression is performed.

  The temporary storage memory 21 is a memory that can freely read and write data, and is used as a work area for temporarily storing various data by accessing the microcomputer 11.

  The power supply interface 22 generates a stable power supply voltage (+5 V, etc.) required inside the vehicle-mounted device 10 based on predetermined power supply (BATT: +12 V, etc.) supplied from the vehicle side, A function of converting an on / off signal (IGN) into a voltage suitable for processing of the microcomputer 11 is provided.

  The memory card 32 is a card that conforms to a predetermined standard with a built-in nonvolatile memory, such as a commercially available SD card or CF card, and has a shape that can be mounted in a card slot of the card interface 18. Moreover, about the memory card 32 with which the onboard equipment 10 is mounted | worn, information, such as ID corresponding to the specific crew member who uses this, or a specific vehicle, is written beforehand.

  On the other hand, an office PC 40 is installed in an office of a company or the like that manages the operation of a vehicle equipped with the vehicle-mounted device 10. This office PC 40 is configured by incorporating dedicated software for performing vehicle operation management, crew labor management, vehicle-mounted device management, and the like into a general personal computer. The software includes a video reference point determination program of the present invention. The office PC 40 also has a large capacity storage device such as a card slot into which the memory card 32 can be mounted and a hard disk for placing the database DB1.

  In the present embodiment, it is assumed that data is transferred between the vehicle-mounted device 10 and the office PC 40 by physical movement of the memory card 32. However, the vehicle-mounted device 10 has a wireless communication function. In this case, it is possible to transfer data without moving the memory card 32.

  An operation example of the vehicle-mounted device 10 is shown in FIG. That is, the microcomputer 11 in the vehicle-mounted device 10 shown in FIG. 2 executes the processing of FIG.

  In the in-vehicle device 10, when the image processing unit 12 executes image recognition, the FOE reference point as shown in FIG. 4 is registered in advance as a constant so that the FOE can be used efficiently. Recognition processing can be performed.

  For example, when the ignition of the vehicle equipped with the vehicle-mounted device 10 is turned on, the process starts from step S31 in FIG. In S31, the microcomputer 11 identifies whether or not constant data representing the reference point of the FOE is already registered on the vehicle-mounted device 10. If it is registered, the process proceeds to S32. If it is not registered, the process proceeds to S33.

  In step S32, the microcomputer 11 reflects this in the control of the vehicle-mounted device 10 so that the registered FOE reference point is used by the image processing unit 12 or the like. For example, when the latest information indicating the reference point of the FOE exists on the memory card 32 mounted on the in-vehicle device 10, the FOE reference point registered in the in-vehicle device 10 is updated by reading this information. Rewrite to the data.

  In step S33, the recording operation in the vehicle-mounted device 10 is started. After that, the microcomputer 11 periodically collects operation information such as the acceleration detected by the G sensor 13, the current position detected by the GPS receiver 33, the vehicle speed corresponding to the input speed pulse, the travel distance, etc. The information is recorded on the memory card 32 together with the information.

  Although not shown in FIG. 2, a general digital tachograph has a plurality of buttons that can be operated by the driver. For example, there are buttons for selecting the type of road on which the vehicle is running, such as a “high speed” button, a “dedicated” button, and a “general” button. Therefore, the vehicle-mounted device 10 can identify whether the currently traveling road is “highway”, “exclusive road”, or “general road” by reading a button operation. In addition, when this vehicle is equipped with an ETC (Electronic Toll Collection) on-vehicle device (not shown), the type of road currently running is determined based on the recognition state of this ETC on-vehicle device. Can also be identified. Therefore, the operation information recorded in the memory card 32 by the vehicle-mounted device 10 may include information indicating a distinction such as “highway”, “exclusive road”, and “general road”.

  In step S34, the microcomputer 11 executes image recording. That is, digital image data for each frame output by the image processing unit 12 is recorded on the memory card 32.

  While the operation of the vehicle is continued, step S34 is repeatedly executed, so that digital image data such as a recorded image frame 50 shown in FIG. 4 is sequentially recorded on the memory card 32, for example.

For example, when the microcomputer 11 detects that the ignition of the vehicle has been turned off, the operation in FIG. Note that it is not necessary to record all the frames continuously for the digital image data recorded in step S34, and it is possible to record only about one frame of data per second.

  FIG. 1 shows a main processing procedure when the video reference point determination method of the embodiment of the present invention is applied to the operation of the office PC 40. Further, the processing procedure shown in FIG. 1 corresponds to an operation realized by executing the video reference point determination program according to the embodiment of the present invention.

  The processing procedure shown in FIG. 1 will be described below. Before executing the processing procedure shown in FIG. 1 on the office PC 40, the memory card 32 on which predetermined operation information is recorded is removed from the vehicle-mounted device 10 and taken back to the office, and is inserted into the card slot of the office PC 40. There is a need to.

  In step S11 of FIG. 1, the office PC 40 starts reading the data recorded from the attached memory card 32, that is, the operation information of the vehicle on which the vehicle-mounted device 10 is mounted and the digital image data of each frame. Further, since this recording data is data arranged in time series in the order of recording, it is read in order from the data with the earlier recorded time.

  In step S12, the office PC 40 identifies whether the data at each time read from the memory card 32 is data recorded while traveling on the highway. As described above, when the driver operates the “high speed” button, the “dedicated” button, the “general” button or the like while the on-vehicle device 10 is recording data, When the passage of the entrance gate or the passage of the exit gate is detected, information indicating the type of road at that time is written as a part of the recording data of the memory card 32. Therefore, the office PC 40 extracts specific data representing the type of road from the read data, and specifies the type of road at each time point. If the recording data is traveling on a highway (which may be a dedicated road), the process proceeds to S13. If the recording data is traveling on a general road, the process proceeds to S16.

  Further, when the operation information recorded in the memory card 32 includes information on the current position of the vehicle detected by the GPS receiver 33 or the like, road map data accessible by the office PC 40 is preliminarily stored. If the vehicle position is prepared, the vehicle position in the recorded operation information is matched with each position on the map, and the type of the road on which the vehicle is traveling can be identified. Therefore, even if information indicating the road type is not recorded in the memory card 32, the road type can be identified in S12.

  In step S13, a microcomputer or an image recognition unit (not shown) in the office PC 40 executes predetermined image recognition on the digital image data of one frame that is currently referred to. Thus, when processing data such as the recorded image frame 50 shown in FIG. 4, the preceding vehicle 52, the lane left lane marking 53, the lane right lane marking 54, the horizon 55, and the like are based on known techniques. Each is automatically recognized.

  In step S14, as a result of the image recognition in S13, it is determined whether or not the digital image data of the currently processed frame has been successfully recognized for the lane left lane line 53, the lane right lane line 54, and the horizon 55. PC 40 identifies. If the three recognitions of the lane left lane line 53, the lane right lane line 54, and the horizon line 55 are successful, the process proceeds to S15, and if not, the process proceeds to S16.

  In step S15, the office PC 40 calculates FOE coordinates based on the contents of the image frame recognized in step S13. Specifically, a point at which the extension line of the lane left division line 53, the extension line of the lane right division line 54, and the horizon 55 intersect is detected as FOE. In the state of the recorded image frame 50 shown in FIG. 4, the extension line of the left lane marking line 53, the extension line of the lane right marking line 54, and the horizon 55 intersect at one point, which is the FOE. Since it almost coincides with the position, the position coordinates of the intersected point are specified as the FOE of the frame being processed.

  However, since the FOE detected based on only one frame image is likely to contain an error, a plurality of FOEs detected for a plurality of frame images are aggregated in S15 and averaged. Calculate the average coordinates of the results. Then, the master data of the calculated FOE average coordinates is registered in the database DB1 in S15.

  That is, the FOE position for each image frame detected in S15 varies, but by calculating the average FOE position for a number of frames, for example, an appropriate FOE position is specified as shown in FIG. it can.

  On the other hand, when an erroneous recognition occurs in S13 or depending on the situation of the road or the like reflected in the recorded image frame 50, an extension line of the lane left lane line 53, an extension line of the lane right lane line 54, and a horizon 55 There is a case where the points do not intersect at one point, and if a FOE is calculated forcibly, a large error may occur. For example, when the white line marked on the road is faint and difficult to recognize, or when it is affected by uphill / downhill slopes, a large error occurs. Therefore, when the recognized extension line of the left lane marking line 53, the extension line of the lane right lane marking line 54, and the horizon 55 do not intersect at one point, this frame is excluded from the FOE calculation target. Process in S15. Thereby, the increase in the error resulting from the road condition can be suppressed.

  When the process shown in FIG. 1 is periodically executed, master data of FOE average coordinates calculated previously for the same vehicle may already be registered in the database DB1. In this state, when new digital image data is read from the memory card 32, the averaging process is performed in S15 using the master data on the database DB1 and the FOE coordinates or FOE average coordinates calculated for the current image frame. And register the result in the database DB1. That is, every time new data is read from the memory card 32, the master data on the database DB1 is updated sequentially.

  The processing of S12 to S16 is repeated for all the image frames recorded on the memory card 32, and the contents of each image frame are reflected in the master data on the database DB1. When the processing of all the image frames recorded on the memory card 32 is completed, the process proceeds from S16 to S17.

  At the time of executing step S17, all data recorded on the memory card 32 has been processed and is unnecessary, so the office PC 40 clears the contents of the memory card 32 in S17. In the next step S18, the office PC 40 downloads the contents of the master data registered in the database DB1 and writes them in the memory card 32. That is, data representing the latest FOE coordinates obtained by the current processing of FIG. 1 is newly registered in the memory card 32.

Next, the operation after determining the FOE coordinates will be described.
The memory card 32 in which the latest FOE coordinates are written in step S18 in FIG. 1 is removed from the office PC 40, and then when a specific crew member driving the corresponding vehicle starts the on-board device on the corresponding vehicle. 10 is attached. The vehicle-mounted device 10 reads the contents of the attached memory card 32 when the ignition of the vehicle is turned on. At this time, since the data representing the latest FOE coordinates is written in the memory card 32, the microcomputer 11 of the vehicle-mounted device 10 proceeds from step S31 to step S32 in FIG. The data is reflected in the subsequent control.

  For example, when the white line marked on the road is faint, it is difficult to recognize the white line as a pattern. However, when the position of the FOE is known, the direction in which the white line is directed on the image frame can be easily specified, so that there is a high possibility that the white line can be correctly recognized even when there is a partial blur.

  As described above, by adopting the video reference point determination method corresponding to the processing procedure shown in FIG. 1, it is not necessary to perform the calibration of the FOE position while the vehicle is stopped. Thus, there is no need to prepare a special work space or a target.

  Moreover, since the position of FOE is determined based on the image | video of the state which the vehicle is actually drive | working, the more suitable position of FOE can be pinpointed in the same state as an actual driving environment. For example, if the vehicle height or the inclination of the vehicle body changes according to the load state of the load, the FOE position will deviate compared to the case of an empty vehicle, but the position of the FOE is specified based on the image of the load state. By doing so, errors can be reduced.

  In addition, since the position of the FOE is calculated by extracting only data that satisfies special conditions such as when driving on an expressway, the position accuracy can be improved. For example, an expressway has less curved places and road surface irregularities than ordinary roads, so an image of an environment suitable for calculating the FOE position can be obtained, and calculation errors are unlikely to occur. Further, since the amount of data to be processed is reduced, the time required for executing the processing of FIG. 1 can be shortened.

  Note that the process of extracting only the data satisfying the special condition may be performed when analyzing the data read by the office PC 40 as shown in FIG. 1, or the vehicle-mounted device 10 stores the digital image data in the memory card 32. It may be done when recording. For example, a road suitable for calculation of FOE if an instruction to start or stop recording in the vehicle-mounted device 10 is generated based on the driver's manual switch operation, the detection state of the ETC vehicle-mounted device, the detected current position of the vehicle, etc. Digital image data can be recorded on the memory card 32 only when traveling in the section.

  Note that the processing procedure shown in FIG. 1 can be executed by the microcomputer 11 on the vehicle-mounted device 10. In the processing procedure shown in FIG. 1, the position coordinates of a plurality of FOEs are averaged in step S15, but statistical processing other than this may be performed.

Here, the features of the embodiments of the video reference point determination system, the video reference point determination method, and the video reference point determination program according to the present invention described above are briefly summarized and listed in the following [1] to [6], respectively.
[1] In order to determine a vehicle-mounted camera (high-quality vehicle-mounted camera 31) installed on the vehicle and a specific reference point (FOE) on the image used when recognizing the image captured by the vehicle-mounted camera. A video reference point determination system comprising: a management device (office PC40);
The management device reads recorded image data of a video obtained by photographing with the in-vehicle camera while the vehicle is running, and extracts processing target recorded image data satisfying a predetermined condition from the recorded image data Extracting unit (CPU 11, image processing unit 12, S11, S12),
A predetermined image recognition process is performed on the processing target recorded image data, and the first line on the left side of the vehicle appearing in the direction along the road on the road on which the vehicle is traveling (lane left side partition line 53). ), And a detection unit (CPU 11, image processing unit 12, S 13) that detects a second line (lane right side dividing line 54) on the right side of the vehicle and a third line (horizon line 55) corresponding to the horizon. ,
A video reference point determination system, comprising: a reference point determination unit (CPU 11, S15) that determines the reference point based on an intersection of the first line, the second line, and the third line.
[2] Video reference point determination for determining a specific reference point (FOE) on the video used when recognizing a video shot by a vehicle-mounted camera (high-quality vehicle-mounted camera 31) installed on the vehicle A method,
The recorded image data of the video actually captured by the in-vehicle camera while the vehicle is running is read (S11), and the processing target recorded image data satisfying a predetermined condition in the recorded image data is read. Extract (S12),
A predetermined image recognition process is performed on the processing target recorded image data (S13), and the first line on the left side of the vehicle appearing in the direction along the road on the road on which the vehicle is traveling (the left side of the lane) A lane line 53), a second line on the right side of the vehicle (lane right side lane line 54), and a third line (horizon line 55) corresponding to the horizon,
Determining the reference point based on an intersection of the first line, the second line, and the third line (S15);
A video reference point determination method characterized by the above.

[3] The video reference point determination method according to [2],
As the processing target recording image data, data of a section in which the vehicle is traveling on an expressway or an automobile exclusive road is extracted (S12),
A video reference point determination method characterized by the above.

[4] The video reference point determination method according to [2],
The reference point is detected for each of the plurality of recording sections of the processing target recording image data, and the result of performing statistical processing by collecting the information of the plurality of reference points is output as a definite value of the reference point (S15). ),
A video reference point determination method characterized by the above.

[5] The video reference point determination method according to [2],
When new data is added as the recorded image data, the determined value is updated using the new reference point detected based on the added data (S15),
A video reference point determination method characterized by the above.

[6] A video reference point determination program that causes a computer to execute the procedure of the video reference point determination method according to any one of [2] to [5].

DESCRIPTION OF SYMBOLS 10 Onboard equipment 11 Microcomputer 12 Image processing part 13 G sensor 14 Speaker 15 GPS interface 16 Speed interface 17, 20 Flash memory 18 Card interface 19 Data processing part 21 Memory for temporary storage 22 Power supply interface 31 High quality vehicle-mounted camera 32 Memory card 33 GPS receiver 40 Office PC
50, 50A Recorded image frame 51 Traveling lane of own vehicle 52 Leading vehicle 53 Lane left side marking line 54 Lane right side marking line 55 Horizon line 56 Own vehicle DB1 database

Claims (6)

Video reference point determination system comprising: an in-vehicle camera installed on a vehicle; and a management device for determining a specific reference point on the video used when recognizing an image captured by the in-vehicle camera Because
The management device reads recorded image data of a video obtained by photographing with the in-vehicle camera while the vehicle is running, and extracts processing target recorded image data satisfying a predetermined condition from the recorded image data An extractor to perform,
For each of a plurality of recording sections included in the processing target record image data, performs predetermined image recognition processing, the left than the vehicle appearing in a direction along the road on the road on which the vehicle is traveling A first line, a second line on the right side of the vehicle, and a third line corresponding to the horizon, respectively.
Among the plurality of recording sections, the data of the recording section in which the three lines of the first line, the second line, and the third line do not intersect at a common point is excluded from the calculation target, and the three lines A reference point determination unit that calculates an intersection of the three lines as a reference point for each of the recording sections that intersect at a common point, and determines a definite value of the reference point based on the plurality of calculated reference points; And a video reference point determination system. A video reference point determination method for determining a specific reference point on the video used when recognizing a video shot by an in-vehicle camera installed on a vehicle,
Read the recorded image data of the video obtained by photographing with the in-vehicle camera during the traveling of the vehicle, extract the processing target recorded image data satisfying a predetermined condition in the recorded image data,
A predetermined image recognition process is performed on each of the plurality of recording sections included in the processing target recording image data, and the left side of the vehicle appearing in the direction along the road on the road on which the vehicle is traveling Detecting the first line, the second line on the right side of the vehicle, and the third line corresponding to the horizon,
Among the plurality of recording sections, the data of the recording section in which the three lines of the first line, the second line, and the third line do not intersect at a common point is excluded from the calculation target, and the three lines For each of the recording sections intersecting at a common point, the intersection of the three lines is calculated as a reference point, and a fixed value of the reference point is determined based on the plurality of calculated reference points .
A video reference point determination method characterized by the above. As the processing target recording image data, extract data of a section in which the vehicle is traveling on an expressway or an automobile exclusive road,
The video reference point determination method according to claim 2, wherein: The result of performing statistical processing by collecting information on the plurality of calculated reference points is output as a fixed value of the reference points.
The video reference point determination method according to claim 2, wherein: When new data is added as the recorded image data, the determined value is updated using the new reference point detected based on the added data.
The video reference point determination method according to claim 4, wherein: A video reference point determination program that causes a computer to execute the procedure of the video reference point determination method according to any one of claims 2 to 5.

Images