JP5849724B2 - Video acquisition apparatus and method - Google Patents

Description

  The disclosed technology relates to a video acquisition device and a video acquisition method.

Conventionally, each acquires each frame of the plurality of video signals input in parallel are respectively captured by the plurality of the imaging unit, predetermined image processing for each frame acquired, for example, Ya measurement of distance to the object A technique for performing processing such as generation of a video obtained by combining a plurality of videos is known. In this technique, the imaging by a plurality of imaging units can be synchronized with each other, but in this case, the configuration becomes complicated and the cost of each imaging unit increases. For this reason, imaging by a plurality of imaging units is performed asynchronously, each frame of a plurality of video signals input asynchronously is sequentially acquired, and predetermined image processing is performed when one frame is acquired for each of the plurality of video signals. In many cases, a configuration that repeats the above is adopted. Thereby, the configuration is simplified and the cost of the imaging unit is also suppressed.

  In addition, in relation to the above, in order to detect time lag more than the frame time, the video timing of the moving image group captured asynchronously from multiple cameras into the memory is shifted from the amount of luminance change between videos. A technique for detecting the occurrence of this is also known.

JP 2003-52056 A

  However, when a plurality of video signals are input asynchronously as described above, at the timing when the corresponding frames of the plurality of video signals used for the same image processing are captured and acquired, one cycle of the frame rate at the maximum. There is a problem that a shift of minutes occurs. Then, the difference in imaging / acquisition timing of corresponding frames of a plurality of video signals appears as an error in measurement distance when distance measurement is performed as predetermined image processing, for example. In addition, when a plurality of videos are synthesized as predetermined image processing, each frame of the synthesized video becomes an image obtained by synthesizing frames with different imaging timings, thereby reducing the accuracy of the synthesized video. Become.

  Hereinafter, taking as an example the case where distance measurement is performed as predetermined image processing, the magnitude of the error that the difference in imaging / acquisition timing of the corresponding frames of a plurality of video signals has on the distance measurement result will be described with a specific example. . As shown in FIG. 21, video signals output from two imaging units 300 and 302 that are mounted on a moving body such as a vehicle and operate asynchronously are used to detect an object 304 that is captured as a subject in the video. Consider the case of performing stereo ranging to measure distance.

Stereo distance measurement is shown as a subject in two images photographed by the imaging units 300 and 302 on the assumption that the subject is photographed simultaneously by two imaging units 300 and 302 installed at intervals. It measures the distance to an object by the principle of triangulation. In order to simplify the description, in FIG. 21, the imaging units 300 and 302 are arranged so that the optical axes are orthogonal to the base line connecting the imaging units 300 and 302 (parallel stereo). Shall move in a direction parallel to the baseline. The focal length of the imaging units 300 and 302 is f, the base length of the imaging units 300 and 302 is b, and the projection position of the object 304 on the corresponding frame of the two video signals output from the imaging units 300 and 302 is shown. Let the difference (parallax) be d (= x ₁ −x ₂ ). In this case, the depth distance Z to the object 304 is given by the following equation (1) (see also FIG. 21A).
Z = f × b / d (1)

  Here, the imaging / acquisition timing of the corresponding frame of the two video signals output from the two imaging units 300 and 302 corresponds to one frame rate (33 ms when the frame rate is 30 frames / second). Consider the case where there is a gap. Specifically, for example, a case where a frame corresponding to a video signal output from the imaging unit 302 is captured with a delay of one cycle of the frame rate with respect to a certain frame of the video signal output from the imaging unit 300 is considered. .

In this case, as shown in FIG. 21B, after the imaging unit 300 captures an image of a certain frame, the imaging unit 302 moves by a distance corresponding to one cycle of the frame rate along the moving direction of the moving object. After the movement, the corresponding frame is imaged by the imaging unit 302. Assuming that the moving amount of the moving body during a period of one cycle of the frame rate is Δb, the imaging unit 302 captures an image at a position separated from the imaging unit 300 by b ′ that is Δb longer than the baseline length b. Will be going. Accordingly, the projection position of the object 304 on the corresponding frame imaged by the imaging unit 302 is x ₂ ′, and the parallax used for the calculation is also d ′ (= x ₁ −x ₂ ′). For this reason, the accurate depth distance Z is
Z = f × b ′ / d ′ (= f × (b + Δb) / d ′, d ′ = x ₁ −x ₂ ′)
It becomes.

However, since the stereo distance measurement does not normally take into account the shift in the imaging timing of the corresponding frame, the depth distance Z _NG represented by the following equation (2) is calculated as the depth distance Z.
Z _NG = f × b / d ′ (2)
Therefore, a distance measurement error ΔZ (= Z _NG −Z) is added to the measurement result of the depth distance Z. The error rate ΔZ / Z is obtained as follows.
ΔZ / Z = (Z−Z _NG ) / Z
= (Z−f × b / d ′) / Z
= (Z−b / (b + Δb) × f × b / d) / Z
= 1−b / (b + Δb)
= Δb / (b + Δb)

  Since Δb is the moving amount of the moving body during the time of one period of the frame rate, for example, at 20 km / h where the moving speed of the moving body is relatively low, one period of the frame rate is 33 ms (30 frames). / B), Δb = about 20 cm. The base line length b is about 50 cm at the longest when the moving body is a vehicle. Therefore, the error rate ΔZ / Z for distance measurement is about 20 / (50 + 20) = 0.3, and a distance measurement error of about 30% occurs even if the time difference is one period of the frame rate.

  Note that, as described above, the technique for detecting the occurrence of a time shift equal to or longer than the frame time from the amount of luminance change between videos cannot reduce the shift in imaging / acquisition timing of corresponding frames of a plurality of videos.

  An object of the disclosed technique is to reduce a shift in acquisition timing of a corresponding frame of each of a plurality of video signals input asynchronously.

In the disclosed technique, each frame includes an odd field and an even field, and a plurality of video signals having the same frame rate are input asynchronously. The notifying unit notifies at least one of an input start timing and an input end timing of the video signal in units of fields for the plurality of video signals. The determination unit determines the frame structure of each corresponding frame of the plurality of video signals based on the timing notified to each of the plurality of video signals by the notifying unit, and the corresponding frame of each of the plurality of video signals. The order of acquisition is determined in units of video signals. Note that the determination by the determination unit is performed so that the shift in the acquisition timing of the corresponding frame of each of the plurality of video signals is less than ½ of the period of the frame rate. In the determination of the frame structure, it is determined whether the first frame structure is combined in the order of odd fields and even fields, or the second frame structure is combined in the order of even fields and odd fields. The output unit outputs a frame corresponding to each of the plurality of video signals acquired according to the acquisition order in units of the frame structure and the video signal determined by the determination unit.

Further, according to the disclosed technique, at least one of the input start timing and the input end timing of the video signal in units of frames is notified by the notification unit for a plurality of video signals input at the same frame rate and asynchronously. . The determination unit determines an acquisition order in units of video signals of corresponding frames of the plurality of video signals based on timings notified by the notification unit for the plurality of video signals. The determination by the determination unit, the corresponding shift of the acquisition timing of the frame of each of the plurality of video signals is carried out such that less than one round-life of the frame rate. Then, the output unit outputs a corresponding frame of each of the plurality of video signals acquired according to the acquisition order in units of the video signals determined by the determination unit.

  The disclosed technique has an effect that a shift in acquisition timing of a corresponding frame of each of a plurality of video signals input asynchronously can be reduced.

It is the schematic which shows the structure of the video processing apparatus which concerns on 1st Embodiment. It is a functional block diagram of the video acquisition device concerning a 1st embodiment. It is an image figure which shows the write-in area | region of each field on a physical memory. It is a chart which shows the contents of the buffer circulation table concerning a 1st embodiment. It is a chart which shows the contents of the logical buffer physical memory correspondence table concerning a 1st embodiment. It is a timing chart for demonstrating the principle which optimizes an acquisition order and a frame structure. It is a flowchart which shows the timing determination process by the timing determination part of the video acquisition apparatus which concerns on 1st Embodiment. It is a flowchart which shows the flow of the capture process by the capture process part of the video acquisition apparatus which concerns on 1st Embodiment. It is a timing chart for demonstrating the process which each writes the data of each field of an interlace video signal in two buffer regions for writing selected cyclically. It is a timing chart for demonstrating the output of the synchronous acquisition completion notification by the completion notification part of the video acquisition device concerning 1st Embodiment. It is a flowchart which shows the flow of the video output process by the capture process part of the video acquisition apparatus which concerns on 1st Embodiment. 6 is a timing chart for explaining replacement of a standby buffer area and an output buffer area when a video acquisition request notification is received from the image processing apparatus. It is a timing chart for demonstrating the time gap of the corresponding flame | frame output from each capture process part. It is a functional block diagram of the image | video acquisition apparatus which concerns on 2nd Embodiment. It is a chart which shows the contents of the buffer circulation table concerning a 2nd embodiment. It is a chart which shows the contents of the logical buffer physical memory correspondence table concerning a 2nd embodiment. It is a flowchart which shows the timing determination process by the timing determination part of the image | video acquisition apparatus which concerns on 2nd Embodiment. It is a flowchart which shows the flow of the capture process by the capture process part of the video acquisition apparatus which concerns on 2nd Embodiment. In the second embodiment, it is a timing chart for explaining the time lag of the corresponding frame output from each capture processing unit. It is the schematic which shows the structure of the video processing apparatus which concerns on 3rd Embodiment. FIG. 10 is a schematic diagram for explaining the influence of a shift in imaging / acquisition timing of frames corresponding to a plurality of videos on stereo distance measurement in the conventional technology.

  Hereinafter, an example of an embodiment of the disclosed technology will be described in detail with reference to the drawings.

[First Embodiment]
FIG. 1 shows a video processing apparatus 10 according to the first embodiment. The video processing apparatus 10 includes a video acquisition apparatus 12 that acquires video signals output from n (n ≧ 2) imaging units 28, an image processing apparatus 14, a main memory 16, and a connection between them. Main bus 18. Note that the n imaging units 28 are provided at different positions of the moving body (for example, a vehicle) in directions to image the surroundings of the moving body, and operate asynchronously with each other. The n imaging units 28 output interlaced video signals (hereinafter referred to as interlaced video signals) each having the same frame rate and each frame having an odd field and an even field. .

  The video acquisition device 12 includes a buffer memory 20, a bus I / F (interface) 22, a video input control unit 24 connected to each of n imaging units 28, and a local bus for connecting these components to each other. 26. The buffer memory 20 is a memory for storing video signals input from the n imaging units 28 as data. The bus I / F 22 is provided between the main bus 18 and the local bus 26 and manages communication between the video acquisition device 12 and the image processing device 14 via the main bus 18. The video input control unit 24 acquires video signals from the n imaging units 28 and stores the data of each video signal in the buffer memory 20. In response to a request from the image processing device 14, the video input control unit 24 reads the data of each video signal stored in the buffer memory 20 in units of corresponding frames and transmits the data to the image processing device 14.

  The image processing device 14 receives data of each video signal from the video acquisition device 12 in units of corresponding frames through the local bus 26, bus I / F 22, and main bus 18, and stores the received data in the main memory 16. To do. Based on the data of each video signal stored in the main memory 16, the image processing device 14 combines, for example, stereo distance measurement processing or video represented by a plurality of video signals using the data of the corresponding frame of each video signal as a unit. Perform image processing such as video generation.

  The video acquisition device 12 is realized by a plurality of electronic circuits, for example. As illustrated in FIG. 2, the video acquisition device 12 includes n capture processing units 30, a timing determination unit 32, and a completion notification unit 34, each provided corresponding to n imaging units 28. The timing determination unit 32 is an example of a determination unit in the disclosed technology, and the completion notification unit 34 is an example of a completion notification unit in the disclosed technology.

  The n capture processing units 30 capture each interlaced video signal output asynchronously from the corresponding imaging unit 28. In the present embodiment, the storage area of the buffer memory 20 is divided into n storage areas (buffers) corresponding to different capture processing units 30 among the n capture processing units 30. Each capture processing unit 30 stores the captured interlaced video signal data in a corresponding buffer among n buffers provided in the buffer memory 20. The n buffers are provided with four buffer areas each capable of storing data for one frame of the interlaced video signal. The data of the interlaced video signal is written in the four buffer areas. Each of them is stored in two buffer areas set for inclusion (details will be described later).

  Each capture processing unit 30 detects the input start timing for each field of the captured interlaced video signal, detects whether the input start field is an odd field or an even field, and timings the detection result. Each is notified to the determination unit 32. Each capture processing unit 30 stores data for one frame of a frame structure (described later) notified from the timing determination unit 32 in the corresponding buffer of the buffer memory 20 as interlaced video signal data. The acquisition completion notification is output to the completion notification unit 34. Furthermore, when receiving the video acquisition request notification from the image processing device 14, each capture processing unit 30 reads the data of the video signal for one frame from the corresponding buffer and transmits it to the image processing device 14.

  When the start of capturing video signals is instructed, the timing determination unit 32 obtains n detection results of input start timing and input field type (odd / even field) for each field of n interlaced video signals. Each is acquired from the capture processing unit 30. In addition, the timing determination unit 32 is configured so that, based on the detection result acquired from the n capture processing units 30, the shift in the acquisition timing of the corresponding frame of the n interlaced video signals is minimized. The acquisition order and frame structure for each video signal are determined. The frame structure includes a first frame structure in which the frames are combined in the order of odd fields and even fields, and a second frame structure in which the frames are combined in the order of even fields and odd fields. Is determined. When the timing determination unit 32 determines that each of the n capture processing units 30 has the first frame structure according to the determined frame structure, the timing determination unit 32 sets “even-number completion notification setting” to the second frame structure. If it is determined, “Odd completion notification setting” is output. In addition, the timing determination unit 32 notifies the completion notification unit 34 of the determination result of the acquisition order for each video signal.

  The completion notification unit 34 monitors input of acquisition completion notifications from the individual capture processing units 30. Then, the completion notification unit 34 notifies the image processing device 14 of the synchronous acquisition completion every time an acquisition completion notification corresponding to the last video signal in the acquisition order for each video signal notified from the timing determination unit 32 is input. Send.

  Each of the n capture processing units 30 includes an input unit 36, an odd / even notification unit 38, a buffer management unit 40, a write processing unit 42, and a video output unit 44. The odd / even notification unit 38 is an example of a notification unit in the disclosed technology, the buffer management unit 40 is an example of a buffer management unit in the disclosed technology, and the write processing unit 42 is an example of the write processing unit in the disclosed technology. The video output unit 44 is an example, and is an example of an output unit in the disclosed technology.

  The input unit 36 acquires the interlaced video signal output from the corresponding imaging unit 28, and converts the acquired interlaced video signal into a digital signal if the acquired interlaced video signal is an analog signal such as NTSC, and then outputs it to the odd / even notification unit 38. To do.

  The odd / even notification unit 38 detects the input start timing for each field of the interlaced video signal input from the input unit 36, and detects whether the input start field is an odd field or an even field. The detection result is notified to the buffer management unit 40 (the buffer switching unit 50), the write processing unit 42, and the timing determination unit 32, respectively. In addition, the interlaced video signal input from the input unit 36 is output to the write processing unit 42.

  The write processing unit 42 manages the interlaced video signal data input from the odd / even notification unit 38 in two buffer areas set for writing by the buffer management unit 40 in units of one field. Each writing process is performed through the unit 40. In the physical memory corresponding to each buffer area, for example, as shown in FIG. 3A, an odd field write area and an even field write area may be provided in order in a field unit. For example, as shown in FIG. 3B, odd-numbered field write areas and even-numbered field write areas may be alternately provided for each line. The write processing unit 42 writes the data to the odd field writing area when the data to be written is an odd field, and writes the data to the even field writing area when the data to be written is an even field. .

  Further, when “even number completion notification setting” is input from the timing determination unit 32, the write processing unit 42 notifies the completion at the timing when the writing of the data of the even field is completed in the buffer area for writing. An acquisition completion notification is output to the unit 34. On the other hand, when the “odd completion notification setting” is input from the timing determination unit 32, the write processing unit 42 notifies the completion at the timing when the writing of the data in the odd field is completed in the write buffer area. An acquisition completion notification is output to the unit 34.

  When receiving the acquisition request from the image processing apparatus 14, the video output unit 44 outputs an output buffer setting request for requesting the buffer management unit 40 to set a buffer area for output. When the buffer management unit 40 sets an output buffer area in accordance with the output buffer setting request, the video output unit 44 performs buffer management of interlaced video signal data for one frame from the buffer area set for output. Read through unit 40. Then, the video output unit 44 transmits the read interlace video signal data for one frame to the image processing device 14.

  The buffer management unit 40 sets a use for writing / output / standby for four buffer areas provided in a buffer corresponding to a single capture processing unit 30 and The setting is appropriately changed and the logical address of each buffer area is managed. The buffer management unit 40 includes a buffer circulation table storage unit 46, a buffer number calculation unit 48, a buffer switching unit 50, a logical buffer physical memory correspondence table storage unit 52, and a physical memory number calculation unit 54.

  The buffer circulation table storage unit 46 stores a buffer circulation table as shown in FIG. 4 as an example. The buffer circulation table is a table in which information defining the usage of each of the four buffer areas is registered when the state number of the buffer area is each value (1 to 3 in the first embodiment). As shown in FIG. 4, in the first embodiment, two of the four buffer areas are set for writing, and one of the remaining two buffer areas is for standby. The other is set for output. “Buffer 1” to “Buffer 4” set in the buffer circulation table are logical addresses assigned to the four buffer areas, respectively. The buffer area with the logical address {buffer 4} is always set for output regardless of the transition of the state number of the buffer area, and the buffer areas with the logical addresses {buffer 1} to {buffer 3} are in the state of the buffer area. It is set for writing or standby according to the transition of the number.

  The buffer number calculation unit 48 holds the current state number of the buffer area, and every time the buffer switching unit 50 instructs the transition of the state number of the buffer area, the held state number is 1 → 2 → 3. Change the state number so that it cycles through → 1 →. When the write processing unit 42 is instructed to write data to the write buffer area, the buffer number calculation unit 48 refers to the buffer circulation table and writes the state number of the held buffer area. Recognize the logical addresses of the two buffer areas set for insertion. Then, the logical addresses of the two recognized buffer areas are output to the physical memory number calculation unit 54.

  Further, when the output buffer setting is requested from the video output unit 44, the buffer number calculation unit 48 refers to the buffer circulation table and is set for standby or output by the state number of the held buffer area. The logical addresses of the two buffer areas are recognized. Then, the logical addresses of the two recognized buffer areas are output to the physical memory number calculation unit 54, and the replacement of each buffer area is instructed. Further, when the video output unit 44 instructs the buffer number calculation unit 48 to read data from the output buffer, the buffer number calculation unit 48 sets the logical address {buffer 4} of the buffer area set for output to the physical memory number calculation unit. 54 to instruct the reading of data.

  Whenever the detection result of whether the field of the interlaced video signal whose input is started from the input unit 36 is an odd field or an even field is input from the odd / even notification unit 38, the buffer switching unit 50 To instruct the transition of the buffer area state number.

  As an example, the logical buffer physical memory correspondence table storage unit 52 stores a logical buffer physical memory correspondence table as shown in each row of FIG. The logical buffer physical memory correspondence table is a table in which the logical addresses {buffer 1} to {buffer 4} assigned to the four buffer areas are associated with the physical addresses of the four buffer areas.

  When the logical addresses of the two buffer areas for writing are input from the buffer number calculating section 48, the physical memory number calculating section 54 refers to the logical buffer physical memory correspondence table and reads the two buffers for writing. Recognize the physical address of the region. Then, the write target data input from the write processing unit 42 is written in each of the two buffer areas whose physical addresses are recognized. When the physical address of the two buffer areas is input from the buffer number calculator 48 and the replacement is instructed, the physical memory number calculator 54 replaces the logical addresses assigned to the two buffer areas. The logical buffer physical memory correspondence table is updated as follows. Furthermore, when the logical address of the buffer area for output is input from the buffer number calculation unit 48 and the reading of data is instructed, the physical memory number calculation unit 54 refers to the logical buffer physical memory correspondence table and outputs Recognizes the physical address of the buffer area. Then, one frame of data is read from the output buffer area whose physical address is recognized, and the read data is output to the video output unit 44.

  Next, the operation of the first embodiment will be described. When a plurality of video signals are input asynchronously, as described above, there is a maximum shift of one cycle of the frame rate in the imaging / acquisition timing of the corresponding frames of the plurality of video signals used for the same image processing. Arise. On the other hand, in this embodiment, the interlaced video signal input from each of the n imaging units 28 is a video in which each frame has an odd field and an even field, and is usually synthesized in the order of odd field → even field. Thus, an image for one frame is formed.

Here, even when images of one frame are formed by combining odd-numbered fields → even-numbered fields from a plurality of video signals input asynchronously, the acquisition order of the plurality of video signals can be changed and set. There is a possibility that the shift of the corresponding frame can be reduced. That is, in the example of FIG. 6, the video signal acquisition order is as follows: video signal 3 → video signal 4 → video signal 1 → video signal 2 as indicated by “minimum time shift in frame units”. The time lag of the imaging (input) timing of the corresponding frame of the video signal is minimized. In this case, the time lag of the corresponding frames of the plurality of video signals is less than ± 1/2 of one cycle of the frame rate at the maximum (= 1 field time, 16.7 ms if 30 frames / second) , that is, the frame The rate can be shortened to less than one period .

  In addition, the two fields forming an image for one frame are not limited to the odd field being input first, and the even field having the two fields input first is equivalent to one frame. Can be formed. That is, even if the even field of the i-th frame input at time t and the odd field of the i + 1-th frame input at time t + α (α is a half of the frame rate period) are combined, one frame will be combined. Can be formed. For this reason, the acquisition order of a plurality of video signals is changed and set, and the boundary for forming an image for one frame from each video signal is selectively set from the boundary of the field unit of each video signal. Then, the shift of the corresponding frames of the plurality of video signals is further reduced.

That is, in the example of FIG. 6, the video signal acquisition order is set in the order of video signal 2 → 3 → 4 → 1 and the image for one frame is displayed as “minimum time shift in field units”. The video signals 1, 3, and 4 are acquired in the order of odd-to-even, and the video signal 2 is acquired in the order of even-to-odd. In addition, the acquisition order of the video signal is set in the order of the video signal 2 → 3 → 4 → 1, and the image for one frame is displayed in the order of the video signal 1, 3, 4 even → odd, and the video signal 2 is odd → even. You may acquire in order. As a result, the time lag of the imaging (input) timing of the corresponding frame of the video signal is further reduced, and at most ± 1/4 of one cycle of the frame rate (= 1/2 of one field time, 30 frames / It can be shortened to less than 8.4 ms) , that is, less than ½ of one cycle of the frame rate . Note that acquiring an image for one frame in the order of odd to even field is an example of changing the frame structure of the image to the first frame structure in the disclosed technology. Acquiring an image for one frame in the order of even to odd fields is an example of changing the frame structure of the image to the second frame structure in the disclosed technique.

  In the first embodiment, the timing determination unit 32 performs the timing determination process described below so that the time lag of the corresponding frame of each video signal is minimized, The frame structure of the corresponding frame of the video signal is optimized.

  That is, when the user operates the operation unit (not shown) to start the operation of the video processing apparatus 10, the n imaging units 28 start imaging, and the n imaging units 28 capture the captured video. Is output asynchronously. Also, a video capture start notification is given from the image processing device 14 to the video acquisition device 12, and the timing determination process shown in FIG. 7 is executed by the timing determination unit 32 of the video acquisition device 12 using this video capture start notification as a trigger. The

  In step 100 of the timing determination process, the timing determination unit 32 sets a variable dt_min that represents the minimum value of the time lag of each corresponding frame of n interlaced video signals input asynchronously from the n imaging units 28. Set a large value as the initial value. In step 102, the timing determination unit 32 instructs the n capture processing units 30 to start capturing interlaced video signals. Thereby, the detection result of the input start timing of each field of the interlaced video signal and the determination result of the odd / even of each field are repeatedly output from each of the n capture processing units 30.

  In the next step 104, the timing determination unit 32 sets an odd field as a determination target field for the interlaced video signal as the head (reference). In step 106, the timing determination unit 32 sets 1 as an initial value to a variable i for identifying the interlaced video signal that is the head (reference).

  In step 108, the timing determination unit 32 determines whether an odd field is set as a determination target field for the interlaced video signal that is the head (reference). If an odd field is set as the determination target field for the first (reference) interlaced video signal, the determination in step 108 is affirmed and the process proceeds to step 110.

  In step 110, the timing determination unit 32 waits until the input start timing of the odd field of the interlaced video signal Ai is detected by the capture processing unit 30 that takes in the interlaced video signal Ai as the head (reference). When the input start timing of the odd field of the interlaced video signal Ai is detected, the process proceeds from step 110 to step 116, and the timing determination unit 32 resets the timer to zero.

  On the other hand, if an even field is set as the determination target field for the interlaced video signal Ai as the head (reference), the determination in step 108 is negative and the process proceeds to step 112. In step 112, the timing determination unit 32 waits until the input processing timing 30 for capturing the interlace video signal Ai as the head (reference) detects the input start timing of the even field of the interlace video signal Ai. When the input start timing of the even field of the interlaced video signal Ai is detected, the process proceeds from step 112 to step 114, and the timing determination unit 32 resets the timer to zero.

  In the next step 118, the timing determination unit 32 detects the input start timing of the first field of the remaining interlaced video signal by each of the n-1 capture processing units 30 that capture the remaining interlaced video signal. Wait until When the input start timing of the first field is detected for each of the remaining n-1 interlaced video signals, the process proceeds from step 118 to step 120, and the timing determination unit 32 sets the time from the timer as the shift time dt. get.

  In the next step 122, the timing determination unit 32 determines the interlace in which the input start timing of the field is last detected based on the order in which the input start timing of the first field is detected by the n−1 capture processing units 30. The number S of the video signal is acquired. In addition, the timing determination unit 32 acquires an odd / even list flist indicating whether the first field of each of the n interlaced video signals including the interlaced video signal Ai as the head (reference) is an odd field or an even field.

  In the next step 124, the timing determination unit 32 determines whether or not the deviation time dt acquired in step 120 is smaller than the variable dt_min. If the shift time dt is equal to or greater than the variable dt_min, the determination in step 124 is negative and the process proceeds to step 128 described later. On the other hand, when the shift time dt is smaller than the variable dt_min, the timing determination unit 32 determines that the minimum value of the shift time has been updated (a new minimum value of the shift time has been obtained), and proceeds to step 126. .

  In step 126, the timing determination unit 32 sets the newly obtained minimum value of the shift time to the variable dt_min, and a variable i_min that represents the number of the first interlace video signal when the minimum value of the shift time is obtained. To the current value of the variable i. In addition, the timing determination unit 32 adds the number S (the input start timing of the field last in the last step 122) to the variable S_min representing the number of the last interlace video signal when the minimum value of the shift time is obtained. Set the number of the detected interlaced video signal). Further, the timing determination unit 32 sets the odd / even list flist acquired in the previous step 122 to the odd / even list flist_min when the minimum value of the shift time is obtained.

  In the next step 128, the timing determination unit 32 increments the variable i by 1. In step 130, the timing determination unit 32 determines whether or not the variable i is equal to or less than the total number n of interlaced video signals. If the variable i is less than or equal to the total number n, the determination in step 130 is affirmed and the routine returns to step 108, and steps 108 to 130 are repeated until the determination in step 130 is negative. As a result, when the field to be determined is an odd field, the number (i_min) of the interlace video signal as the head (reference), the number S_min of the last interlace video signal, and the odd / even list when the shift time dt is minimum flist_min is obtained.

  When the variable i becomes larger than the total number n, the determination at step 130 is negative and the routine proceeds to step 132. In step 132, the timing determination unit 32 determines whether the odd field and the even field are set as the determination target fields and the above processing is performed. If the determination in step 132 is negative, the process proceeds to step 134, and the timing determination unit 32 sets an even field as the determination target field for the interlaced video signal that is the head (reference). If the process of step 134 is performed, it will return to step 106 and will repeat step 106-step 134 until determination of step 132 is affirmed. When the determination at step 132 is affirmed, the timing determination unit 32 ends the timing determination process.

  As a result, the number i_min of the interlace video signal used as the head (reference) and the last interlace video signal when the shift time dt is minimum including the cases where the field to be judged is odd / even are included. The number S_min and the odd / even list flist_min are obtained. When the timing determination process is completed, the timing determination unit 32 sets the final (interlaced) last acquired interlace video signal number S_min as the optimal acquisition order of the interlace video signal. Output to.

  In addition, the timing determination unit 32 performs “odd completion notification setting” or “evenness completion notification” to the write processing units 42 of the n capture processing units 30 based on the finally obtained odd / even list flist_min. "Setting" is output. For example, when an odd field is set as the first field for an interlaced video signal acquired by a certain capture processing unit 30, the timing determination unit 32 sends a write processing unit 42 to the capture processing unit 30. To output “Even number completion notification setting”. Further, when an even field is set as the first field for an interlaced video signal acquired by a certain capture processing unit 30, the timing determination unit 32 instructs the write processing unit 42 of the capture processing unit 30. To output “Notification when odd number completion”.

  In addition, when the timing determination process is performed by the timing determination unit 32 and the “odd completion notification setting” or the “even completion notification setting” is output to the write processing unit 42 of the n capture processing units 30, The capture processing unit 30 captures an interlaced video signal. Hereinafter, with reference to FIG. 8, a flow of processing in capturing an interlaced video signal by each capturing processor 30 will be described.

  In step 140, the input unit 36 of the capture processing unit 30 captures the interlace video signal from the corresponding image capturing unit 28, and if the captured interlace video signal is an analog signal, converts it to a digital signal and then sends it to the odd / even notification unit 38. Starts output processing. In step 142, the odd / even notification unit 38 determines whether or not the timing for starting input of an arbitrary field of the interlaced video signal has been detected, and repeats step 142 until the determination is affirmed.

  When the odd / even notification unit 38 detects the input start timing of an arbitrary field of the interlaced video signal, the determination in step 142 is affirmed and the process proceeds to step 144. In step 144, the odd / even notification unit 38 detects whether the input start field of the interlaced video signal is an odd field or an even field, and writes the detection result to the buffer management unit 40 (the buffer switching unit 50) and the writing. Each is notified to the processing unit 42.

  In step 146, the buffer switching unit 50 of the buffer management unit 40 that has been notified of the odd / even detection result of the field where the input is started from the odd / even notification unit 38 sends the buffer area to the buffer number calculation unit 48 of the buffer management unit 40. Instructs the transition of the state number. When the state number transition is instructed, the buffer number calculation unit 48 increments the state number held by itself by 1, and returns the state number to 1 when the state number after the increment is larger than 3. As a result, the state number of the buffer area is changed. As a result, as shown in FIG. 4, of the two buffer areas for writing, the first buffer for writing is the standby buffer (frame determination buffer), and the second buffer for writing is the first writing buffer. The buffer for standby is switched to the second buffer for writing.

  In the next step 148, the write processing unit 42 instructs the buffer management unit 40 to write the data of the field (field to be written) whose input start timing is detected by the odd / even notification unit 38 to the write buffer area. To do. In step 150, the buffer number calculation unit 48 of the buffer management unit 40 instructed to write data refers to the buffer circulation table stored in the buffer circulation table storage unit 46. Then, the buffer number calculation unit 48 obtains the logical addresses of the two buffer areas set for writing with the state number of the buffer area currently held, and obtains the logical addresses of the obtained two buffer areas. The data is output to the physical memory number calculation unit 54 and instructed to write data.

  In step 152, the physical memory number calculation unit 54 refers to the logical buffer physical memory correspondence table stored in the logical buffer physical memory correspondence table storage unit 52. The physical memory number calculation unit 54 recognizes the physical addresses of the two write buffer areas based on the logical address input from the buffer number calculation unit 48. In the next step 154, the physical memory number calculation unit 54 writes the data of the field to be written into the two write buffer areas whose physical addresses are recognized.

  When the writing of the data of the write target field to the two write buffer areas is completed, in the next step 156, the write processing unit 42 notifies the completion of the writing of the write target field. It is determined whether it is a notification field. This determination is affirmed when the field to be written is an odd field when “notification completion setting at odd number completion” is input from the timing determination unit 32 to the write processing unit 42, and the field to be written Negated if is an even field. The determination in step 156 is affirmed when the field to be written is an even field when “even number completion notification setting” is input from the timing determination unit 32 to the write processing unit 42. Negated if the field to be embedded is an odd field.

  If the determination in step 156 is negative, the process proceeds to step 160. If the determination in step 156 is affirmed, the process proceeds to step 158, and the write processing unit 42 outputs an acquisition completion notification to the completion notification unit 34 in step 158. In step 160, the capture processing unit 30 determines whether or not the input of the interlaced video signal is completed. If the determination is negative, the process returns to step 142, and steps 142 to 160 are repeated while the input of the interlace video signal is continued. Then, when the input of the interlaced video signal is finished, the above-described series of processing is finished.

  The processing described above will be further described with reference to FIG. As shown in FIG. 9, the interlaced video signal input to each capture processing unit 30 has data in each field whose input address is in the buffer area of logical addresses {buffer 1} to {buffer 3}. Are respectively written in the two buffer areas set for writing. Also, as shown in FIG. 9, two buffer areas set for writing among the three buffer areas are cyclically switched for each field of the input interlaced video signal.

  For example, in FIG. 9, the odd field data input first is written in the buffer area of the logical address {buffer 1, buffer 2} set for writing at the input start time of the odd field. Further, the writing buffer area is switched when the writing of the data in the first odd field is completed and the input of the data in the second field (even field) is started. Accordingly, in FIG. 9, the data in the second field (even field) is written in the buffer areas of the logical addresses {buffer 2 and buffer 3} set for writing at the input start time of the field. At the time when the writing of the data in the second field (even field) is completed, the buffer area of the logical address {buffer 2} is one frame in which the data in each field is synthesized in the order of the previous odd field → even field. Is stored.

  At the time when the writing of data in the second field (even field) is completed and the input of data in the third field (odd field) is started, the buffer area for writing is further switched. Therefore, in FIG. 9, the data of the third field (odd field) is written in the buffer area of the logical address {buffer 3, buffer 1} set for writing at the input start time of the field. At the time when the writing of the data in the third field (odd field) is completed, the buffer area of the logical address {buffer 3} is one frame in which the data of each field is synthesized in the order of the previous even field → odd field. Is stored.

  At the time when the writing of data in the third field (odd field) is completed and the input of data in the fourth field (even field) is started, the buffer area for writing is further switched. Therefore, in FIG. 9, the data in the fourth field (even field) is written in the buffer areas of the logical addresses {buffer 1 and buffer 2} set for writing at the input start time of the field. At the time when the writing of the data of the fourth field (even field) is completed, the buffer area of the logical address {buffer 1} is one frame in which the data of each field is synthesized in the order of the previous odd field → even field. Is stored.

  As described above, in this embodiment, the capture processing unit 30 cyclically circulates two buffer areas as buffer areas for writing from among the three buffer areas of the logical addresses {buffer 1} to {buffer 3}. And write the data of each field to each of the two buffer areas. In addition, the buffer area in which the latest data is stored among the three buffer areas is also switched cyclically, and the data for one frame stored in the buffer area is also an odd number → even number order data and an even number. → It switches cyclically to the data in the order of odd fields.

  Also, when “odd completion notification setting” is input to the write processing unit 42, writing of data in the odd field is completed as shown in FIG. 9 with “odd completion notification setting”. An acquisition completion notification is output from the writing processing unit 42 to the completion notification unit 34 at the timing. On the other hand, when “even number completion notification setting” has been input to the write processing unit 42, writing of even field data is completed as shown in FIG. 9 with “even number completion notification setting”. An acquisition completion notification is output from the writing processing unit 42 to the completion notification unit 34 at the timing.

  The processing shown in FIG. 8 is performed by each capture processing unit 30, but since the interlaced video signal is asynchronously input to each capture processing unit 30, as shown in FIG. An acquisition completion notification is also output asynchronously from the 30 write processing units 42. Whenever an acquisition completion notification is input from an arbitrary capture processing unit 30, the completion notification unit 34 is the last interlaced video signal number S_min acquired by the input source of the acquisition completion notification input from the timing determination unit 32. It is determined whether or not it is the capture processing unit 30 corresponding to.

  The completion notification unit 34 performs no processing when the acquisition completion notification is not input from the capture processing unit 30 corresponding to the last interlaced video signal number S_min. When the acquisition completion notification is input from the capture processing unit 30 corresponding to the last interlaced video signal number S_min, the completion notification unit 34 transmits a synchronous acquisition completion notification to the image processing device 14. .

  In FIG. 10, the first (first acquired) interlaced video signal is the video signal 2 and the last (last acquired) interlaced video signal is the video signal 1 from the capture processing unit 30 corresponding to the video signal 1. An example is shown in which a synchronous acquisition completion notification is transmitted when an acquisition completion notification is input. In this way, the completion notification unit 34 transmits a synchronous acquisition completion notification every time an acquisition completion notification is input from the capture processing unit 30 corresponding to the last acquired interlaced video signal. Repeat every frame.

  In response to the synchronization acquisition completion notification received from the completion notification unit 34 of the video acquisition device 12, the image processing device 14 acquires one frame of data corresponding to each of n interlaced video signals. The video acquisition request notification is transmitted to the video acquisition device 12. When receiving the video acquisition request notification from the image processing device 14, the video acquisition device 12 distributes the received video acquisition request notification to the video output units 44 of the n capture processing units 30. As a result, the data of the interlaced video signal for one frame is output by each capture processing unit 30. Hereinafter, with reference to FIG. 11, a processing flow in outputting data of an interlaced video signal for one frame by each capture processing unit 30 will be described.

  In step 170, the video output unit 44 determines whether a video acquisition request notification has been received from the image processing apparatus 14, and repeats step 170 until the determination is positive. When a video acquisition request notification is received from the image processing apparatus 14, the determination at step 170 is affirmed and the routine proceeds to step 172. In step 172, the video output unit 44 requests the buffer management unit 40 to set an output buffer.

  When the setting of the output buffer is requested, in the next step 174, the buffer number calculation unit 48 of the buffer management unit 40 refers to the buffer circulation table stored in the buffer circulation table storage unit 46. In addition, the buffer number calculation unit 48 acquires the logical addresses of the two buffer areas set for output or standby with the state number of the buffer area currently held from the buffer circulation table. Then, the buffer number calculation unit 48 outputs the acquired logical addresses of the two buffer areas to the physical memory number calculation unit 54 and instructs the replacement of the logical addresses of the respective buffer areas.

  In step 176, the physical memory number calculation unit 54 of the buffer management unit 40 refers to the logical buffer physical memory correspondence table stored in the logical buffer physical memory correspondence table storage unit 52. Then, the physical memory number calculation unit 54 recognizes the physical addresses of the two buffer areas to which the logical address is input from the buffer number calculation unit 48. In step 178, the physical memory number calculation unit 54 updates the logical buffer physical memory correspondence table so that the logical addresses of the two buffer areas whose physical addresses are recognized are switched. Specifically, the logical address of the buffer area set for output is changed to the logical address of the buffer area set for standby, and the logical address of the buffer area set for standby is output. Change to the logical address of the buffer area set for use. Thereby, the physical memory of the buffer area for output and the buffer area for standby is switched, and the setting of the output buffer is completed.

  In step 180, the video output unit 44 requests the buffer management unit 40 to output data from the output buffer. In the next step 182, the buffer number calculation unit 48 refers to the buffer circulation table, and acquires from the buffer circulation table the logical address of the buffer area set for output with the state number of the currently held buffer area. . Then, the buffer number calculation unit 48 outputs the acquired logical address to the physical memory number calculation unit 54, and instructs to read data from the output buffer area.

  In step 184, the physical memory number calculation unit 54 refers to the logical buffer physical memory correspondence table and recognizes the physical address of the output buffer area to which the logical address is input from the buffer number calculation unit 48. In the next step 186, the physical memory number calculation unit 54 reads out data for one frame from the output buffer area in which the physical address is recognized, and outputs the read data to the video output unit 44. The data for one frame output to the video output unit 44 is transmitted from the video output unit 44 to the image processing device 14.

  The processing described above will be further described with reference to FIG. As an example, FIG. 12 shows that the “odd completion notification setting” is input to the write processing unit 42, and the completion notification unit 34 is reached when writing of data in the odd field is completed in the write buffer area. An operation in the capture processing unit 30 to which an acquisition completion notification is output is shown.

  First, when the writing of data for one frame is completed in the order of even field → odd field in the buffer area of the logical address {buffer 3} (see (1) in FIG. 12), An acquisition completion notification is output from the processing unit 42 to the completion notification unit 34. As described above, the completion notification unit 34 transmits a synchronous acquisition completion notification to the image processing device 14 every time an acquisition completion notification is input from the capture processing unit 30 corresponding to the interlaced video signal acquired last. When receiving the synchronization acquisition completion notification from the video acquisition device 12, the image processing device 14 transmits a video acquisition request notification to the video acquisition device 12 (see (2) in FIG. 12).

  When the capture processing unit 30 of the video acquisition device 12 receives the video acquisition request notification from the image processing device 14, the video output unit 44 requests the buffer management unit 40 to set an output buffer. Here, at the timing of receiving the video acquisition request notification from the image processing device 14, the buffer area of the logical address {buffer 3} is set for standby, and the buffer area of the logical address {buffer 4} is set for output (FIG. 4 (state number = 1 state shown in FIG. 4).

  Therefore, in accordance with the output buffer setting request, as shown as a transition from “initial” to “C1” in FIG. 5, the logical area of the buffer area of the logical address {buffer 3} and the logical area of the buffer area of the logical address {buffer 4} The addresses are switched (see (3) in FIG. 12). As a result, the logical address {buffer 3} is originally allocated, and the buffer area in which data for one frame (indicated as “data 1” in FIG. 12) is stored in the order of even field → odd field is stored in the logical address { It is changed to buffer 4} to become an output buffer area. When the video output unit 44 requests the buffer management unit 40 to output data from the output buffer, the latest one frame of data from the buffer area replaced with the output buffer area (“data 1” in FIG. 12). Data) is read out.

  Further, even when the writing of data for one frame is completed in the order of even field → odd field in the buffer area of the logical address {buffer 2} (see (4) in FIG. 12), the writing of the capture processing unit 30 An acquisition completion notification is output from the processing unit 42 to the completion notification unit 34. Thereafter, when the capture processing unit 30 of the video acquisition device 12 receives a video acquisition request notification from the image processing device 14 (see (5) in FIG. 12), the video output unit 44 sends an output buffer to the buffer management unit 40. Request settings. Here, at the timing of receiving the video acquisition request notification from the image processing device 14, the buffer area of the logical address {buffer 2} is set for standby, and the buffer area of the logical address {buffer 4} is set for output (see FIG. 4 (state number = 3 state shown in FIG. 4).

  Therefore, in accordance with the output buffer setting request, as shown as a transition from “C1” to “C2” in FIG. 5, the logical area of the buffer area of the logical address {buffer 2} and the logical area of the buffer area of the logical address {buffer 4} The addresses are switched (see (6) in FIG. 12). As a result, the logical address {buffer 2} is originally assigned, and the buffer area in which data for one frame (indicated as “data 2” in FIG. 12) is stored in the order of even field → odd field is represented by the logical address { It is changed to buffer 4} to become an output buffer area. When the video output unit 44 requests the buffer management unit 40 to output data from the output buffer, the latest one frame of data from the buffer area replaced with the output buffer area (“data 2” in FIG. 12). Data) is read out.

  Further, even when the writing of data for one frame is completed in the order of even field → odd field in the buffer area of the logical address {buffer 1} (see (7) in FIG. 12), the writing of the capture processing unit 30 An acquisition completion notification is output from the processing unit 42 to the completion notification unit 34. Thereafter, when the capture processing unit 30 of the video acquisition device 12 receives the video acquisition request notification from the image processing device 14 (see (8) in FIG. 12), the video output unit 44 sends an output buffer to the buffer management unit 40. Request settings. Here, at the timing of receiving the video acquisition request notification from the image processing device 14, the buffer area of the logical address {buffer 1} is set for standby, and the buffer area of the logical address {buffer 4} is set for output (FIG. 4 (state number = 2 state shown in FIG. 4).

  Therefore, in accordance with the output buffer setting request, as shown as a transition from “C2” to “C3” in FIG. 5, the logical area of the buffer area of the logical address {buffer 1} and the logical area of the buffer area of the logical address {buffer 4} The addresses are switched (see (9) in FIG. 12). Thereby, the logical address {buffer 1} is originally assigned, and the buffer area in which data for one frame (indicated as “data 3” in FIG. 12) is stored in the order of even field → odd field is stored in the logical address { It is changed to buffer 4} to become an output buffer area. When the video output unit 44 requests the buffer management unit 40 to output data from the output buffer, the latest one frame of data from the buffer area replaced with the output buffer area ("data 3" in FIG. 12). Data) is read out.

  The processing shown in FIG. 11 and the operation shown in FIG. For this reason, each time the video acquisition device 12 receives a video acquisition request notification from the image processing device 14, each capture processing unit 30 receives the frame structure determined by the timing determination unit 32 as shown in FIG. 13. The latest one frame of data is transmitted to the image processing device 14. Based on the data of the corresponding frame received from each capture processing unit 30 of the video acquisition device 12, the image processing device 14 performs stereo distance measurement processing or a video obtained by combining the video represented by the plurality of video signals in units of frames. Image processing such as generation is performed. Then, the image processing device 14 outputs the result of measuring the distance to the object to be measured when the stereo distance measurement process is performed, and the generated image is output when the image generation process is performed. Processing such as displaying on the display unit in units of frames is performed.

Thus, in the first embodiment, the acquisition order of each interlace video signal and each interlace video signal are set so that the time lag of the corresponding frames of n interlace video signals input asynchronously is minimized. The frame structure of the corresponding frame is determined. When it is notified from the individual capture processing units 30 that the writing of the determined frame structure data has been completed in the determined order of acquisition, the corresponding capture unit 30 outputs the corresponding frame data. Yes. As a result, as shown in FIG. 13, the time lag of the corresponding frames of n interlaced video signals is less than ± 1/2 field time ( ± 1/4 of the frame rate period) at most , that is, one field. It is suppressed to less than time (1/2 of the frame rate period) . Therefore, the processing accuracy of the image processing performed by the image processing device 14 can be improved. For example, in the stereo distance measurement process, the distance measurement error is reduced as compared with the case where there is a time lag for one cycle of the frame rate. it is possible to decrease cutting.

  In the first embodiment, four buffer areas are provided for each video signal, one of them is set for output, and two of the remaining three buffer areas are written. One is set for standby. In addition, the data of the same field is written in two buffer areas for writing, and the two buffer areas for writing are switched so as to circulate at the timing when the input of each field is started. When a video acquisition request notification is received from the image processing apparatus 14, the standby buffer area in which the latest data is stored and the output buffer area are switched. As a result, the process of copying data between the four buffer areas is not required, and the latest one frame of data can be output in a short time in response to a video acquisition request notification from the image processing apparatus 14.

[Second Embodiment]
Next, a second embodiment of the disclosed technique will be described. In addition, the same code | symbol is attached | subjected to the part same as 1st Embodiment, and description is abbreviate | omitted. Compared to the first embodiment, the second embodiment has a fixed frame structure of one frame of data to be written in the buffer area of the buffer memory 20 (for example, an image of one frame is obtained in the order of odd to even fields. The main difference is that the frame structure is fixed).

  As shown in FIG. 14, the video acquisition device 12 according to the second embodiment includes n capture processing units 230 provided corresponding to n imaging units 28 (n is an integer of 2 or more), timing determination A section 232 and a completion notification section 34. The timing determination unit 232 is an example of a determination unit in the disclosed technology.

  The n capture processing units 230 capture interlaced video signals input from the corresponding imaging units 28 among the n imaging units 28. In the second embodiment, the storage area of the buffer memory 20 is divided into n storage areas (buffers) corresponding to the individual capture processing units 230, and the n buffers have one frame of the interlaced video signal. Three buffer areas each capable of storing the data are provided. Each capture processing unit 230 writes data for one frame with a fixed frame structure into any one of the buffer areas while switching the buffer area in which the data of the captured interlaced video signal is switched in units of frames. Then, every time a video acquisition request notification is received from the image processing device 14, the capture processing unit 230 reads the latest one frame of data stored in the buffer area and transmits the data to the image processing device 14.

  When a video capture start notification is given from the image processing device 14 to the video acquisition device 12, the timing determination unit 232 obtains an optimal acquisition in units of interlaced video signals of corresponding frames of n interlaced video signals. Determine the order. This optimal acquisition order is based on the frame start timings detected by the individual capture processing units 230 for n interlaced video signals that are asynchronously input from the n imaging units 28, and n video images. It is determined so that the time lag of the corresponding frame of the signal is minimized. In addition, the timing determination unit 232 outputs an optimal acquisition order determination result in units of interlaced video signals to the completion notification unit 34.

  Each of the n capture processing units 230 includes an input unit 36, a frame notification unit 234, a buffer management unit 40, a write processing unit 236, and a video output unit 44. The frame notification unit 234 is an example of a notification unit in the disclosed technology.

  The frame notification unit 234 detects the input start timing in units of frames of the interlaced video signal input from the input unit 36, and sends the detection result to the buffer management unit 40 (the buffer switching unit 50) and the timing determination unit 32. Notify each. The frame notification unit 234 outputs the interlaced video signal input from the input unit 36 to the writing processing unit 236.

  The write processing unit 236 buffers the interlaced video signal data input from the frame notification unit 234 in a single buffer area set for writing by the buffer management unit 40 in units of one frame. Write through part 40. The writing processing unit 236 outputs an acquisition completion notification to the completion notification unit 34 every time writing of one frame of data to the writing buffer area is completed.

  The buffer management unit 40 sets usages for writing / output / standby for the three buffer areas provided in the buffer corresponding to the single capture processing unit 30 of the buffer memory 20, and uses Are appropriately changed, and the logical address of each buffer area is managed. As in the first embodiment, the buffer management unit 40 includes a buffer circulation table storage unit 46, a buffer number calculation unit 48, a buffer switching unit 50, a logical buffer physical memory correspondence table storage unit 52, and a physical memory number calculation unit 54. ing.

  The buffer circulation table storage unit 46 stores a buffer circulation table as shown in FIG. 15 as an example. The buffer circulation table is a table in which information defining the usage of each of the three buffer areas is registered when the state number of the buffer area is each value (1, 2 in the second embodiment).

  As shown in FIG. 15, in the second embodiment, one buffer area of the three buffer areas is used for writing, another buffer area is used for waiting, and the remaining one buffer area is used. The buffer area is set for output. “Buffer 1” to “Buffer 3” set in the buffer circulation table are logical addresses set for the three buffer areas, respectively. The buffer area whose logical address is {buffer 3} is always set for output regardless of the transition of the state number of the buffer area, and the buffer areas whose logical addresses are {buffer 1} and {buffer 2} are the state of the buffer area It is set for writing or standby according to the transition of the number.

  The buffer number calculation unit 48 holds the current state number of the buffer area, and every time the buffer switching unit 50 instructs the transition of the state number of the buffer area, the held state number is 1 → 2 → 1. Change to circulate as → 2 →…. When the write processing unit 42 is instructed to write data to the write buffer area, the buffer number calculation unit 48 refers to the buffer circulation table and writes the state number of the held buffer area. Recognizes the logical address of one buffer area set for insertion. Then, the logical address of one recognized buffer area is output to the physical memory number calculation unit 54.

  Also, when an output buffer setting request is input from the video output unit 44, the buffer number calculation unit 48 refers to the buffer circulation table and is set for standby or output by the state number of the buffer area that is held. The logical addresses of the two buffer areas are recognized. Then, the logical addresses of the two recognized buffer areas are output to the physical memory number calculation unit 54, and the replacement of each buffer area is instructed. Furthermore, when the video output unit 44 instructs the buffer number calculation unit 48 to read data from the output buffer area, the buffer number calculation unit 48 sets the logical address {buffer 3} of the buffer area set for output to the physical memory number. It outputs to the calculation part 54 and instruct | indicates reading of data.

  Whenever the detection result of the timing at which the input of one frame of the interlaced video signal is started from the input unit 36 is input from the frame notification unit 234, the buffer switching unit 50 stores the buffer area of the buffer number calculation unit 48 in the buffer area. Instructs state number transition.

  As an example, the logical buffer physical memory correspondence table storage unit 52 stores a logical buffer physical memory correspondence table as shown in each row of FIG. The logical buffer physical memory correspondence table is a table in which physical addresses of three buffer areas to which logical addresses {buffer 1} to {buffer 3} are assigned are registered.

  When the logical address of one buffer area for writing is input from the buffer number calculating section 48, the physical memory number calculating section 54 refers to the logical buffer physical memory correspondence table and writes one buffer for writing. Recognize the physical address of the region. Then, the write target data input from the write processing unit 42 is written in one write buffer area whose physical address is recognized. When the physical address of the two buffer areas is input from the buffer number calculator 48 and the replacement is instructed, the physical memory number calculator 54 replaces the logical addresses assigned to the two buffer areas. The logical buffer physical memory correspondence table is updated as follows. Furthermore, when the logical address of the buffer area for output is input from the buffer number calculation unit 48 and the reading of data is instructed, the physical memory number calculation unit 54 refers to the logical buffer physical memory correspondence table and outputs Recognizes the physical address of the buffer area. Then, one frame of data is read from the output buffer area whose physical address is recognized, and the read data is output to the video output unit 44.

Next, the operation of the second embodiment will be described. When a plurality of video signals are input asynchronously, as described above, at the timing when the corresponding frames of the plurality of video signals used for the same image processing are imaged and acquired, a maximum of one period of the frame rate is obtained. Deviation occurs. On the other hand, as described in the first embodiment with reference to FIG. 6, a plurality of video signals can be obtained even when data having a fixed frame structure is acquired for each frame from a plurality of video signals input asynchronously. By changing the acquisition order, the time lag of the corresponding frame can be reduced. For example, in the example of FIG. 6, if the video signal acquisition order is video signal 3 → video signal 4 → video signal 1 → video signal 2, the time lag of the corresponding frames of the plurality of video signals is the frame rate. It is shortened to less than ± 1/2 of one cycle (16.7 ms for 30 frames / second) , that is, less than one cycle of the frame rate .

  Based on the above, in the second embodiment, the timing determination unit 232 performs the timing determination process described below, so that the time lag of the corresponding frame of the interlaced video signal is minimized. The acquisition order of the video signal is optimized.

  That is, when the user operates the operation unit (not shown) to start the operation of the video processing apparatus 10, the n imaging units 28 start imaging, and the n imaging units 28 capture the captured video. Is output asynchronously. Also, a video capture start notification is given from the image processing device 14 to the video acquisition device 12, and the timing determination process shown in FIG. 17 is executed by the timing determination unit 232 of the video acquisition device 12 using this video capture start notification as a trigger. The

  In step 100 of the timing determination process, the timing determination unit 232 initially sets a variable dt_min that represents the minimum value of the time lag of the corresponding frame of n interlaced video signals input asynchronously from the n imaging units 28. Set a large value. In step 102, the timing determination unit 232 instructs the n capture processing units 230 to start capturing interlaced video signals. Accordingly, the detection result of the frame input start timing is repeatedly output from each of the n capture processing units 230. In step 106, the timing determination unit 232 sets 1 as an initial value to a variable i for identifying an interlace video signal that is a head (reference).

  In the next step 260, the timing determination unit 232 waits until the input start timing of the frame of the interlace video signal Ai is detected by the specific capture processing unit 230 that captures the head (reference) interlace video signal Ai. To do. When the input start timing of the frame of the interlaced video signal Ai is detected, the process proceeds from step 260 to step 116, and the timing determination unit 232 resets the timer to zero.

  In the next step 262, the timing determination unit 232 detects the input start timing of the remaining interlaced video signal frames by each of the n-1 capture processing units 230 that capture the remaining interlaced video signals. Wait until. When the frame input start timing is detected for each of the remaining n-1 interlaced video signals, the process proceeds from step 262 to step 264, and the timing determination unit 232 acquires the time from the timer as the shift time dt. .

  In the next step 266, the timing determination unit 232 determines the interlaced video signal in which the frame input start timing is detected last based on the order in which the frame input start timing is detected by the n−1 capture processing units 230. Number S is obtained. In step 124, the timing determination unit 232 determines whether or not the deviation time dt acquired in step 264 is smaller than the variable dt_min. If the shift time dt is equal to or greater than the variable dt_min, the determination in step 124 is negative and the process proceeds to step 128 described later. On the other hand, if the deviation time dt is smaller than the variable dt_min, the timing determination unit 232 determines that the minimum value of the deviation time has been updated (a new minimum value of the deviation time has been obtained), and the process proceeds to step 268. .

  In step 268, the timing determination unit 232 sets the newly obtained minimum value of the shift time to the variable dt_min, and also sets a variable i_min representing the number of the first interlace video signal when the minimum value of the shift time is obtained. Is set to the current value of variable i. In addition, the timing determination unit 232 sets the number S (the frame input start timing last in the last step 266) to the variable S_min indicating the number of the last interlace video signal when the minimum value of the shift time is obtained. Set the number of the detected interlaced video signal).

  In the next step 128, the timing determination unit 232 increments the variable i by 1. In step 130, the timing determination unit 232 determines whether or not the variable i is equal to or less than the total number n of interlaced video signals. When the variable i is less than or equal to the total number n, the determination in step 130 is affirmed and the process returns to step 260, and steps 260, 116, 262 to 266, 124, 268, 128 are repeated until the determination in step 130 is negative. As a result, the number (i_min) of the interlace video signal used as the head (reference) and the number S_min of the last interlace video signal when the shift time dt is minimized are obtained. If the variable i becomes larger than the total number n, the determination in step 130 is denied, and the timing determination unit 232 ends the timing determination process.

  As a result, the number (i_min) of the interlace video signal used as the head (reference) and the number S_min of the last interlace video signal when the shift time dt is minimized are obtained. When the timing determination process ends, the timing determination unit 232, as the optimal acquisition order of the interlaced video signal, the last obtained (last acquired) interlaced video signal number S_min as the completion notification unit 34. Output to.

  Further, when the timing determination process is performed by the timing determination unit 232, the interlace video signal is captured by each capture processing unit 230. Hereinafter, with reference to FIG. 18, the flow of processing in capturing an interlaced video signal by each capturing processing unit 230 will be described.

  In step 270, the input unit 36 of the capture processing unit 230 captures the interlaced video signal from the corresponding image capturing unit 28, and converts the captured interlaced video signal into a digital signal if the captured interlaced video signal is an analog signal, and then sends it to the frame notification unit 234. Starts output processing. In step 272, the frame notification unit 234 determines whether or not the timing at which input of an arbitrary frame of the interlaced video signal is started is detected, and repeats step 272 until the determination is affirmed. When the frame notification unit 234 detects the input start timing of any frame of the interlaced video signal, the detection result of the frame input start timing is sent to the buffer management unit 40 (the buffer switching unit 50) and the write processing unit 236, respectively. Notice. As a result, the determination at step 272 is affirmed and the routine proceeds to step 274.

In step 274, the buffer switching unit 50 of the buffer management unit 40 that is notified of the detection result of the frame input start timing from the frame notification unit 234 sends the buffer region state number to the buffer number calculation unit 48 of the buffer management unit 40. Instruct the transition. When the transition of the state number is instructed, the buffer number calculation unit 48 increments the state number held by itself by 1, and returns the state number to 1 when the incremented state number is greater than 2. As a result, the state number of the buffer area is changed. As a result, as shown in FIG. 15, the write buffer area is replaced with the standby buffer (frame determination buffer), and the standby buffer is replaced with the write buffer.

  In the next step 276, the write processing unit 236 instructs the buffer management unit 40 to write the data of the frame (frame to be written) whose input start timing is detected by the frame notification unit 234 to the write buffer area. To do. In step 278, the buffer number calculation unit 48 of the buffer management unit 40 instructed to write data refers to the buffer circulation table stored in the buffer circulation table storage unit 46. Then, the buffer number calculation unit 48 acquires the logical address of the buffer area set for writing with the state number of the buffer area currently held, and uses the acquired logical address of the buffer area as the physical memory number calculation unit 54. To write data.

  In step 280, the physical memory number calculation unit 54 refers to the logical buffer physical memory correspondence table stored in the logical buffer physical memory correspondence table storage unit 52. The physical memory number calculation unit 54 recognizes the physical address of the write buffer area based on the logical address input from the buffer number calculation unit 48. In the next step 282, the physical memory number calculation unit 54 writes the data of the write target frame in the write buffer area in which the physical address is recognized.

  When the writing of the data of the write target frame to the write buffer area is completed, the write processing unit 236 outputs an acquisition completion notification to the completion notification unit 34 in step 284. In the next step 286, the capture processing unit 230 determines whether or not the input of the interlace video signal has been completed. If the determination is negative, the process returns to step 272, and steps 272 to 286 are repeated while the input of the interlaced video signal is continued. Then, when the input of the interlaced video signal is finished, the above-described series of processing is finished.

  The processing shown in FIG. 18 is performed by each capture processing unit 230. Since an interlaced video signal is input asynchronously to each capture processing unit 230, as shown in FIG. An acquisition completion notification is also output asynchronously from the write processing unit 42. Whenever an acquisition completion notification is input from any capture processing unit 230, the completion notification unit 34 is the last interlaced video signal number S_min that the acquisition completion notification is input from the timing determination unit 232. It is determined whether or not the import processing unit 230 corresponds to.

  The completion notifying unit 34 does not perform any processing when the acquisition completion notification is not input from the capture processing unit 230 corresponding to the last interlaced video signal number S_min. When the acquisition completion notification is input from the capture processing unit 230 corresponding to the last interlaced video signal number S_min, the completion notification unit 34 transmits a synchronous acquisition completion notification to the image processing device 14. .

  FIG. 19 shows that the first (first acquired) interlaced video signal is the video signal 3 and the last (last acquired) interlaced video signal is the video signal 2 from the capture processing unit 30 corresponding to the video signal 2. An example of transmitting a synchronous acquisition completion notification when an acquisition completion notification is input is shown. In this way, the completion notification unit 34 transmits a synchronous acquisition completion notification every time an acquisition completion notification is input from the capture processing unit 230 corresponding to the last acquired interlaced video signal. Repeat every frame.

  Note that the processing in the image processing device 14 that has received the synchronous acquisition completion notification and the processing in each capture processing unit 230 that has received the video acquisition request notification transmitted from the image processing device 14 are the same as in the first embodiment. Therefore, explanation is omitted.

  As described above, in the second embodiment, the optimal acquisition order of interlaced video signals is determined so that the time lag of the corresponding frames of the interlaced video signals input asynchronously is minimized. In addition, when each capture processing unit 230 is notified that the writing of data for one frame has been completed in the determined order of acquisition, the data of the corresponding frame is output from each capture processing unit 230. .

Accordingly, as shown in FIG. 19, n time-interlaced images can be clearly understood even when the “time shift between images based on the video signal 3 standard” is compared with the “time shift between videos based on the video signal 1 standard”. The time lag of the corresponding frame of the signal can be reduced. Specifically, the time lag of the corresponding frames of n interlaced video signals is less than ± 1 field time ( ± 1/2 of the frame rate period) at most , that is, 2 field times (one cycle of the frame rate). Is suppressed to less than . Therefore, the processing accuracy of the image processing performed by the image processing device 14 can be improved. For example, in the stereo distance measurement process, the distance measurement error is reduced as compared with the case where there is a time lag for one cycle of the frame rate. it is possible to decrease cutting.

  In the second embodiment, three buffer areas are provided for each video signal, one of which is for output, another is for writing, and the other is for standby. Is set. Further, the buffer area for writing and the buffer area for waiting are switched at the timing when the input of each frame is started. When a video acquisition request notification is received from the image processing apparatus 14, the standby buffer area in which the latest data is stored and the output buffer area are switched. As a result, as in the first embodiment, the process of copying data between the three buffer areas is not required, and the latest one frame of data is sent to the video acquisition request notification from the image processing apparatus 14 in a short time. Can be output.

  In the second embodiment, the case where an interlace video signal is acquired has been described as an example. However, the present invention is not limited to this, and other types of video signals divided in units of frames may be acquired. It may be.

[Third Embodiment]
Next, a third embodiment of the disclosed technology will be described. In addition, the same code | symbol is attached | subjected to the part same as 1st Embodiment, and description is abbreviate | omitted. The third embodiment is different from the first embodiment in that a buffer memory to which an input interlaced video signal is written is provided in the main memory 16 of the video processing device 290.

  As shown in FIG. 20, a video processing device 290 according to the third embodiment includes a video input control unit 24, an image processing device 14, a main memory 16, and a main bus 18 for connecting them together. I have.

  The video input control unit 24 captures interlaced video signals output from n (n ≧ 2) imaging units 28 mounted on the moving body. The video input control unit 24 also obtains the n interlaced video signals in the order of acquisition and the corresponding video signals so that the time lag of the corresponding frames of the n interlaced video signals is minimized. Processing for determining the frame structure of the frame is performed. Subsequently, the video input control unit 24 sequentially writes the data of the interlaced video signal captured from the imaging unit 28 in a buffer memory provided in the main memory 16 in units of fields.

  The buffer memory in the main memory 16 is provided with four buffer areas corresponding to each interlaced video signal. The video input control unit 24 writes the data in the same field to each of the n buffered interlaced video signals in two buffer areas set for writing out of the four buffer areas. Switch so that the embedded buffer area circulates in field units. Further, when the video input control unit 24 writes the corresponding frame data of each of the n interlaced video signals to the buffer memory, the video input control unit 24 transmits a synchronization acquisition completion notification to the image processing device 14 for each frame. repeat.

  When receiving the synchronization acquisition completion notification from the video input control unit 24, the image processing device 14 outputs a video acquisition request notification to the video input control unit 24. When receiving a video acquisition request notification from the image processing device 14, the video input control unit 24 stores a buffer area (standby buffer area) that stores the latest one frame of data for each of n interlaced video signals. The physical memory corresponding to is set to an accessible state. As a result, the image processing apparatus 14 accesses the physical memory set in an accessible state in the buffer memory, and reads the latest one frame data of each of the n interlaced video signals. Then, the image processing device 14 performs image processing such as stereo distance measurement processing using the read data.

  In addition, since the other structure and effect | action of the video processing apparatus 290 which concern on 3rd Embodiment are the same as that of 1st Embodiment, description is abbreviate | omitted.

  In the third embodiment, in the configuration in which the buffer memory is provided in the main memory 16 of the video processing device 290, the acquisition order of n interlaced video signals and each video signal are the same as in the first embodiment. The frame structure of each corresponding frame is optimized. However, the present invention is not limited to this, and in the above configuration, as in the second embodiment, data having a fixed frame structure is written to the buffer, and only the acquisition order of n interlaced video signals is optimized. It may be.

  In the first and third embodiments, the input start timing of each field of the interlaced video signal is detected, and the acquisition order of each video signal and the frame structure of each corresponding frame of each video signal are optimized. However, the present invention is not limited to this. For example, in the first and third embodiments, the input end timing of each field of each video signal is detected, and the acquisition order of each video signal and the frame structure of each corresponding frame of each video signal are optimized. May be. In the second embodiment, the input start timing of each frame of the interlaced video signal is detected and the acquisition order of each video signal is optimized. However, the present invention is not limited to this. For example, also in the second embodiment, the input end timing of each field of each video signal may be detected and the acquisition order of each video signal may be optimized.

Further, in the first embodiment, a mode is described in which the timing determination unit 32 determines the frame structure or the video signal acquisition order for each video signal so that the time lag of each corresponding frame of the video signal is minimized. However, the present invention is not limited to this. For example, a solution in which the time lag of each corresponding frame of the video signal is less than ± 1/4 of the frame rate period, that is, less than 1/2 of the frame rate period (frame structure for each video signal or video signal When there are a plurality of (acquisition orders), a solution that does not minimize the frame time lag among the plurality of solutions may be selected. In the second embodiment, the timing determination unit 232 determines the acquisition order of the video signal so that the time lag between the corresponding frames of the video signal is minimized. However, the present invention is not limited to this. It is not a thing. For example, when there are a plurality of solutions (video signal acquisition order) in which the time lag of each corresponding frame of the video signal is less than ± 1/2 of the frame rate cycle, that is, less than one cycle of the frame rate. Of these solutions, a solution that does not minimize the frame time lag may be selected.

  Furthermore, in the first embodiment, four buffer areas are provided for one video signal, and one buffer area is used for standby and another buffer area is used for output, and a video acquisition request notification is received. Then, the aspect which swaps the buffer area for standby | standby and output was demonstrated. However, the present invention is not limited to this. When the output buffer is omitted and a video acquisition request notification is received, data for one frame written in the standby buffer area (frame determination buffer) is preset. You may make it copy to a memory area. Also in the second embodiment, three buffer areas are provided for one video signal, and one buffer area is used for standby and another buffer area is used for output. When receiving, the buffer areas for standby and output are switched. However, the present invention is not limited to this. When the output buffer is omitted and a video acquisition request notification is received, data for one frame written in the standby buffer area (frame determination buffer) is preset. You may make it copy to a memory area.

  Also, in the above, when each capture processing unit 30 (or capture processing unit 230) receives a video acquisition request notification from the image processing device 14, it reads out one frame of data from the output buffer and transmits it to the image processing device 14. The aspect to do was demonstrated. However, the present invention is not limited to this. Instead of the process of reading and transmitting data from the output buffer, the image processing apparatus 14 may perform a process for enabling the data to be read from the output buffer. Good. As processing for enabling the image processing device 14 to read data from the output buffer, for example, processing for performing bus mapping so that the image processing device 14 can access the output buffer via the bus I / F 22 can be cited. . Further, in a configuration in which the image processing device 14 can directly access the output buffer via the bus I / F 22, processing for notifying the image processing device 14 of the address of the output buffer may be used. These processes are examples of “a process for outputting a corresponding frame of each of a plurality of video signals” in the disclosed technique.

  Also, in the above, logical addresses are assigned to 3 or 4 buffer areas per video signal, the use of each buffer area is associated with the logical address by the buffer circulation table, and the logical address by the logical buffer physical memory correspondence table. Physical memory is associated. However, the present invention is not limited to this, and the use of each buffer area (for writing / standby / for output) and the corresponding physical memory may be directly associated by a table.

  All documents, patent applications and technical standards mentioned in this specification are to the same extent as if each individual document, patent application and technical standard were specifically and individually stated to be incorporated by reference. Incorporated by reference in the book.

DESCRIPTION OF SYMBOLS 10 Image processing apparatus 12 Image acquisition apparatus 14 Image processing apparatus 20 Buffer memory 24 Image | video input control part 28 Image pick-up part 30,230 Capture process part 32,232 Timing determination part 34 Completion notification part 38 Odd / even notification part 40 Buffer management part 42,236 Write processing unit 44 Video output unit 46 Buffer circulation table storage unit 48 Buffer number calculation unit 50 Buffer switching unit 52 Logical buffer physical memory correspondence table storage unit 54 Physical memory number calculation unit 234 Frame notification unit 290 Video processing device

Claims (12)

Each frame has an odd field and an even field, and at least one of an input start timing and an input end timing of the video signal in units of fields for a plurality of video signals having the same frame rate and input asynchronously. A notification section for each notification;
Based on the timing notified to each of the plurality of video signals by the notification unit, a shift in acquisition timing of the corresponding frame of each of the plurality of video signals is less than ½ of the period of the frame rate. A frame structure of a corresponding frame of each of the plurality of video signals, a first frame structure in which the odd field and the even field are combined in order, and a second frame structure in which the even field and the odd field are combined in order. And a determination unit for determining an acquisition order in units of the video signals of the corresponding frames of the plurality of video signals, respectively
Outputting a corresponding frame of each of the plurality of video signals acquired according to an acquisition order based on the frame structure and the video signal determined by the determination unit, or each of the plurality of video signals An output unit for performing processing for outputting the corresponding frame of
Including video acquisition device. A buffer memory provided with a plurality of buffers corresponding to the different video signals among the plurality of video signals;
Each of the video signals is provided corresponding to each of the video signals, and the corresponding video signal is stored in the corresponding buffer among the plurality of buffers, and one frame of the frame structure determined by the determination unit A plurality of write processing units for outputting an acquisition completion notification each time the video signal is stored in the corresponding buffer;
A plurality of said writing process unit, every time the acquisition completion notification corresponding to the last video signal definitive the acquisition order in units of the video signal determined in the determination unit is input, the completion notification synchronization A completion notification part to be output;
Further comprising
The output unit reads and outputs a corresponding frame of each of the plurality of video signals from the plurality of buffers each time the synchronization notification is output from the completion notification unit, or the plurality of the plurality of video signals. The video acquisition apparatus according to claim 1, wherein a process for outputting a corresponding frame of each video signal is performed. Each of the plurality of buffers is provided with three or more buffer areas for storing the video signal for one frame.
Each of the write processing units stores the corresponding video signal in each of two buffer areas set for writing among the three or more buffer areas of the corresponding buffer,
Provided for each of the video signals, two of the three or more buffer areas of the corresponding buffer are set for writing, and the field of the corresponding video signal is set in units of fields. Each time the input start timing or the input end timing is notified by the notification unit, a buffer area in which the video signals of the latest two fields of the two buffer areas set for writing are stored 3. The video according to claim 2, further comprising: a plurality of buffer management units that switch a buffer area to be written by setting a buffer area that is not set for writing to a standby buffer area for writing. Acquisition device. Each of the plurality of buffers is provided with four or more buffer areas for storing the video signal for one frame,
Each time each of the buffer management units outputs the synchronization completion notification from the completion notification unit, the video signals of the two latest fields of the corresponding four or more buffer areas of the buffer are stored. Set the buffer area for output,
The output unit outputs a corresponding frame of each of the plurality of video signals by each of the buffer management units corresponding to each of the video signals each time the synchronization completion notification is output from the completion notification unit. Each of the plurality of video signals is set for output by each buffer management unit corresponding to each of the video signals. The video acquisition apparatus according to claim 3, wherein processing for reading and outputting from each buffer area is performed. For a plurality of video signals that have the same frame rate and are input asynchronously, a notification unit that respectively notifies at least one of an input start timing and an input end timing of the video signal in units of frames;
Wherein based on said timing are respectively notified for the plurality of video signals by the notification unit, so that the deviation of the timing of acquiring the corresponding frame of each of the plurality of video signals is less than one round-life of the frame rate, A determination unit for determining an acquisition order in units of the video signals of the corresponding frames of the plurality of video signals;
Output a corresponding frame of each of the plurality of video signals acquired according to an acquisition order based on the video signal determined by the determination unit, or output a corresponding frame of each of the plurality of video signals An output unit that performs processing for
Including video acquisition device. A buffer memory provided with a plurality of buffers corresponding to the different video signals among the plurality of video signals;
Each video signal is provided corresponding to each video signal, and the corresponding video signal is stored in the corresponding buffer among the plurality of buffers, and the video signal for one frame is stored in the corresponding buffer. A plurality of write processing units that output an acquisition completion notification each time,
A plurality of said writing process unit, every time the acquisition completion notification corresponding to the last video signal definitive the acquisition order in units of the video signal determined in the determination unit is input, the completion notification synchronization A completion notification part to be output;
Further comprising
The output unit reads out and outputs a corresponding frame of each of the plurality of video signals from the plurality of buffers or outputs the plurality of video signals each time the synchronization completion notification is output from the completion notification unit. The video acquisition apparatus according to claim 5, wherein a process for reading and outputting each corresponding frame is performed. Each of the plurality of buffers is provided with two or more buffer areas for storing the video signal for one frame.
Each of the write processing units stores the corresponding video signal in one buffer area set for writing out of two or more buffer areas of the corresponding buffer,
Provided for each of the video signals, one of the two or more buffer areas of the corresponding buffer is set for writing, and the corresponding video signal is in units of frames. Each time the input start timing or the input end timing is notified by the notification unit, a buffer area in standby that is not set for writing instead of one buffer area set for writing The video acquisition device according to claim 6, further comprising: a plurality of buffer management units that switch a buffer area to be written by setting for writing. Each of the plurality of buffers is provided with three or more buffer areas for storing the video signal for one frame.
Each of the buffer management units is a buffer region in which the video signal of the latest frame is stored among the three or more buffer regions of the corresponding buffer each time the synchronization completion notification is output from the completion notification unit. Set for output,
The output unit outputs a corresponding frame of each of the plurality of video signals by each of the buffer management units corresponding to each of the video signals each time the synchronization completion notification is output from the completion notification unit. Each of the plurality of video signals is set for output by each buffer management unit corresponding to each of the video signals. The video acquisition apparatus according to claim 7 , wherein processing for reading and outputting each from the buffer area is performed. Each of the buffer areas is given a physical address,
The buffer management unit manages the identification information indicating the usage of each buffer area including the writing and the output and the physical address of the buffer area allocated to each usage in association with each other, and 9. The read / write operation is performed by converting the identification information of the buffer area designated as a write target or a read target from the read processing unit and the output unit into a physical address of the buffer area. Video acquisition device.   The imaging unit for inputting the plurality of video signals to the video acquisition device is provided at a position different from each other in a direction in which the periphery of the moving body is imaged. The video acquisition device according to claim 1. Each frame has an odd field and an even field, and at least one of an input start timing and an input end timing of the video signal in units of fields for a plurality of video signals having the same frame rate and input asynchronously. A notification step for each notification;
Based on the timing notified to each of the plurality of video signals by the notification step, a shift in acquisition timing of the corresponding frame of each of the plurality of video signals is less than ½ of the cycle of the frame rate. A frame structure of a corresponding frame of each of the plurality of video signals, a first frame structure in which the odd field and the even field are combined in order, and a second frame structure in which the even field and the odd field are combined in order. A determination step of determining an acquisition order in units of the video signal of the corresponding frame of each of the plurality of video signals;
An output step of outputting a corresponding frame of each of the plurality of video signals acquired in accordance with an acquisition order based on the frame structure and the video signal determined in the determination step;
Video acquisition method including A notification step of notifying at least one of input start timing and input end timing of the video signal in units of frames for a plurality of video signals having the same frame rate and input asynchronously;
Wherein based on said timing are respectively notified for the plurality of video signals by the notification step, as the deviation of the timing of acquiring the corresponding frame of each of the plurality of video signals is less than one round-life of the frame rate, A determination step of determining an acquisition order in units of the video signals of corresponding frames of the plurality of video signals;
An output step of outputting a corresponding frame of each of the plurality of video signals acquired in accordance with an acquisition order based on the video signal determined in the determination step;
Video acquisition method including

Images