TWM634644U - frame alignment system - Google Patents

Abstract

A frame alignment system, in which a plurality of high-speed photography devices take pictures synchronously according to a pulse signal per second, and shoot at the same frequency M to generate multiple groups of continuous frames, and at the same time, each high-speed photography device is reset according to the pulse signal per second A counter re-counts and obtains a current standard time; and each of the high-speed photography devices takes a count value of the counter and the standard time of each frame as an identification code of each frame, and each of the groups The continuous frames with the identification code are provided to an image processing device, so that the image processing device aligns the identification code of the first frame of each group of continuous frames with the identification code provided by each of the high-speed photography devices. Equal groups of consecutive frames with the identification code.

Description

frame alignment system

The present invention relates to a system for aligning frames from multiple sources, in particular to a frame alignment system for synchronously shooting multiple groups of frames by multiple high-speed cameras and aligning multiple groups of frames in shooting order.

Unlike ordinary cameras that shoot 24 frames per second (frame, also known as frame), high-speed cameras can perform high-speed photography at a higher frequency (such as shooting 240 frames per second to tens of thousands of frames, etc.), Therefore, the occurrence, development and trajectory of high-speed motion can be clearly photographed and displayed in front of people. Therefore, high-speed cameras are often used to analyze, for example, crash tests, ballistics, the movement of living things or the movement of athletes.

And when using a plurality of high-speed cameras to shoot the same target from different angles at the same time, each of these high-speed cameras produces a group of continuous frames and provides them to a frame analysis device at the back end, because the internal clock of each of the high-speed cameras and the generated The frame numbers are independent. For example, if there are three high-speed cameras, the frame numbers of the group shot by the first high-speed camera are 101, 102, 103..., and the frame numbers of the group shot by the second high-speed camera are 103. , 104, 105..., the frame numbers of the group shot by the third high-speed camera are respectively 100, 101, 102..., therefore, even if an external trigger device is used, the three groups of consecutive frames received by the frame analysis device Time synchronization, for example, the 101st, 103rd, and 100th frames of the three high-speed cameras were shot at the same time point, but because the recording time of the three high-speed cameras is not precisely synchronized, the frame analysis device does not know this How to align three groups of consecutive frames according to the shooting sequence (1st frame, 2nd frame...), that is, the frame analysis device cannot separate the three groups of frames based on the time stamp (time point) and frame number of the frame being shot. A group of consecutive frames are aligned according to the order of shooting; and if the frame analysis device uses these three groups of consecutive frames whose shooting order is not aligned for frame analysis, the analysis results will be generated because the three groups of frames are not aligned in order errors or inaccuracies.

Therefore, the purpose of the present invention is to provide a frame alignment system that can at least solve the above-mentioned problems.

Thus, a frame alignment system of the present invention includes an image processing device and a plurality of high-speed photographing devices capable of communicating with the image processing device. Positive integer) shooting to produce multiple groups of continuous frames; and each of the high-speed photography devices simultaneously resets a counter according to the pulse signal per second and obtains a current standard time; each of the high-speed photography devices is used to shoot each frame A count value of the counter and the standard time are used as an identification code of each frame, and then each group of continuous frames with the identification code is provided to the image processing device; For each set of consecutive frames with the identification code provided, the identification code of the first frame is aligned with the groups of consecutive frames with the identification code.

In some embodiments of the present invention, when the image processing device finds that a certain group of continuous frames with the identification code lacks the first frame, the image processing device The identification code of the 2nd frame and the identification code of the 2nd frame in other groups of frames with the identification code, combine a certain group of consecutive frames with the identification code with other groups of frames with the identification code align.

In some implementation aspects of the present invention, the image processing device also aligns the groups of consecutive frames with the identification code at intervals of N frames according to the identification code of each of the frames. The frame of the identification code, where N is a positive integer and N≥0.

In some implementation aspects of the present invention, when each of the high-speed photography devices resets the counter therein again according to the pulse per second signal, the standard time will be updated at the same time.

In some embodiments of the present invention, each of the high-speed photography devices includes a photography lens module, a pulse signal and time supply module and a microcontroller, and the pulse signal and time supply module can generate the pulse per second signal and can obtain the current standard time at the geographic location, the microcontroller is electrically connected to the pulse signal and time providing module to request the pulse signal and time providing module to provide the pulse signal per second and the standard time ; and the microcontroller includes a trigger signal generator, the counter, a time register and a storage unit, the trigger signal generator generates a trigger signal with the frequency M according to the pulse signal per second to trigger the photographic lens module The group is photographed synchronously with the frequency M, the time register temporarily stores the standard time, and the storage unit temporarily stores each group of continuous frames with the identification code.

In other embodiments of the present invention, the frame alignment system also includes a pulse signal and time providing device capable of wireless communication with each of the high-speed photography devices, and the pulse signal and time providing device will provide the pulse signal per second and the current standard time of the geographic location to these high-speed photography devices; and each of the high-speed photography devices includes a camera lens module and a microcontroller, and the microcontroller includes a trigger signal generator, the counter, a A time register and a storage unit, the trigger signal generator generates a trigger signal with the frequency M according to the pulse signal per second to trigger the photography lens module to take pictures synchronously at the frequency M, and the time register temporarily stores the standard For a period of time, the storage unit temporarily stores each group of consecutive frames with the identification code.

The function of this new model is: control multiple high-speed photographing devices to take pictures synchronously through the pulse signal per second to generate multiple groups of frames, and make the high-speed photographing devices reset the counters in the pulse signal per second at the same time to recount and obtain the Standard time, and make each high-speed photographing device take the count value of the counter and the standard time when shooting each frame as the identification code of each frame, and then provide each group of consecutive frames with the identification code to the frame An image processing device, so that the image processing device can align the groups of frames according to the identification code of the first frame in each group of frames, so that the groups of frames are aligned frame by frame in time, so that multiple high-speed Multiple groups of consecutive frame frames produced by simultaneous shooting by the photographic device are aligned with each other in time.

Before the present invention is described in detail, it should be noted that in the following description, similar elements are denoted by the same numerals.

Referring to shown in Figure 1, it is the main process of the frame alignment of the novel frame alignment system, and as shown in Figure 2, an embodiment of the novel frame alignment system 100 mainly includes a plurality of high-speed photography devices (the present embodiment uses three A high-speed photography device A, B, C as an example) and an image processing device 1, the image processing device 1 can communicate with these devices through a wired connection or wireless transmission (such as but not limited to the Internet or short-distance wireless transmission, etc.) High-speed camera devices A, B, and C communicate.

And for the purpose of realizing frame alignment, as shown in step S1 of Fig. 1, make these high-speed photography devices A, B, C according to a pulse-per-second (Pulse-Per-Second, be called for short PPS) signal (hereinafter collectively referred to as PPS signal) ) synchronous photography, and respectively shoot at the same frequency M (M is a positive integer) Hz to generate three groups of continuous frames (a static image is a frame, or called a frame), so each high-speed photography The frame rate of each group of consecutive frames generated by devices A, B, and C is M (that is, M frames are shot per second).

Specifically, as shown in FIG. 2 and FIG. 3 , each of the high-speed photography devices A, B, and C includes a photography lens module 21 , a pulse signal and time supply module 22 and a microcontroller 23 . The photographing lens module 21 is a high-speed camera; the pulse signal and time providing module 22 can generate the PPS signal and can obtain a current standard time of the geographic location, for example, the pulse signal and time providing module 22 can be But not limited to the global satellite positioning system (hereinafter referred to as GPS) module that can generate PPS signals and provide the standard time, the GPS module can communicate with global satellites to receive global satellite positioning signals, and according to the received global satellite positioning signals Generate the PPS signal and the standard time.

In addition, as shown in Figure 4, another implementation of this embodiment is to replace the pulse signal and time supply modules built in each of the high-speed photography devices A, B, and C with a pulse signal and time supply device 3. Group 22, therefore, each of the high-speed photographic devices A, B, and C only needs to include the photographic lens module 21 and the microcontroller 23, and each of the high-speed photographic devices A, B, and C can communicate via wireless (such as but not Limited to the built-in WiFi or Bluetooth wireless communication module) to receive the pulse signal and the PPS signal and the standard time output (transmitted) by the time providing device 3 .

Therefore, in step S1, the microcontroller 23 of each of the high-speed photography devices A, B, and C provides the pulse signal and the PPS signal output by the time providing module 22 to a trigger signal generator 231 therein, Make the trigger signal generator 231 generate a trigger signal of frequency M to the photographic lens module 21 according to the PPS signal and the preset frequency M (M can be but not limited to 240Hz~tens of thousands of Hz) to trigger the The camera lens module 21 shoots images at the frequency M to generate a group of continuous frames; and because the camera lens modules 21 of the high-speed camera devices A, B, and C are simultaneously triggered by the trigger signal, the high-speed The photographic devices A, B, and C will respectively and synchronously generate three groups of continuous frames.

Simultaneously, as shown in step S2 of Fig. 1, this microcontroller 23 can reset a counter 232 wherein according to this PPS signal, makes this counter count from 0, and this standard that this microcontroller 23 will receive Time (such as 8:42:36 in the morning) is stored in a time register 233 (such as represented by 084236). That is, when the trigger signal generator 231 receives the PPS signal, the counter 232 will also be reset and the standard time will be written into the time register 233. Therefore, in the photographing lens module 21 While being triggered by the trigger signal to continuously shoot one frame after another, the counter 232 is also continuously counting.

Therefore, as shown in step S3 of Figure 1, whenever the photographic lens module 21 captures a frame and outputs it to the microcontroller 23, the microcontroller 23 reads the counter 232 after receiving the frame. A current count value and the standard time temporarily stored in the time register 233 are combined together, for example, according to the (standard time+count value of the counter) mode to become a unique identification code of the frame , and then store the image frame and its corresponding identification code into a storage unit 234 therein.

And even if the counters 232 of these high-speed photographing devices A, B, and C are reset at the same time, there will still be slight differences in their counting values. Therefore, for example, as shown in FIG. 5 , if these high-speed photographing devices A, B, and C Obtain the standard time ( 8:42:36 ) at 8:42:36 in the morning and start synchronous photography according to the PPS signal, then the high-speed photography device A takes pictures within 1 second of 8:42:36 in the morning 300 frames, starting from the 1st frame, its identification code will be for example 084236 00 002 (1st frame), 084236 00 100 (2nd frame), 084236 00 203 (3rd frame) …, 084236 29905 (the 300th frame); and starting from the first frame shot by the high-speed camera B, its identification code will be, for example, 084236 00 001 , 084236 00 102 , 084236 00 202 ..., 084236 29903 (the 300th frame) ; And starting from the first frame taken by the high-speed photography device C, its identification code will be such as 084236 00 003 , 084236 00 101 , 084236 00 204 ..., 084236 29901 (the 300th frame); Each time the counter 232 counts about 100, each high-speed camera will shoot a frame. But the counting frequency of each of the counters 232 is not limited to this, the above is just an example, that is, each of the counters 232 as long as adopt the same frequency and its count value will not exceed the value range that can be stored in the temporary register that it adopts. For example, if Each of the counters 232 adopts a 16-bit register, and the counting frequency only needs to be less than 65KHz (65535Hz).

And when this microcontroller 23 receives next PPS signal ( 8:42:37 in the morning), it will reset the counter 232 wherein to count again according to the PPS signal, and update the temporary register at the same time This standard time in 233 is 084237. Therefore, as shown in Figure 5, for the 300 frames shot within 1 second of 8:42:37 in the morning, the unique identification code of each frame will be generated according to (standard time (084237) + count value of the counter) .

Then, as shown in step S4 of Figure 1, each of the high-speed photographic devices A, B, and C will be stored in each of the storage units 234 in time and have (multiple) frame frames (or a group of frame frames) with the identification code provided (transmitted) to the image processing device 1 .

After the image processing device 1 receives each group of continuous frame frames with the identification code from each of the high-speed photography devices A, B, and C, as shown in step S5 of Figure 1, the image processing device 1 performs according to each group of continuous and The identification code of the first frame (for example, the first frame shot at the first second) among the frames with the identification code (that is, each of the first frames is used as the alignment point), aligning the groups of consecutive Frame, as shown in FIG. 5 as an example, when the image processing device 1 judges that the identification codes 08423600002, 08423600001, and 084236000 of the first frame captured by each of the high-speed photography devices A, B, and C in the first second are similar, That is, if the difference between the values of these three identification codes is within a preset value, such as within 30, it is determined that each of the first frames shot by each of the high-speed photography devices A, B, and C in the first second is synchronously shot at the same time , that is, use the three first frames as alignment points to align the three groups of frames from each of the high-speed photography devices A, B, and C, so that the three groups of frames are synchronously aligned frame by frame (frame by frame) in time.

In addition, if each of the high-speed photography devices A, B, and C transmits 300 frames (a group) of frames with identification codes to the image processing device 1 per second, the image processing device 1 can perform frame alignment once per second, However, it is not limited thereto, and the image processing device 1 can also be set to select N (N is a positive integer and N≧0) frames at intervals from each group of consecutive frames with the identification code (for example, if N=10, that is, the 11th frame, the 21st frame, the 31st frame...), and according to the identification code of each frame (that is, the 11th frame, the 21st frame, the 31st frame...), the These groups of continuous frames with the identification code perform frame alignment to more accurately ensure that the three groups of frames are synchronized frame by frame in time. It is worth mentioning that when N=0, it means that each frame of the three groups of frames is aligned frame by frame.

In addition, because each of the photographic lens modules 21 of the high-speed photographic devices A, B, and C will randomly lose frames due to various potential factors, some frames will be lost in the continuous frames shot every second, for example, when When shooting 300 frames (shooting frequency M is 300Hz), an average of 2~4 frames will be lost per second. Therefore, assuming that the high-speed photography device A just lost the first frame shot in the first second, after the image processing device 1 receives the group of frames from the high-speed photography device A, the image processing device 1 compares the high-speed photography The identification code (08423600100) of the first frame from device A and the identification code (08423600001) of the first frame from high-speed camera device B and/or the first frame from high-speed camera device C The identification code (0842360003), find the last 3 codes (100) of the identification code of the first frame sent by high-speed camera device A and the identification code of the first frame of high-speed camera device B and/or high-speed camera device C When the gap between the last 3 yards (001, 003) exceeds the preset value (for example, 30), the image processing device 1 immediately determines that the high-speed photography device A has lost the first frame shot in the first second, and determines that the high-speed photography device A has lost the first frame. The frame with the identification code (08423600100) should be the second frame shot by high-speed camera A in the first second, because the identification code (08423600100) of high-speed camera A is identical to the identification of the second frame of high-speed camera B code (08423600102) and/or the identification code (08423600101) of the second frame of the high-speed camera C is similar.

Then, the image processing device 1 is automatically based on the identification code (08423600100) of the second frame taken by the high-speed photography device A in the first second and the identification code (08423600102) of the second frame of the high-speed photography device B and/or Or the identification code (08423600101) of the second frame of the high-speed photography device C, the group of continuous frames with the identification code taken by the high-speed photography device A in the first second and the frames taken by the high-speed photography device B and C in the first second Each group is continuous and has the frame alignment of the identification code; and for the high-speed photography devices B and C that do not lose the first frame shot in the first second, the image processing device 1 is still based on the first second shot by the high-speed photography device B. The identification code (08423600001) of the first frame and the identification code (08423600003) of the first frame of the high-speed photography device C are combined with the group of continuous frames with the identification code captured by the high-speed photography device B in the first second The group of continuous frames with the identification code captured by the high-speed photography device C in the first second are aligned.

In the same way, when the image processing device 1 finds that one of the frames in a certain group of frames should be used as the alignment point and the frame is missing, it will promptly use the identification code of the next frame (the frame to be used as the alignment point) and other groups. The frame whose identification code is similar among the frames is used as the alignment point, and the frame of the certain group is aligned with other frames of the group.

In summary, the above-mentioned embodiment uses pulse per second (PPS) signal to trigger multiple high-speed photography devices to take pictures at frequency M synchronously to generate multiple sets of frames, and makes these high-speed photography devices simultaneously reset them with pulse per second signal The counter recounts and obtains the standard time, and makes each high-speed photographic device take the count value of the counter and the standard time at the moment of shooting each frame as the identification code of each frame, and then each of the consecutive groups with the The frame of the identification code is provided to the image processing device, whereby the image processing device only needs to use the identification code of the first frame among the groups of consecutive frames with the identification code, and every interval N after the first frame The identification code of the frame can align the groups of continuous frames with the identification code, so that the groups of frames are aligned frame by frame in time, so as to achieve the multiple groups produced by synchronous shooting of multiple high-speed photographic devices. The effect and purpose of consecutive frames aligned with each other.

But the above-mentioned ones are only embodiments of the present invention, and should not limit the scope of implementation of the present invention with this. All simple equivalent changes and modifications made according to the patent scope of the present application and the content of the patent specification are still within the scope of the present invention. Within the scope covered by this patent.

100: frame alignment system 1: Image processing device 3: PPS signal and time providing device A, B, C: high-speed photography device 21: Photographic lens module 22:PPS signal and time supply module 23: Microcontroller 231:Trigger signal generator 232: counter 233: Time register 234: storage unit S1~S5: steps

Other features and functions of the present invention will be clearly shown in the implementation manner with reference to the drawings, wherein: Fig. 1 is the main process steps of frame alignment performed by the novel frame alignment system; Fig. 2 is a schematic block diagram of hardware devices and modules included in an embodiment of the novel frame alignment system; Fig. 3 is a schematic block diagram of detailed hardware elements of the high-speed photographic device of the embodiment shown in Fig. 2; Fig. 4 is a schematic block diagram of hardware devices and modules included in another embodiment of the novel frame alignment system; and FIG. 5 shows the identification codes of each of the three groups of consecutive frames shot in this embodiment and their corresponding relationships in time sequence.

100: frame alignment system

1: Image processing device

A, B, C: high-speed photography device

21: Photographic lens module

22:PPS signal and time supply module

23: Microcontroller

Claims (6)

A frame alignment system comprising: an image processing device; and A plurality of high-speed photography devices capable of communicating with the image processing device, and these high-speed photography devices take pictures synchronously according to a pulse signal per second, and shoot at a frequency M (M is a positive integer) to generate multiple sets of continuous frames; And each of the high-speed photography devices resets a counter according to the per-second pulse signal and obtains a current standard time at the same time; each of the high-speed photography devices takes a count value of the counter and the standard time at the moment of shooting each frame as an identification code for each of the frames, and then provide each of the group of consecutive frames with the identification code to the image processing device; The image processing device aligns the groups of consecutive frames with the identification code according to the identification code of the first frame of the groups of consecutive frames with the identification code provided by each of the high-speed photography devices. The frame alignment system according to claim 1, wherein when the image processing device finds that a certain group of continuous frames with the identification code lacks the first frame, the image processing device will The identification code of the second frame of the frame and the identification code of the second frame of the other group of frames with the identification code, and the identification code of a certain group of consecutive frames with the identification code and other groups with the identification code frame alignment. The frame alignment system as described in claim 1 or 2, wherein, the image processing device also aligns the groups of consecutive frames with the identification code at intervals of N frames according to the identification code of each frame A group of consecutive frames with the identification code, where N is a positive integer and N≥0. The frame alignment system as described in Claim 1, wherein when each of the high-speed photography devices resets the counter therein again according to the pulse-per-second signal, the standard time will be updated at the same time. The frame alignment system as described in claim item 1, wherein each of the high-speed photography devices includes a photography lens module, a pulse signal and time supply module and a microcontroller, and the pulse signal and time supply module can generate the pulse signal per second and can obtain the current standard time at the geographic location, the microcontroller is electrically connected to the pulse signal and time providing module to request the pulse signal and time providing module to provide the pulse signal per second and The standard time; and the microcontroller includes a trigger signal generator, the counter, a time register and a storage unit, the trigger signal generator generates a trigger signal with the frequency M according to the pulse signal per second to trigger the The camera lens module takes pictures synchronously with the frequency M, the time register temporarily stores the standard time, and the storage unit temporarily stores each group of consecutive frames with the identification code. The frame alignment system as described in claim 1 also includes a pulse signal and time providing device capable of wireless communication with each of the high-speed photography devices, and the pulse signal and time providing device will provide the pulse signal per second and the geographic location The current standard time of the position is given to these high-speed photography devices; and each of the high-speed photography devices includes a photography lens module and a microcontroller, and the microcontroller includes a trigger signal generator, the counter, and a time register and a storage unit, the trigger signal generator generates a trigger signal with the frequency M according to the pulse signal per second to trigger the photography lens module to take pictures synchronously with the frequency M, the time register temporarily stores the standard time, and the storage The unit temporarily stores each group of consecutive frames with the identification code.

Images