JP2011124814A - Image synchronization apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image synchronization apparatus that synchronizes image signals generated by a plurality of image signal sources using simple method and structure. <P>SOLUTION: The image synchronization apparatus 20 has an image memory 40 which stores image signals for every channel, a frame number cue 50 which stores frame numbers of stored images, and a timing controller 60 which synchronizes output images. The timing controller 60 compares frame numbers stored in the frame number cue 50 between channels to determine a next output image signal frame number. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention relates to an image synchronization apparatus and a synchronization control apparatus for synchronizing and displaying images generated by a plurality of image signal sources on one screen.

  When adjacent image signals of the same object are generated by a plurality of image signal sources, and the images based on these image signals are combined and displayed, the scenes at the same time in the images are simultaneously displayed on the display device. It is necessary to

  At this time, in the image signal format normally used in the image equipment or the computer, the values of the pixels constituting one image are transmitted in time series in a predetermined order. It is necessary to make the start timings coincide with each other.

For example, let us consider a case where one screen is divided and reproduced by four image reproduction devices serving as image signal sources, and image signals of the four divided images are directly input to the projector, and are arranged vertically and horizontally and output as a composite screen.
In this case, as shown in FIG. 10, even if the composite screen G is normally displayed at time t <b> 1, if the image signals displayed on g <b> 1 to g <b> 4 of each channel 1 to 4 are not synchronized, There is a problem that a time difference occurs between the divided images g1 to g4 of the composite screen G at time t2.

  From the viewpoint of eliminating the above problems, an image synchronization apparatus having a synchronization function with other image signal sources is realized so that the start timings of the divided images generated by the image signal sources are matched. (Patent Document 1 etc.).

  The image synchronization apparatus described in Patent Document 1 is arranged between each image signal source and a projector, temporarily stores a plurality of image signals having different start timings in a frame memory, and uses timing information provided from each image signal source. Based on this, they are output to one projector at the timing of synchronizing them.

JP 2004-228747 A

However, the timing information related to image generation given from the image reproduction device is not always accurately related to the timing of the image signal input to the projector.
For example, in the case of a real-time rendering system that generates an image in real time from a three-dimensional CG model, as shown in FIG. 11, the timings that can be grasped by the control program of the system are the timings t1, t2, It may be t3 and not the output timing t1 ′, t2 ′, t3 ′ of the image signal reflecting the drawing command.

  Furthermore, the time from when a rendering command is given to when an image signal reflecting the rendering command is output may vary depending on the performance and load state of the rendering engine. As a result, as shown in FIG. 12, not only the timing grasped by the control program and the output timing of the image signal are different, but also images are given at different timings although the drawing command is given to each channel at the same timing. A signal may be output.

  Similarly, as shown in FIG. 13, when a plurality of image reproduction devices (players) 602, 603, and 604 serving as image signal sources are to be synchronized and reproduced by an external command from a computer 601, each image reproduction device is provided with each image reproduction device. Even when a playback command is given at the same time, the playback operation of each image playback device is not always started at the same time due to a command transmission time, a response time difference with respect to the command, or the like. Even when it is possible to inquire about the current reproduction position from the computer, an error may occur due to the accuracy of the value to be returned and the delay of the command or response.

  In particular, when a personal computer or image playback device for general users is used instead of a professional image playback device having a synchronized playback function, the scenes at the same time of the image signals generated by a plurality of image signal sources are simultaneously displayed. It is difficult to control to be displayed on the display device.

  The present invention has been made in consideration of the above circumstances, and an object of the present invention is to provide an image that can synchronize each image signal generated by a plurality of image signal sources regardless of the characteristics of the image signal source device. It is to provide a synchronization device.

  In order to achieve the above object, an invention according to claim 1 is an image synchronizer for synchronizing image signals of the same number of channels as the number of image signal sources transmitted from each of a plurality of image signal sources. An image signal for eliminating a time difference between the channels based on a timing signal included in the image signal of each channel or a timing signal transmitted from the plurality of image signal sources together with the image signal. A timing control means is provided.

  According to a second aspect of the present invention, the timing signal is a frame number of the image signal, and the image synchronizer stores the image signal supplied from the image signal source in units of frames for each channel. An image memory comprising an arbitrary number of memory areas, and a frame number queue comprising the same number of levels as the number of the memory areas for storing the frame numbers included in the image signal, and eliminating the time difference Is performed on the basis of the comparison result by comparing the frame numbers stored in the frame number queue between different channels.

  In the invention corresponding to claim 3, the elimination of the time difference performed based on the comparison result is performed by searching for the same frame number in a level below a specific level of the frame number queue, If a common frame number is found, the frame number is determined as the next frame to be output.

  Further, in the invention corresponding to claim 4, the elimination of the time difference performed based on the comparison result is performed by searching for the same frame number within a level below a specific level of the frame number queue, If a common frame number is not found in the channel number, the channel number in which the latest frame number is stored in the frame number queue is compared for all channels, and the channel in which the smallest frame number is stored ΔFm, which is the difference between Fm and Fn, when the level 0 frame number of n is Fn and the level 0 frame number of another channel m (m ≠ n) is Fm, the image of the channel m is obtained. A command to pause image output for a period corresponding to ΔFm is transmitted to the signal source, and again within a level below a specific level of the frame number queue. Thus, the same frame number is searched, and when a common frame number is found in all channels, the frame number is determined as the next frame to be output.

  In the image synchronization apparatus of the present invention, other image signals are obtained by performing image signal timing control for eliminating the time difference between the image signals received from the respective image signal sources based on the timing signals received from the respective image signal sources. It is possible to synchronize each image signal generated by a plurality of image signal sources having no synchronization function with respect to the source, and to prevent the occurrence of a time difference.

  In the image synchronizer of the present invention, each image signal is stored in the image memory in units of frames, and at the same time, the frame number is stored in the frame number queue, and the frame number stored in the frame number queue is set between the channels. By selecting the next output frame, it is possible to synchronize the image signals generated by multiple image signal sources with a simple method and configuration, and prevent the occurrence of time differences between channels. can do.

1 is a schematic diagram illustrating a configuration of an image display system including an image synchronization apparatus according to the present invention and a plurality of image signal sources. It is a figure showing the image signal input into the image synchronizer concerning this invention. It is a figure showing the structure of the image memory inside the image synchronizer concerning this invention. It is a figure for demonstrating the method to store an image signal in the image memory inside the image synchronizer concerning this invention. It is a figure for demonstrating the method to store a frame number in the frame number queue inside the image synchronization apparatus concerning this invention. It is a flowchart which shows the method of searching the inside of a frame number queue | queue. It is a figure showing a storage pattern of a frame number queue for explaining a method of searching the inside of the frame number queue. It is a figure showing a storage pattern of a frame number queue for explaining a method of searching the inside of the frame number queue. It is a flowchart which shows the recovery procedure from the synchronization impossible when the same frame number is not found by the search method shown in FIG. It is a figure for demonstrating the subject at the time of combining the image output from several image signal sources. It is a figure for demonstrating the output timing of the image signal in a real-time rendering system. It is a figure for demonstrating the output timing of the image signal in a real-time rendering system. FIG. 10 is a diagram for explaining a problem in the case where a plurality of image reproduction apparatuses are synchronously reproduced from one computer.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a schematic diagram illustrating a configuration of an image display system including a plurality of image signal sources, an image synchronization device, and a plurality of projectors. The example of the image display system shown in FIG. 1 includes four image signal sources 101 to 104, one image synchronizer 20, and four projectors 301 to 304.
Then, one synthesized image is displayed by arranging the output images of the four projectors adjacent to each other.
In FIG. 1, the devices 101 to 104 and 301 to 304 and the internal functional blocks thereof have the same channel (signal system) in which the last one digit (1 to 4) is equal. Note that these four channels are merely examples, and it is needless to say that any two or more channels can be applied to any number of channels. Each device is not limited to being realized as a dedicated device, and may be realized as a general-purpose computer such as a personal computer in which a program for realizing the function is read from a storage medium or a network and installed. Good. The same applies to the following embodiments.

  Here, the image signal sources 101 to 104 generate image signals (image information) for displaying the divided images for each frame for each of the channels 1 to 4, and output them to the image synchronization apparatus 20. Here, the output timing to the image synchronization apparatus 20 may differ between the channels.

  Each frame of the image or a signal output integrally with each frame of the image includes information for specifying the time when the frame should be displayed. The information for specifying the time corresponds to the timing signal in claim 1, the content of which is an image output to each channel, such as the frame number sequentially assigned to each image frame, the elapsed time from the start of reproduction, etc. The format is not limited as long as it is information that can determine the temporal relationship of signals.

  And the image synchronization apparatus 20 has the timing control part 60 for synchronizing the output image from the image signal sources 101-104 based on the information for specifying the said time, About the timing control method, Details will be described below. The timing controller 60 corresponds to the image signal timing control means in claim 1.

  First, the images output from the image signal sources 101 to 104 include the frame number of each image frame as information for specifying the time. FIG. 2 shows an example in which the frame number of each frame is added to the pixel value of the first raster of the image signal. That is, in FIG. 2, the frame number 25, frame number 26, and frame 27 number of channel 1 are associated with the pixel value of the first raster of the image signal of that frame.

  On the other hand, the image synchronization apparatus 20 includes an image memory 40 that can store a plurality of continuous image frames for each channel. The image memory includes memory areas 401 to 404 for each channel. In the example shown in FIG. 3, an image memory for 8 frames is provided for each of the four channels.

  The image synchronizer 20 stores the image signals input from the image signal sources 101 to 104 of the channels 1 to 4 in the image memory areas 401 to 404 shown in FIG. 3 for each frame. FIG. 4 shows an example of a method of selecting the image memories 0 to 7 to be written based on the values 0 to 7 of the lower 3 bits of the frame numbers 25 to 27, but the image memories 0 to 7 are sequentially added for each frame. Any method can be used as long as it can hold the most recent multiple image frames, such as selection.

The image synchronization apparatus 20 has a frame number queue 50 that can store frame number information for image signals input from each channel.
This is for holding the accumulation order of the image frames accumulated in the image memory, and the frame number corresponding to each image frame accumulated in the image memory is stored in a frame number queue provided for each channel. In this example, an eight-level cue corresponding to an image of eight image frames is provided. Level 0 holds the frame number of the latest image frame, and level 1 holds the frame number of the image frame one frame before the latest frame. The same applies hereinafter.
In FIG. 5, for example, the frame number 15 of the latest image frame 15 of channel 1 stored in the image memory 401 is held at level 0 of the frame number queue 501 of channel 1 and the frame number of the previous frame 14 is stored. 14 indicates that it is held at level 1 of the frame number queue 501 of channel 1.

Next, every time an image signal for one frame is output, the image synchronizer 20 performs a search for determining a frame to be output next from the accumulated frames. The procedure is shown in the flowchart of FIG.
First, the search depth D = 0 is set (S11).
The range from level 0 to level D of the queues 501 to 504 of each channel is searched (S12), and when the same frame number is held for all four channels (S13), the frame is transferred to the next image synchronizer. The output frame from 20 is determined and the search is completed (S14).
If the same frame number is not found in the range from level 0 to level D, then the search depth D = D + 1 is set, and the search is repeated (S17).
If the search is not completed even when the search depth D = 7 (S15), it is determined that synchronization is impossible (S16).

For example, when the queue is in the state shown in FIG. 7, it is determined that synchronization is impossible because the same frame number does not exist even when searching up to level 7. Furthermore, it can be determined from channel number comparison between channels at level 0 that channel 2 is 11 frames ahead of other channels 4.
In this case, the image synchronization apparatus 20 issues a command to the image signal source 102 corresponding to the channel 2 so as to pause for 11 frame times.

  FIG. 8 shows the state of the queue after the elapse of 11 frame times. The preceding state of channel 2 is canceled, and the time difference of all channels becomes about 1 frame, so that the synchronization is disabled.

FIG. 9 shows a generalized process when synchronization is impossible.
The channel n that is most delayed (the latest frame number held at level 0 is the smallest) is identified, and the latest frame number is set to Fn (S21).
Then, for all channels m other than channel n, the latest frame number Fm and the difference ΔFm between Fn are obtained (S22).
Since the image generation operation for channel m precedes channel n by ΔFm, a pause command for ΔFm frame time is issued to the image signal source for channel m (S23).

If the processing when synchronization is impossible shown in FIG. 9 is applied to the case of FIG. 7, the most delayed channel is the channel 4 (n = 4) with the latest frame number held at level 0, so The latest frame number F4 is 14.
When the differences between the latest frames F1, F2, F3, and F4 are obtained for all other channels 1, 2, and 3, ΔF1 = 1, ΔF2 = 11, and ΔF3 = 1.
Therefore, a pause command of 1 frame time for channel 1, 11 frame time for channel 2, and 1 frame time for channel 3 is issued to each image signal source.

  When ΔFm is less than 7, synchronization is possible by the search procedure shown in FIG. 6, so the temporary stop command in S23 of FIG. 9 may be issued only when ΔFm ≧ 7.

  As described above, according to the present invention, each image signal generated by a plurality of image signal sources having no synchronization function can be synchronized by using the image synchronization device without using any special measure as a signal source. In addition, it is possible to prevent the occurrence of a time difference in units of frames and to eliminate image shift between channels.

  101-104: Image signal source, 20: Image synchronization device, 301-304: Projector, 40: Image memory, 401-404: Image memory area, 50: Frame number queue, 501-504 ... Cue, 60... Timing control unit, 601... Computer, 602 to 604... Image reproduction apparatus, 701.

Claims (4)

An image synchronization apparatus for synchronizing image signals of the same number of channels as the number of image signal sources transmitted from each of a plurality of image signal sources,
Image signal timing control means for eliminating a time difference between the channels based on a timing signal included in the image signal of each channel or a timing signal transmitted from the plurality of image signal sources integrally with the image signal. With
An image synchronizer characterized by that. The timing signal is a frame number of the image signal;
The image synchronizer for each channel,
An image memory composed of an arbitrary number of memory areas for storing image signals supplied from the image signal source in units of frames;
A frame number queue configured to store the frame number included in the image signal and having the same number of levels as the number of the memory areas;
2. The image synchronization apparatus according to claim 1, wherein the elimination of the time difference is performed on the basis of a comparison result obtained by comparing frame numbers stored in the frame number queue between different channels.   The elimination of the time difference performed based on the comparison result is performed by searching for the same frame number within a level below a specific level of the frame number queue, and when a common frame number is found in all channels. 3. The image synchronization apparatus according to claim 2, wherein the image synchronization apparatus is performed by determining a frame having the frame number as a frame to be output next. The elimination of the time difference performed based on the comparison result is performed when the same frame number is searched in a level lower than a specific level of the frame number queue and a common frame number is not found in all channels. Is
The level 0 frame number in which the latest frame number is stored in the frame number queue is compared in all channels,
This is the difference between Fm and Fn, where Fn is the level 0 frame number of channel n storing the smallest frame number and Fm is the level 0 frame number of the other channel m (m ≠ n). Find ΔFm,
A command to pause image output for a period corresponding to ΔFm to the image signal source of the channel m;
When the same frame number is searched again in a level below a specific level of the frame number queue and a common frame number is found in all channels, the frame with the frame number is output next. Done by deciding,
The image synchronizer according to claim 2.

Images