JP2012191284A - Image transmitter, image transmission method, image receiver, and image reception method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem of a conventional technique in which processing on an image receiving side is complicated in order to simultaneously display images received from plural image transmitters.SOLUTION: The image transmitter is provided with: compression means for digital compression coding of an image signal captured with a fixed period with a reference signal; network processing means for transmitting via a network the digitally compression-coded image signal to the image receiver connected via the network; and control means which controls the compression means and the network processing means. In response to a request from an image receiver, the control means changes a processing period from digital compression coding of the image signal to transmission of the image signal to the network.

Description

The present invention relates to an apparatus for transmitting video.

Regarding the above technical field, for example, Patent Document 1 discloses a transmission apparatus having a function of adjusting a display time when transmitting an image via a network.

JP-A-9-51515

However, the technique described in Patent Document 1 has a problem that the processing on the video receiving side is complicated because the video received from a plurality of video transmitting apparatuses is displayed simultaneously.

Therefore, in this specification, for example, the video transmission device controls the output delay time in accordance with the control of the video reception device.

ADVANTAGE OF THE INVENTION According to this invention, the video transmission system which considered output delay time can be provided.

It is a figure which shows an example of the video transmission system containing a video transmitter and a video receiver. It is a figure which shows an example of an internal block structure of a video transmission apparatus. It is a figure which shows an example of the digital compression process of a video transmitter. It is a figure which shows an example of the digital compression video signal of a video transmitter. It is a figure which shows an example of the packet of the digital compression video signal of a video transmitter. It is a figure which shows an example of the LAN packet of a video transmission apparatus. It is a figure which shows an example of an internal block structure of a video receiver. It is a figure which shows another example of the internal block structure of a video receiver. It is a figure which shows an example of the flowchart of the delay time confirmation process of a video receiver. It is a figure which shows an example of the flowchart of the delay time reply process of a video transmitter. It is a figure which shows an example of the flowchart of the delay time setting process of a video receiver. It is a figure which shows an example of the flowchart of the delay time setting process of a video transmission device. It is a figure which shows an example of the transmission process timing of a video transmitter, and the reception process timing of a video receiver. It is a figure which shows another example of the transmission process timing of a video receiver, and the reception process timing of a video receiver.

  FIG. 1 is an example of a video transmission system including a camera which is a video transmission apparatus. In FIG. 1, 1 is a camera and 2 to 3 are other cameras. Reference numeral 4 denotes a LAN (Local Area Network), 5 denotes a controller, and the cameras 1 to 3 are connected to the controller 5 via the LAN 4. Reference numeral 6 denotes a display. In the network, as a protocol to be used, for example, a method stipulated in the IEEE 802.3 standard that is a data link protocol may be used, or a network protocol IP (Internet Protocol) is used, and a higher level transport is used. As the protocol, TCP (Transmission Control Protocol) and UDP (User Datagram Protocol) may be used. A higher-order application protocol such as RTP (Real-time Transport Protocol) or HTTP (Hyper Text Transfer Protocol) is used for transmission of video and audio. In addition, a protocol method defined in the IEEE 802.3 standard may be used. The controller 5 receives video and audio data distributed from each camera, and outputs video and audio to the display 6 and the speaker 7, respectively. As the configuration of the LAN 4, for example, each camera and the controller 5 are directly connected in a one-to-one relationship, or are connected via a switching hub device (not shown), and two or less cameras or four or more cameras can be connected. It is.

  FIG. 2 is a diagram illustrating an example of an internal block configuration of the camera 1 that is a video transmission apparatus. Reference numeral 100 denotes a lens, 101 an image sensor, 102 a video compression circuit, 103 a video buffer, 104 a system encoder, 105 a packet buffer, 106 a reference signal generation circuit, 107 a LAN interface circuit, 108 a control circuit, 109 It is memory.

  A video signal obtained by the imaging device 101 via the lens 100 is input to the video compression circuit 102, color tone and contrast are corrected, and stored in the video buffer 103. Next, the video compression circuit 102 reads out the video data stored in the video buffer 103 and, for example, ISO / IEC13818-2 (commonly referred to as MPEG2 Video) MP @ ML (Main Profile @ Main Level) standard as a video compression encoding system. Compressed encoded data that conforms to the above is generated. Other video compression encoding methods include H.264. The H.264 / AVC standard method or the JPEG standard method may be used. Also, cameras of different video compression encoding methods may be mixed, or one camera may select and switch the video compression encoding method. The generated compressed encoded video data is input to the system encoder 104. The reference signal generation circuit 106 supplies, for example, a frame pulse indicating a break of the video signal frame to the image sensor 101 and the video compression circuit 102 as a reference signal serving as a reference for processing timing of the image sensor 101 and the video compression circuit 102. In accordance with this reference signal, the image pickup by the image sensor, the compression of the shot image, and the transmission of the compressed image (described later) are performed. This reference signal is a signal synchronized between the cameras. As a synchronization method, for example, there is a method of inputting a synchronization signal of one camera to another camera.

  Next, the compression encoded video data input to the system encoder 104 is packetized as shown below.

  FIG. 3 shows an example of digital compression processing. Intra-frame data compressed in units of frames of a digital compressed video signal and inter-frame data in which only difference information is compressed using predictions from previous and subsequent frame data. It is a relationship. 201 is an intra frame and 202 is an inter frame. The digital compressed video signal has a predetermined number of frames, for example, 15 frames as one sequence, the head of which is an intra frame, and the remaining frames are inter frames compressed using prediction from the intra frame. Of course, an intra frame other than the head may be arranged. Also, only the first frame may be an intra frame, and all subsequent frames may be inter frames, or all the frames may be intra frames.

  FIG. 4 shows the configuration of the digital compressed video signal. 302 is a picture header added in units of frames, and 301 is a sequence header added in units of sequences. The sequence header 301 includes information such as a synchronization signal and a transmission rate. The picture header 302 includes a synchronization signal and identification information such as an intra frame or an inter frame. Usually, the length of each data changes with the amount of information. This digital video compression signal is divided into transport packets to be described later to form a packet sequence.

  FIG. 5 is a configuration example of a transport packet of a digital video compression signal. Reference numeral 40 denotes the transport packet, and one packet is composed of a fixed length, for example, 188 bytes, and is composed of a packet header 401 and packet information 402. The digital compressed video signal described with reference to FIG. 4 is divided and arranged in an area of packet information 402, and the packet header 401 is configured by information such as the type of packet information.

  The digital video compressed signal packetized by the system encoder 104 is temporarily stored in the packet buffer 105, and the packet sequence read from the packet buffer 105 is input to the LAN interface circuit 107.

  The LAN interface circuit 107 in FIG. 2 packetizes the input packet sequence into, for example, a LAN packet that conforms to the IEEE 802.3 standard and outputs the packet.

  FIG. 6 is a diagram illustrating an example of LAN packetization of a packet sequence generated by the system encoder 104. The LAN packet 60 has, for example, one packet having a variable length of a maximum of 1518 bytes, and includes a LAN packet header 601 and LAN packet information 602. The transport packet 40 generated by the system encoder 106 is stored in the area of the LAN packet information 602 together with a data error detection code in accordance with the network protocol described above, and address information on the LAN 4 for identifying each camera is stored. A LAN packet header 601 is added and output as a LAN packet 60 to the LAN.

  The LAN interface circuit 107 exchanges information for control with devices connected to the LAN 4. This is because information such as an instruction from the control circuit 108 is stored in the LAN packet information 602, information is transmitted from the LAN packet 60 of the LAN packet 60 transmitted or received from the LAN 4, and transmitted to the control circuit 108. Done in

  FIG. 7 is a diagram illustrating an example of an internal block configuration of the controller 5. 5011 to 5013 are LAN interface circuits, 5021 to 5023 are system decoders, 5031 to 5033 are video decompression circuits, 504 is an image processing circuit, 505 is an OSD (On-Screen Display) circuit, 506 is a reference signal generation circuit, and 507 is a control. A circuit 508 is a memory.

  In the description of FIG. 7, the system decoders 5021 to 5023, the video expansion circuits 5031 to 5033, and the image processing circuit 504 are described as hardware. However, these functions can also be realized by software by causing the control circuit 507 to develop and execute a program having functions corresponding to each in the memory 508. In the following, for simplification of description, the system decoders 5021 to 5023, the video expansion circuits 5031 to 5033, and the image processing circuit 504 perform each process including the case where the control circuit 507 executes a program corresponding to each function. Explain to be executed.

  LAN packets 60 generated by the cameras 1 to 3 are input to the LAN interface circuits 5011 to 5013, respectively. In the LAN packet 60 input from the camera 1, the LAN packet header 601 is removed by the LAN interface circuit 5011, and the transport packet 40 is extracted from the LAN packet data 602 according to the network protocol described above. The transport packet 40 is input to the system decoder 5021, and the packet information 402 described above is extracted from the transport packet 40 and combined into the digital compressed video signal shown in FIG. This digital compressed video signal is subjected to expansion processing in a video expansion circuit 5031 and is input to the image processing circuit 504 as a digital video signal. The same processing is performed for the LAN packet 60 input from the cameras 2 and 3, and digital video signals are input from the video expansion circuits 5032 and 5033 to the image processing circuit. The image processing circuit 504 performs distortion correction of the video signal from each camera, viewpoint conversion by coordinate replacement, synthesis processing, etc., and outputs to the OSD circuit 505 or recognition of the object shape by the video signal from each camera, distance Image processing such as measurement is performed. The OSD circuit 505 superimposes characters and figures on the video signal from the image processing circuit 504 and outputs it to the display 6.

  The reference signal generation circuit 506 supplies, for example, a frame pulse indicating a video signal frame break to the image processing circuit 504 and the OSD circuit 505 as a reference signal serving as a reference for processing timing of the image processing circuit 504 and the OSD circuit 505. The reference signal is generated with reference to, for example, the time when the video expansion process for one frame is completed, and the reference signal is adjusted by the control circuit 507 controlling the reference signal generation circuit 506.

  Further, in the LAN interface circuits 5011 to 5013, in order to exchange information for control with each camera, information such as an instruction from the control circuit 507 is stored in the LAN packet information 602 and transmitted to each camera. Information is extracted from the LAN packet information 602 of the LAN packet 60 received from each camera and transmitted to the control circuit 507.

  FIG. 8 is a diagram illustrating another example of the internal block configuration of the controller 5. A LAN interface circuit 501 is connected to the cameras 1 to 3 via a switching hub device (not shown). The LAN interface circuit 501 discriminates the LAN packet from each camera from the address information stored in the LAN packet header 601 described above, and the transport extracted from the LAN packet information 602 of the LAN packet 60 according to the network protocol described above. The packet 40 is distributed to the system decoders 5021 to 5023 and output. The processing after the system decoders 5021 to 5023 is the same as the description of FIG.

  Further, in the LAN interface circuit 501, in order to exchange information for control with each camera, information such as an instruction from the control circuit 507 is stored in the LAN packet information 602 and transmitted to each camera, or each camera The information is extracted from the LAN packet information 602 of the LAN packet 60 received from, and transmitted to the control circuit 507.

  FIG. 9 is a flowchart of the delay time acquisition process by the controller in this embodiment. The controller 5 first confirms the camera connected to the LAN 4 (step S101). This can be realized by, for example, a broadcast packet that can transmit a packet to all devices connected to the LAN 4. Further, a confirmation packet may be transmitted to each camera individually. Next, each camera connected to the LAN 4 is inquired about the processing delay time of each camera (step S102), and the response of the processing delay time from each camera is received (step S103). Thereby, the controller 5 can acquire the processing delay time of the camera connected to the LAN 4. These processes are performed, for example, when the controller 5 is turned on.

  FIG. 10 is a flowchart of the delay time answer process in the camera according to this embodiment. As described above, when a delay time inquiry request is received from the controller 5 (step S301), a delay time that can be set by the camera, for example, a range from the shortest delay time to the longest delay time that can be set is given to the controller 5. It transmits as a reply (step S302). As a result, the camera connected to the LAN 4 can transmit the processing delay time of the camera to the controller. The camera calculates the shortest delay time based on the compression method of the video to be acquired and the bit rate of the video before the request from the controller 5 or in response to the request from the control 5. And the shortest delay time is read from the memory 109 as necessary and notified to the controller 5 as described above. When the camera calculates the shortest delay time in response to a request from the controller 5, there is an effect that the shortest delay time can be calculated according to the video compression method and bit rate at the time of the request. This is particularly effective when the controller 5 can instruct the camera to change the compression method or bit rate.

  FIG. 11 is a flowchart of the delay time setting process by the controller. First, the processing delay time to be set is determined (step S201). Here, the longest time among the shortest delay times of each camera obtained by the delay time acquisition process of FIG. 9 is set as the processing delay time set for each camera. However, it is a requirement that the processing delay time shorter than the shortest time among the longest delay times of each camera is set in the camera.

  If this requirement is not satisfied, the controller 5 transmits a request for shortening the shortest delay time to the camera that has transmitted the shortest delay time that is not satisfied, and the longest delay to the camera that has transmitted the longest delay time that is not satisfied. Send time extension request. A camera that has received a request for shortening the shortest delay time can attempt to shorten the shortest processing time by, for example, changing the compression processing method. The controller 5 determines whether the shortest delay time and the longest delay time received from each camera in response to the shortening request satisfy the above requirements. If the requirement is still not met, the controller 5 outputs an error. When the requirement is satisfied, the controller 5 sets the shortest delay time shortened by the shortening request as a processing delay time set for each camera.

  Next, the controller 5 requests each camera to set the determined processing delay time (step S202), and receives an answer of the setting result from each camera (step S203). Thereby, the controller 5 can set the processing delay time for the camera connected to the LAN 4.

  FIG. 12 is a flowchart of the delay time setting process in the camera according to the present embodiment. As described above, when a delay time setting request is received from the controller 5 (step S401), the camera sets a delay time (step S402), and transmits the result as a reply to the controller (step S403). As a result, the camera connected to the LAN 4 can set the processing delay time in response to a request from the controller.

  FIG. 13 is a diagram illustrating an example of the transmission processing timing of each camera and the reception processing timing of the controller 5 in the present embodiment. In the figure, (1-1) to (1-4) are processing timings of the camera 1, (2-1) to (2-5) are processing timings of the camera 2, and (3-1) to (3-8). ) Shows the processing timing of the controller 5.

  (1-1) is the reference signal 1, (1-2) is the imaging timing 1 when the imaging process is performed by the imaging device 101, and (1-3) is the video compression process by the video compression circuit 102. Video compression timing 1 and (1-4) are transmission timing 1 at which transmission processing by the LAN interface circuit 107 is performed. Here, a video signal for one frame is processed for each reference signal. The camera 1 uses the reference signal 1 as a processing reference, starts imaging processing, for example, at the timing of the pulse of the reference signal 1, and then sequentially performs video compression processing and transmission processing. In the camera 1, a time d1 from the reference signal 1 to the start of transmission processing at the transmission timing 1 is a processing delay time.

  Also, (2-1) is the reference signal 2 of the camera 2, (2-2) is the imaging timing 2 at which imaging processing by the imaging device 101 of the camera 2 is performed, and (2-3) is the video compression circuit 102. (2-4) is a transmission timing 2 at which transmission processing by the LAN interface circuit 107 is performed when the processing delay time is not set in the camera 2. The camera 2 starts imaging processing at the timing of the reference signal 2 using the reference signal 2 as a processing reference, and then sequentially performs video compression processing and transmission processing. In the camera 2, a time d2 from the reference signal 2 to the transmission timing 2 is a processing delay time. Further, as described above, the reference signal 1 of the camera 1 and the reference signal 2 of the camera 2 are synchronized.

  Here, the controller 5 acquires the processing delay times of the cameras 1 and 2 as described above. As a result, since the processing delay time of the camera 1 is longer than the processing delay time d2 of the camera 2, the controller 5 sets the processing delay time of the camera 2 so that the processing delay time is d1 for the camera 2. . (2-5) is the transmission timing 2 'after the processing delay time is set. Here, the adjustment of the processing delay time can be realized, for example, by adjusting the timing of reading out the packet sequence stored in the packet buffer 105 from the system encoder 104 to be input to the LAN interface circuit 107, as shown in FIG. Thereby, the transmission timing 1 of the camera 1 and the transmission timing 2 ′ of the camera 2 coincide.

  Next, (3-1) is reception timing 1 at which the controller 5 performs reception processing of the LAN packet from the camera 1, and (3-2) is video expansion timing at which video expansion processing by the video expansion circuit 5031 is performed. 1, (3-3) is the video output timing 1 of the camera 1 for one frame obtained by the video decompression circuit 5031. Also, (3-4) is the reception timing 2 when the controller 5 is receiving the LAN packet from the camera 2, and (3-5) is the video expansion timing when the video expansion circuit 5032 is performing the video expansion processing. 2 and (3-6) are the video output timing 2 of the camera 2 for one frame obtained by the video expansion circuit 5032. Further, (3-7) is the reference signal C in the controller 5, and (3-8) is the display timing C of the display video that the controller 5 outputs to the display 6.

  The controller 5 uses the reception timing 1 from the camera 1 as a processing reference, and sequentially performs the video expansion processing following the reception processing. Similarly, the video expansion process is performed following the reception process from the camera 2. Here, since the transmission timing 1 of the camera 1 and the transmission timing 2 ′ of the camera 2 match, the video output timing 1 and the video output timing 2 match. For example, the reference signal C is generated in accordance with the video output timings 1 and 2, and display processing is performed at the pulse timing of the reference signal C, so that, for example, the video of the camera 1 and the video of the camera 2 are combined, and the display timing C This makes it possible to display the synthesized video on the display 6.

  FIG. 14 is a diagram illustrating another example of the transmission processing timing of each camera in the present embodiment. The controller 5 sets the processing delay time of the camera 2 so that the processing delay time is d1 for the camera 2. In this example, the video compression is performed so that the camera 2 has the processing delay time d1. Adjust the timing to start processing. This adjustment of the processing delay time can be realized, for example, by adjusting the timing at which the video compression circuit 102 reads out the video data stored in the video buffer from the video compression circuit 102 shown in FIG. (2-6) is the video compression timing 2 ′ after the processing delay time is set, and (2-7) is the transmission timing 2 ″ associated therewith. As a result, the transmission timing 1 of the camera 1 and the transmission timing 2 ″ of the camera 2 coincide. Accordingly, similarly to FIG. 13, the video of the camera 1 and the video of the camera 2 can be synthesized and the synthesized video can be displayed on the display 6 at the display timing C.

  In the above description, the processing time delay time is defined as the reference signal that is the starting point to the transmission start time that is the ending point. However, the present invention is not limited to this. For example, the starting point may be the time when the image sensor 101 starts imaging. The end point may be the transmission end time of the transmission timing of each frame.

  Further, for example, the video output timing of each camera can be matched by adding the difference in video expansion processing time due to the difference in compression method or bit rate for each camera to the setting processing time for the camera. In this case, the controller 5 measures the video expansion processing time for each camera, sets the difference from the longest video expansion processing time as an additional processing delay time, and adds the processing delay time of each camera to each camera. By transmitting as a processing delay time and instructing each camera to set a new processing delay time, the video output timing ((3-3), (3-6), etc.) of each camera in the controller 5 can be more accurately determined. It is possible to align.

  In addition, an example in which the processing delay time of each camera is acquired by an inquiry from the controller 5 has been shown. However, for example, when the camera is turned on or connected to the LAN 4, the camera 5 receives the controller 5 from the camera side. You may be notified.

  In the above description, transmission / reception of a video signal has been described. However, transmission of an audio signal is also possible.

  As described above, by adjusting the delay time in each camera, it is possible to display an image with a suitable display timing.

  Further, since the controller 5 does not need to perform processing to absorb the display timing deviation of the video from each camera, it is possible to display the video with the display timing matched without complicating the processing.

1, 2, 3 ... Camera 4 ... LAN
5 ... Controller 6 ... Display 100 ... Lens 101 ... Image sensor 102 ... Video compression circuit 103 ... Video buffer 104 ... System encoder 105 ... Packet buffer 106 ... Reference signal generation circuit 107 ... LAN interface circuit 108 ... Control circuit 201 ... Intra frame 202 ... interframe 301 ... sequence header 302 ... picture header 40 ... transport packet 401 ... packet header 402 ... packet information 501, 5011, 5012, 5013 ... LAN interface circuits 5021, 5022, 5023 ... system decoders 5031, 5032, 5033 ... video Expansion circuit 504 ... Image processing circuit 505 ... OSD circuit 506 ... Reference signal generation circuit 507 ... Control circuit 60 ... LAN packet 601 ... LAN packet header 6 2 ... LAN packet information

Claims (13)

A compression means for digitally compressing and encoding a video signal imaged at a fixed period by a reference signal;
Network processing means for transmitting the digital compression-encoded video signal to the video receiver connected to the network via the network;
Control means for controlling the compression means and the network processing means;
With
The video transmission device according to claim 1, wherein the control means changes a processing time from digital compression coding of the video signal to transmission to the network in response to a request from the video reception device. The video transmission device according to claim 1,
The video transmission apparatus according to claim 1, wherein the control unit changes a compression start time of the compression unit with respect to the reference signal. The video transmission device according to claim 1,
The video transmission apparatus, wherein the control unit changes a transmission processing start time of the network processing unit with respect to the reference signal. Digitally compress and encode a video signal captured at a fixed period using a reference signal,
In a video transmission method for transmitting the digital compression-encoded video signal to a video reception device connected to a network via the network,
In accordance with a request from the video receiver, a video transmission method is characterized in that a processing time from digital compression encoding of the video signal to transmission to the network is changed. Compression means for digitally compressing and encoding the captured video signal;
Network processing means for transmitting the digital compression-encoded video signal to the video receiver connected to the network via the network;
Control means for controlling the compression means and the network processing means;
With
The video transmission apparatus characterized in that the control means notifies the video reception apparatus of a processing time until the video signal is digitally compressed and encoded and transmitted to the network. The video transmission device according to claim 5,
The video transmission apparatus according to claim 1, wherein the control means notifies a processing time until the video signal is digitally compressed and encoded and transmitted to the network in response to a request from the video reception apparatus. Digitally compresses and encodes video signals captured at regular intervals,
In a video transmission method for transmitting the digital compression-encoded video signal to a video reception device connected to a network via the network,
A video transmission method comprising: notifying the video reception device of a processing time until the video signal is digitally compressed and encoded and transmitted to the network. Network processing means for receiving a plurality of digital compression encoded video signal data streams transmitted from a plurality of video transmission devices connected to the network;
Decoding means for decoding the plurality of video data received by the network processing means;
Video display means for displaying video based on the plurality of video signals decoded by the decoding means;
Control means for controlling the video transmission device;
With
The video receiving apparatus characterized in that the control means acquires processing delay time information required for the video transmitting apparatus to perform digital compression encoding and transmit to the network from the plurality of video transmitting apparatuses. Network processing means for receiving a plurality of digital compression encoded video signal data streams transmitted from a plurality of video transmission devices connected to the network;
Decoding means for decoding the plurality of video data received by the network processing means;
Video display means for displaying video based on the plurality of video signals decoded by the decoding means;
Control means for controlling the video transmission device;
With
The video receiving apparatus characterized in that the control means requests the plurality of video transmitting apparatuses to notify processing delay time information required for the video transmitting apparatus to perform digital compression encoding and transmit to the network. In the video receiver according to claim 8 or 9,
The video receiving apparatus characterized in that the control means requests the video transmission apparatus to set a processing delay time so as to have a desired processing delay time. The video receiving device according to claim 10,
The control means requests the video transmission apparatus to set a processing delay time so that the processing delay time having the longest processing delay time among the processing delay time information acquired from the video transmission apparatus is obtained. A featured video receiver. Receiving a plurality of digital compression encoded video signal data streams transmitted from a plurality of video transmission devices connected to a network;
Decoding a plurality of the received video data;
In a video receiving method for displaying video based on a plurality of decoded video signals,
A video receiving method characterized in that processing delay time information required for the video transmission apparatus to perform digital compression encoding and transmission to the network is acquired from the video transmission apparatus. Receiving a plurality of digital compression encoded video signal data streams transmitted from a plurality of video transmission devices connected to a network;
Decoding a plurality of the received video data;
In a video receiving method for displaying video based on a plurality of decoded video signals,
A video receiving method comprising: requesting the plurality of video transmitting apparatuses to notify processing delay time information required for the video transmitting apparatus to perform digital compression encoding and transmit to the network.

Images