WO2015118664A1

WO2015118664A1 - Image transmission device, image reception device, and surveillance camera system, teleconference system, and vehicle-mounted camera system using same

Info

Publication number: WO2015118664A1
Application number: PCT/JP2014/052925
Authority: WO
Inventors: 稲田　圭介; 甲　展明
Original assignee: 日立マクセル株式会社
Priority date: 2014-02-07
Filing date: 2014-02-07
Publication date: 2015-08-13

Abstract

Provided are an image transmission device and an image reception device that can be used in a system that compresses and transmits image data to be transmitted and that achieve high image quality and high recognition performance. The image transmission device comprises: an image input unit into which image data is input; a compression processing unit that generates compressed image data from the image that is input into the image input unit; and a data transfer unit that outputs the compressed image data that is generated by the compression processing unit to a transmission path. The compression processing unit generates a plurality of pieces of compressed image data that differ with respect to compression delay time. The data transfer unit multiplexes and outputs the plurality of pieces of compressed image data that are generated by the compression processing unit and that differ with respect to compression delay time. The image reception device comprises: a data reception unit that receives the multiplexed plurality of pieces of compressed image data that differ with respect to compression delay time and extracts each of the pieces of compressed image data; an expansion processing unit that expands each of the plurality of pieces of compressed image data that differ with respect to compression delay time and that are received by the data reception unit and generates a plurality of pieces of image data; an additional image generation unit that generates additional image data; and an image layering unit that layers a plurality of pieces of image data. The image layering unit layers additional image data from the expansion processing unit or other image data that is generated by the expansion processing unit on one part of the image data that is generated by the expansion processing unit and outputs the result.

Description

Image transmission device and image reception device, and surveillance camera system, video conference system, and in-vehicle camera system using them

The technical field of the present invention relates to video information transmission / reception technology and a system using the technology.

In recent years, in various fields in digital image transmission / reception, demands for higher recognition, lower delay, and higher image quality are increasing year by year.

Conventionally, with regard to the method of transmitting image data between devices, DisplayPort (VESA registered trademark) established by HDMI (High-Definition Multimedia Interface (HDMI Licensing, LLC registered trademark)) standard and VESA (Video Electronics Standards Association) Or a trademark) standard. As an in-vehicle network, Ethernet (registered trademark) AVB (Audio Video Bridge) is standardized.

In the digital image transmission / reception system that realizes the low delay and the high image quality, the following Patent Document 1 describes "a video transmission apparatus 2 that compresses and encodes a video signal from the camera 1 and sends it out." A video transmission system and a video transmission control method including an operation terminal 3 for decoding a compression-encoded video signal received from a video transmission device 2 and displaying it on the video display unit 11 and controlling the camera 1, The apparatus 2 receives a camera control signal from the low delay mode encoding unit 7, the high quality mode encoding unit 8, the mode selection unit 6, and the operation terminal 3 that compress and encode the video signal from the camera 1. A control signal processing unit 9 that controls the mode selection unit 6 to select the low-delay mode encoding unit 7 and controls the imaging direction or imaging range of the camera 1 as a low-delay mode, 1 of display by reducing the delay time on the video display unit 11 'has been described that (see FIG. 1).

Further, in Patent Document 2 below, “encoding means for setting a delay time of decoding processing based on information on decoding processing capability and encoding the coefficient data rearranged by the rearranging means; and The encoded data generated by encoding the coefficient data by the encoding means is transmitted to the information processing apparatus on the receiving side via the transmission path ”(see [0016]).

JP 2003-284051 A JP 2009-213116 A

By the way, conventionally, especially in the transmission system of digital image data, there has been a problem that a low delay transmission of a digital image is required in a communication system using an image, a monitoring system, an in-vehicle surroundings monitoring system, and the like.

To solve this problem, the amount of delay can be set according to the so-called two types of compression, compression that achieves high image quality and compression that achieves low delay, and the required amount of delay. Have been proposed. However, both methods have a problem in that when the compression is performed with a low delay, the image quality is deteriorated as compared with the case where the delay amount is large.

For example, when a system that performs recognition processing on the receiving side using transmitted image data and is viewed by the user is considered, in such a recognition processing system, from the occurrence of an event to feedback to the user However, in this case, there is a problem that accurate recognition processing cannot be performed due to degradation of image quality. In addition, when realizing high-quality viewing by the user, there is a problem that low-delay transmission is not possible.

That is, with the above-described prior art, it has not always been sufficient to transmit image data with high image quality and low delay.

Therefore, the present invention has been achieved in view of the above-described prior art, and an object thereof is an image transmission apparatus, an image transmission method, and an image transmission apparatus that can transmit image data with high image quality and low delay. Is to provide a surveillance camera system, a video conference system, and an in-vehicle camera system using these.

According to the present invention, in order to achieve the above-described object, first, an image transmission device and an image reception device described in the following claims are provided. More specifically, an image input unit to which image data is input, a compression processing unit that generates compressed image data from an image input to the image input unit, and compressed image data generated by the compression processing unit A data transfer unit for outputting to a transmission line, the compression processing unit generates a plurality of compressed image data having different compression delay times, and the data transfer unit is configured to store the compression delay time generated by the compression processing unit. An image transmission apparatus that multiplexes and outputs a plurality of different compressed image data, receives a plurality of compressed image data having different compression delay times, and extracts each compressed image data; and the data A plurality of compressed image data having different compression delay times received by the receiving unit are decompressed to generate a plurality of image data, and additional image data is generated. An image superimposing unit that superimposes a plurality of image data, and the image superimposing unit adds the additional image data from the expansion processing unit to a part of the image data generated by the expansion processing unit. Alternatively, an image receiving apparatus that superimposes and outputs other image data generated by the decompression processing unit is provided.

In addition, according to the present invention, in order to achieve the above-described object, a surveillance camera system, a video conference system, and an in-vehicle camera system are provided as systems using the image transmission device and the image reception device.

According to the above-described present invention, an image transmission apparatus and an image receiving apparatus capable of transmitting image data with high image quality and low delay, and further, using these, a surveillance camera system that is practically excellent. A video conference system and an in-vehicle camera system are provided.

It is a block diagram which shows an example of the image transmitter in the 1st Example of this invention, and an image receiver. It is a figure which shows an example of the effective / blanking period of the image data in said Example. It is a block diagram which shows an example of the detailed structure of the image process part in said Example. It is a block diagram which shows an example of a detailed structure of the compression process part in said Example. It is a figure which shows an example of the output image of the input image in said Example, an image process part, an additional image generation part, and an image superimposition part. It is a block diagram which shows an example of a detailed structure of the expansion | extension process part in said Example. It is a timing diagram which shows an example of the image input and output in said Example. It is a timing diagram which shows the other example of the image input and output in said Example. It is a timing diagram which shows the further another example of the image input and output in said Example. It is a timing diagram which shows the further another example of the image input and output in said Example. It is a timing diagram which shows an example of the timing for every flame | frame of the display image finally displayed in said Example. It is a whole block diagram which shows an example of a structure of the surveillance camera system which employ | adopted the image transmitter of the said this invention, and an image receiver. It is a whole block diagram which shows an example of a structure of the video conference system which employ | adopted the image transmitter of the said invention, and an image receiver. It is a whole block diagram which shows an example of a structure of the vehicle-mounted camera system which employ | adopted the image transmitter of the said this invention, and an image receiver.

Hereinafter, details of some embodiments according to the present invention will be described with reference to the accompanying drawings.

First, an image transmission apparatus and an image reception apparatus according to a first embodiment (embodiment 1) of the present invention will be described.

FIG. 1 is a block diagram showing an image transmission system according to the present embodiment, and the image transmission system has a configuration in which an image transmission apparatus 1 and an image reception apparatus 2 are connected by a cable 3 as is apparent from the figure. .

The image transmission device 1 is an image transmission device that compresses and transmits image data. More specifically, the image transmission device 1 receives image data that has been decoded so that the digital broadcast can be received and viewed, or image data that has been captured by a camera, This is a so-called image recording / playback device that outputs image data to another device via an HDMI cable or a LAN. Examples of the image transmission apparatus 1 include a recorder, a digital TV with a built-in recorder function, a personal computer with a built-in recorder function, a mobile phone with a camera function and a recorder function, a camcorder, an in-vehicle camera, and the like.

The image receiving device 2 is a display device that inputs image data and outputs an image to a monitor using an HDMI cable, a LAN, or the like. Examples of the image receiving device 2 include a digital TV, a display, a projector, a mobile phone, a signage device, an in-vehicle peripheral monitoring device, and the like.

The cable 1000 is a data transmission path for performing data communication such as image data between the devices of the image transmission device 1 and the image reception device 2. As an example of the cable 3, there is a wired cable compatible with the HDMI standard, the DisplayPort standard, and the Ethernet, or a data transmission path for performing wireless data communication.

Next, details of the configuration of the image transmission apparatus 1 will be described.

The image input unit 10 is an input unit for inputting image data to the image transmission apparatus 1. As an example of the image data input to the input unit 10, there is digital image data input from a monitoring camera or an in-vehicle camera.

The image processing unit 11 performs digital image processing on the image data from the image input unit 10 and outputs the image data after the digital image processing to the compression processing unit 12. Examples of digital image processing include rotation, enlargement or reduction processing, frame rate conversion processing, edge extraction, motion vector extraction, high frequency component removal, noise removal, and the like.

The compression processing unit 12 performs compression processing on the image data from the image input unit 10 and the image data from the image processing unit 11 and outputs the compressed data to the data transfer unit 13.

The data transfer unit 13 converts the two types of compressed image data (compressed image data A and compressed image data B) compressed by the compression processing unit 12 into signals in a format suitable for cable transmission and outputs the signals to the cable 3. An example of a signal in a format suitable for cable transmission is described in the HDMI standard. In HDMI, image data adopts a TMDS data transmission format. Another example of a signal in a format suitable for cable transmission is described in the IEEE P1722 standard used in in-vehicle Ethernet. In the P1722 standard, the compressed image data is transmitted according to AVTP (Audio Video Transport Protocol) Video Protocol.

The user IF unit 14 is an input unit for inputting a signal for controlling the operation of the image transmission apparatus 1. An example of the user IF unit 14 is a remote control receiving unit. The control signal from the user IF unit 14 is output to the control unit 15. The control unit 15 controls the entire image transmission apparatus 1 in accordance with the signal from the user IF unit 14. An example of the control unit 15 is a microprocessor. Image data from the image transmission apparatus 1 is supplied to the image reception apparatus 2 via the cable 3.

Next, details of the configuration of the image receiving apparatus 2 will be described.

Data receiving unit 20 receives a signal in a format suitable for cable transmission. The signal input to the data receiving unit 20 is converted into a predetermined digital data from a signal in a format suitable for cable transmission, and two types of compressed image data (compressed image data A and compressed image data B) are processed. ) Are extracted and output to the decompression processing unit 21.

The decompression processing unit 21 decompresses the compression processing performed by the compression processing unit 12 in the image transmission apparatus 1 to generate two types of image data (decoded image data A and decoded image data B). Also, the decoded image data A is output to the image superimposing unit 23, and the decoded image data B is output to the additional image generating unit 22.

The additional image generation unit 22 extracts image features from the input image data B, generates additional image data based on the result, and outputs the additional image data to the image superimposing unit 23. . Examples of image feature extraction include object detection, moving object detection, state detection, edge extraction, extraction of pixels having luminance within a predetermined threshold range, and the like. Examples of object detection include vehicle detection, dive object detection, lane detection, road surface detection, and object detection such as traffic lights in an in-vehicle periphery monitoring system. One example of moving object detection is suspicious person detection in a monitoring system. Examples of state detection include red signal detection, road surface state detection, weather detection, and retrograde detection. Further, the additional image generation unit 22 may output the input image data B as it is or may output image feature extraction information.

The image superimposing unit 23 generates image data in which two types of image data input from the decompressing unit 21 and the additional image generating unit 22 are superimposed, and outputs the generated image data to the display unit 24.

The image superimposing unit 23 includes a memory unit and a memory control unit for temporarily storing images. During the period until the decoded image data A is input to the image superimposing unit 23, the additional image data is temporarily stored in the memory unit, and the decoded image data A and the additional image data are superimposed from the memory unit at a timing of superimposing processing. The additional image data is read and a superimposition process is performed. Further, the image superimposing unit 23 may perform the image quality improvement processing of the decoded image data A using the decoded image data B or the additional image data. As an example of image quality enhancement processing, there is a method of performing edge enhancement processing of decoded image data A using decoded image data B or additional image data including edge detection information.

The display unit 24 converts the input image data into a signal suitable for the display method and displays it on the screen. Examples of the display unit 24 include a display unit such as a liquid crystal display, a plasma display, an organic EL (Electro-Luminescence) display, and a projector projection display.

The user IF unit 25 is an input unit for inputting a signal for controlling the operation of the image receiving device 2. An example of the user IF unit 25 is a remote control receiving unit. A control signal from the user IF unit 25 is supplied to the control unit 26. The control unit 26 is a control unit that controls the entire image receiving apparatus 2 in accordance with a signal from the user IF unit 25.

FIG. 2 is a diagram showing an effective area in which image data in one frame period is transmitted and a blanking period in which image data is not transmitted.

A region indicated by reference numeral 400 is a vertical period, and the vertical period 400 includes a vertical blanking period 401 and a vertical effective period 402. The VSYNC signal is a 1-bit signal in which 1 is set between the number of lines defined from the top of the vertical blanking period 401 and 0 is set between the other vertical blanking periods and the vertical effective period 402. An example of the prescribed number of lines is 4 lines.

An area indicated by a reference numeral 403 is a horizontal period, and the horizontal period 403 includes a horizontal blanking period 404 and a horizontal effective period 405. The HSYNC signal is a 1-bit signal in which 1 is set between the number of pixels defined from the head of the horizontal blanking period 404 and 0 is set between the other horizontal blanking periods and the horizontal effective period 405. An example of the prescribed number of pixels is 40 pixels.

The effective period 406 is an area surrounded by a vertical effective period 402 and a horizontal effective period 405, and image data is allocated to this period. The blanking period 407 is an area surrounded by a vertical blanking period 401 and a horizontal blanking period 404.

In this embodiment, in the above configuration, the compressed image data and the sub-compression code information are transmitted during the effective period 406, and the main compression code information is transmitted during the blanking period 407. In the blanking period 407, data obtained by packetizing audio data and other attached data is transmitted.

A method of sending a packet of voice data or the like as a reliable packet in the blanking period 407 is disclosed in, for example, Japanese translations of PCT publication No. 2005-514873.

According to this configuration, since the error correction code is included in the packet data in the blanking period, it is possible to correct an error occurring in the transmission path, and the error tolerance is increased. Further, the data for data transmission of the packet in the blanking period is transmitted to two physically different channels, and the channel to be transmitted is switched every certain time. For this reason, an error occurring in a burst manner in one channel is not affected by the other channel, so that a data error can be corrected. The error correction rate has an improvement effect of 10 ^{−14 in} the horizontal blanking period versus 10 ^{−9 in the} horizontal effective period.

FIG. 3 is a diagram illustrating a detailed example of the image processing unit 11 described above. The frame thinning unit 110 is a block for thinning out frames of input image data supplied from the image input unit 10. As an example, image data of 15 fps is output by thinning out every other image for input image data input at 30 fps (frame / second). Note that the input image data may be output at the frame rate without performing the thinning. The reduction unit 111 performs n-tap filtering processing (n = positive number) or pixel thinning processing using pixel values of n lines before and after the input image data, and generates and outputs a reduced image. As an example of a method of generating a reduced image, there is a method of generating only a vertical ½ reduced image by outputting only even-line image data of an input image. The image feature extraction image generation unit 112 extracts features of the input image from the input image data, and generates image data obtained by simplifying or omitting image data other than the features. As an example of the feature extraction processing for image data, there are a first-order differential filter (edge detection, line detection) and a second-order differential detection (Laplacian filter). An example of edge detection is a Sobel filter. Although the frame thinning unit 110, the reduction unit 111, and the image feature extraction image generation unit 112 have been described as the components of the image processing unit 11, all the components may not be included. For example, the image processing unit 11 may be configured by only the reduction unit 111 or may be configured by only the image feature extraction image generation unit 112.

FIG. 4 is a diagram illustrating a detailed example of the compression processing unit 12. In the present embodiment, a compression unit A that generates compressed image data A used for viewing and a compression unit B that generates compressed image data B used for recognition are provided. Compared with the compression processing delay in the compression unit A, the compression processing delay in the compression unit B is short.

The compression unit A performs high-quality compression processing on the image data supplied from the image input unit 10. As an example of the compression process, H.264 There is a compression process using H.264 B pictures. B picture is a method of performing compression using past frames and future frames. This method is not suitable for low-delay transmission because a delay of one frame time or more occurs when performing compression processing because a future frame is used, but a high-quality decoded image can be obtained. This is a method suitable for viewing purposes.

The compression unit B performs low-delay compression processing on the image data supplied from the image processing unit 11. As an example of the compression process, H.264 There is a compression process using H.264 P pictures. P picture is a method of performing compression using past frames. This method is suitable for recognition applications such as object detection because a future frame is not used and a compression delay associated with waiting for a future frame generated in the compression unit B does not occur. The image data supplied from the image processing unit 11 is image data that has been subjected to digital image processing for reducing the amount of image information in the image processing unit 11. It is possible to greatly reduce the amount of generated codes after data compression. In other words, by assigning a larger amount of code to the feature-extracted image data, the compression unit A can be compressed with higher image quality.

The memory control unit 122 performs control for temporarily storing the compressed image data B supplied from the compression unit B in the memory unit 123. As an example of this control, there is a method of temporarily storing the compressed image data B processed in the valid period 406 in the memory unit 123 and outputting the compressed image data B in the subsequent blanking period 407.

Subsequently, FIGS. 5A to 5D show examples of input image data and output image data of the additional image generating unit 22 and the image superimposing unit 23. FIG.

An image 50 shown in FIG. 5A shows an example in which image data (decoded image of the compressed image data A) supplied from the decompression processing unit 21 to the image superimposing unit 23 is displayed as an image. That is, in the image 50, two vehicles, a traveling vehicle 501 and a traveling vehicle 502, are drawn on the traveling road surface 500. The traveling vehicle 501 is a vehicle that has been interrupted on the own vehicle traveling lane from the side, and the traveling vehicle 502 is a vehicle far ahead of the own vehicle. A background 503 is also drawn.

An image 51 shown in FIG. 5B shows an example in which image data (decoded image of the compressed image data B) supplied from the decompression processing unit 21 to the additional image generation unit 22 is displayed as an image. That is, the compressed image data B is image data after edge detection by the image processing unit 11, and image information other than the edge information is simplified or removed. A drawing 511 is an image after the edge of the traveling vehicle 501 is extracted. A drawing 512 is an image after edge extraction of the traveling vehicle 502.

An image 52 shown in FIG. 5C is an example in which the image data supplied from the additional image generating unit 22 to the image superimposing unit 23 is displayed as an image. The additional image generation unit 22 generates, based on the input image 51, a rectangular figure surrounding the detected vehicle that does not detect the vehicle. The color of the rectangular figure may be changed according to the distance and direction from the vehicle. Moreover, it is good also as a photographer which changes in time, such as blinking. A rectangular figure 521 shows a traveling vehicle 501. Rectangular view 522 shows traveling vehicle 502.

An image 53 shown in FIG. 5D is an example in which image data supplied from the image superimposing unit 23 to the display unit 24 is displayed as an image.

FIG. 6 is a diagram illustrating a detailed example of the decompression processing unit 21.

In the figure, the decompression unit A210 performs decompression processing on the compressed image data A to generate decoded image data A. The decompression unit B211 performs decompression processing on the compressed image data B and generates decoded image data B.

FIG. 7 is a diagram showing an example of an image input and output timing chart in the present embodiment.

In the figure, a signal 600 indicates a vertical blanking signal for the

input images

604 and 605 of the image transmission apparatus 1. A signal 610 indicates a vertical blanking signal for the

output images

614, 615, and 617 of the image transmission apparatus 1. A signal 620 indicates a vertical blanking signal for the

input images

624, 625, and 627 of the image superimposing unit 23 of the image receiving device 2. A signal 630 indicates a vertical blanking signal for the output image 631 of the image superimposing unit 23 of the image receiving device 2. The vertical blanking signals 600, 610, 620, and 630 are composed of an effective period 601 and a blanking period 602.

The output image 614 is compressed image data B obtained by compressing the input image 604 by the compression unit B. The output image 617 is compressed image data A obtained by compressing the input image 604 by the compression unit A. The compression delay time 618 for the compressed image data B is shorter than the compression delay time 619 for the compressed image data A. In this example, the compression unit A shows an example in which compression processing is performed using a future frame for one frame.

The input image 624 is decoded image data B obtained by decompressing the compressed image data 614 by the decompression unit B.

The image data 626 is an additional image generated after the recognition processing time 628 for the compressed image data 624 in the additional image generation unit 22. The input image 627 is decoded image data A obtained by decompressing the input image 614 by the decompression unit A.

The output image 631 is image data after the image data 626 and the input image 627 are superimposed.

FIG. 8 is a diagram showing another example of an image input and output timing diagram in the present embodiment. In this figure as well, signals having the same numbers as those in FIG. 7 are the same as those in FIG.

That is, the present embodiment is characterized in that the compressed image data B614 compressed with a low delay is transmitted in both the effective period and the blanking period, and by this method, the compressed image data B614 is made redundant. Thus, it is possible to improve error tolerance when a transmission error occurs. In the additional image generation unit 22 in the image receiving device 2, when there is an abnormality in the decoded image data B624, the transmission error resistance is improved by using the decoded image data B726.

FIG. 8 is a diagram showing another example of an image input and output timing diagram in the present embodiment.

As is apparent from the figure, this example is characterized in that the transmission of the compressed image data B (see reference numeral 614 in FIG. 7) in the effective period is eliminated in the image transmission apparatus 1 shown in FIG. is there. According to this method, only the compressed image data A is transmitted as the compressed image data during the effective period, so that the image quality of the decoded image data A can be further improved.

FIG. 10 is a diagram showing still another example of the image input and output timing chart in the present embodiment.

As is apparent from the figure, in this example, in the image transmission apparatus 1 shown in FIG. 1, the compression unit A (see reference numeral 120 in FIG. 4) of the image transmission apparatus 1 displays frames up to two frames ahead. It is an example which enabled compression processing by using. With this method, it is possible to further improve the image quality of the decoded image data A. The compression unit A may be configured to perform compression processing using frames up to N frames ahead. Also, in FIGS. 7, 8, and 9, the compression processing may be performed using frames up to N frames ahead.

Further, FIG. 11 shows an example of a timing chart for each frame of the input image data, the compressed image data A, the compressed image data B, the additional image, and the finally displayed display image in the present embodiment.

In the lower part of the figure, the APVF Header of the APVF Video PDU format defined by the AVTP Video Protocol in the P1722 standard is shown.

Hereinafter, an example in which the compressed image data A and the compressed image data B are transmitted by AVTP Video Protocol will be described.

Frames

1402 and 1404 are compressed image data for the input image data 1401. In this example, a person 1409 exists in the frame 1401. The compressed image data A (frame 1402) is compressed with a small delay. On the other hand, the compressed image data B (frame 1404) is compressed with less compression deterioration with respect to the compressed image data A by using more time and wider range of frame information. As a result, the compressed image data B achieves higher image quality than the compressed image data A.

The additional image 1406 is a frame image that is drawn around the person in the frame by performing an expansion process and a person recognition process on the frame 1402.

The frame 1405 is an image displayed on the monitor. Reference numeral 1407 denotes an image obtained by superimposing the decoded image (person) obtained by decompressing the compressed image data B (frame 1404) and the additional image 1406 (frame image).

As shown in this example, two types of compressed image data (compressed image data A and compressed image data B) are generated from the input image and output from the image transmission apparatus. Is provided. In the image receiving apparatus, the compressed image data A and the compressed image data B are decompressed with a time difference of one frame or more. Further, the image transmission apparatus generates an additional image 1406 based on the compressed image data B, and displays the additional image 1406 by superimposing it on the decoded image obtained by the decompression process on the compressed image data B.

That is, in this example, the frcount values of the compressed image data A (frame 1401) and the compressed image data B (frame 1402) generated based on the same frame (frame 1401) of the input image are the same. As a result, when there is a difference in decompression processing timing of one frame or more on the image receiving device side, or there is a difference of one frame or more in post-processing (recognition processing in this example) for a plurality of decoded images obtained by the decompression processing. Even in some cases, the image receiving apparatus can determine the compressed image data A (frame 1401) and the compressed image data B (frame 1402) for the same frame (frame 1401) of the input image.

Here, the AVTP Video Protocol has been described as an example. However, the present invention is not limited to this example. For example, the frame count information may be MPEG2-TS or H.264. It may be stored in the user data area of the H.264 / AVC NAL.

That is, according to the image transmission apparatus and the image reception apparatus according to the present embodiment described above, the image data transmitted by the image transmission apparatus is compressed and transmitted by two types of compression methods having different delay times. It is possible to achieve both high image quality and high recognition performance. Furthermore, by making the compressed image data compressed with a low delay redundant, it is possible to improve transmission error tolerance.

Subsequently, an image transmission system to which the above-described image transmission device and the image reception device according to the present invention are applied will be described with reference to FIGS. As an example of such an image transmission system, FIG. 12 shows a surveillance camera system, FIG. 13 shows a video conference system, and FIG. 14 shows an example of an in-vehicle camera system.

In FIG. 12,

reference numerals

1101, 1102, and 1103 are monitoring cameras installed at points A, B, and C, respectively, 1104 is a monitoring center that receives images captured by the

monitoring cameras

1101, 1102, and 1103, and Reference numeral 1105 denotes a wide area network (WAN) such as an Internet line. Images captured by the monitoring cameras 1101 to 1103 can be displayed on a monitor or the like in the monitoring center 1104 via the WAN 1105. Although FIG. 11 shows an example in which there are three surveillance cameras, the number of in-vehicle cameras may be two or less, or four or more.

In the system described above, the image transmission apparatus according to the present embodiment is mounted on, for example, the monitoring cameras 1101 to 1103. That is, the image transmission apparatus performs a compression process, which will be described later, on the input image input via the lenses of the monitoring cameras 1101 to 1103, and the compressed input image is output to the WAN 1105. On the other hand, the image receiving apparatus of the present embodiment is mounted on the monitoring center 1104, for example. The monitoring center 1104 includes a display monitor. That is, the image receiving apparatus performs a decompression process, a recognition process, an additional image generation process, a superimposition process of the decompressed input image and the additional image, etc., which will be described later, on the compressed input image, and performs a superimposition process on the display monitor. Display the later image. The image displayed on the monitor may be an image after decompression processing or an additional image.

Subsequently, in FIG. 13,

reference numerals

1201, 1202, and 1203 are video conference apparatuses installed at points A, B, and C, respectively, and 1204 is a WAN such as an Internet line. In such a system, images captured by the cameras of the video conference apparatuses 1201 to 1203 can be displayed on the monitors of the video conference apparatuses 1201 to 1203 via the WAN 1204. Note that FIG. 12 shows an example in which there are three video conference systems. However, the number of video conference systems may be two, or four or more.

In such a system, the image transmission apparatus according to the present embodiment is mounted on each camera of the video conference apparatuses 1201 to 1203, for example. That is, the image transmission apparatus performs compression processing, which will be described later, on the input image input via the camera lens of each of the video conference apparatuses 1201 to 1203, and outputs the compressed input image to the WAN 1205. Is done. On the other hand, the image receiving apparatus according to the present embodiment is also installed in each of the

video conference apparatuses

1201, 1202, and 1203, for example. That is, each of the

video conference apparatuses

1201, 1202, and 1203 may include a display monitor by itself, or may be externally attached. In this case, the image receiving apparatus performs a compression process on an input image. A decompression process, a recognition process, an additional image generation process, a decompressed input image and an additional image are superimposed, and the image after the superimposition process is displayed on the display monitor. Note that the image displayed on the monitor may be an image after expansion processing or an additional image.

Further, in FIG. 14 showing an example of the in-vehicle camera system, reference numeral 1301 denotes an automobile, 1302 and 1303 denote in-vehicle cameras mounted on the

automobile

1301, and 1306 denotes decompression of compressed image data compressed by the in-vehicle camera. ECU (Electronic Control Unit) that performs processing and recognition processing, 1304 displays an image captured by the in-

vehicle cameras

1302 and 1303, a video expanded by the ECU 1406, an OSD (On Screen Display) reflecting the recognition processing result, and the like Reference numeral 1305 denotes a local area network (LAN) in the automobile 1301. Images captured by the in-

vehicle cameras

1302 and 1303 can be displayed on the monitor 1304 after being subjected to image processing such as expansion processing, recognition processing, and OSD addition by the ECU 1306 via the LAN 1305. Although FIG. 13 shows an example in which two in-vehicle cameras are mounted, however, the number of in-vehicle cameras may be one or three or more.

In such a system, the image transmission apparatus of the present embodiment is mounted on, for example, in-

vehicle cameras

1302 and 1303. That is, the image transmission apparatus performs a compression process, which will be described later, on the input image input via the lenses of the in-

vehicle cameras

1302 and 1303, and the compressed input image is output to the LAN 1305. On the other hand, the image receiving apparatus of the present embodiment is mounted on the ECU 1306, for example. Note that the image receiving apparatus may have both functions of the ECU 1306 and the monitor 1304. Then, the image receiving apparatus performs a decompression process, a recognition process, an additional image generation process, a decompressed input image and an additional image superimposing process, etc., which will be described later, on the compressed input image, and the processed image Is output to the monitor 1304.

In addition, this invention is not limited to the above-mentioned Example, Various modifications are included. For example, the above-described embodiments are described in detail for the entire system in order to explain the present invention in an easy-to-understand manner, and are not necessarily limited to those having all the configurations described. Further, a part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of one embodiment. Further, it is possible to add, delete, and replace other configurations for a part of the configuration of each embodiment.

In addition, each of the above-described configurations, functions, processing units, processing means, and the like may be realized by hardware by designing a part or all of them with, for example, an integrated circuit. Each of the above-described configurations, functions, and the like may be realized by software by interpreting and executing a program that realizes each function by the processor. Information such as programs, tables, and files that realize each function can be stored in a memory, a hard disk, a recording device such as an SSD (Solid State Drive), or a recording medium such as an IC card, an SD card, or a DVD.

Also, the control lines and information lines indicate what is considered necessary for the explanation, and not all the control lines and information lines on the product are necessarily shown. Actually, it may be considered that almost all the components are connected to each other.

DESCRIPTION OF SYMBOLS 1 ... Image transmission apparatus, 2 ... Image receiving apparatus, 3 ... Cable part, 10 ... Image input part, 11 ... Image processing part, 12 ... Input compression process part, 13 ... Data transfer part, 14 ... User IF part, 15 ... Control unit 20 ... Data receiving unit 21 ... Expanding unit 22 ... Additional image generating unit 23 ... Image superimposing unit 24 ... Display unit 25 ... User IF unit 26 ... Control unit

Claims

An image input unit for inputting image data;
A compression processing unit that generates compressed image data from the image input to the image input unit;
A data transfer unit that outputs the compressed image data generated by the compression processing unit to a transmission path;
The compression processing unit generates a plurality of compressed image data having different compression delay times,
The image transmission apparatus, wherein the data transfer unit multiplexes and outputs a plurality of compressed image data having different compression delay times generated by the compression processing unit.
2. The image transmission apparatus according to claim 1, wherein the compression processing unit compresses the compressed image data to be generated based on a request delay time transmitted from another device connected via the transmission path. Determining an image transmission apparatus.
The image transmission apparatus according to claim 2, further comprising an image processing unit, wherein the image processing unit performs image processing before performing compression processing with a short compression delay time in the compression processing unit. An image transmission device.
4. The image transmission device according to claim 3, wherein the image processing includes means for performing at least one of frame thinning processing, reduction processing, and edge filter processing. .
3. The image transmission apparatus according to claim 2, wherein the compressed image data after compression processing with a short compression delay time is transmitted during a blanking period of the image data to be transferred.
A data receiving unit that receives a plurality of compressed image data having different compression delay times and extracts each compressed image data; and
A decompression processing unit that decompresses each of the compressed image data having different compression delay times received by the data receiving unit, and generates a plurality of image data;
An additional image generation unit for generating additional image data;
An image superimposing unit that superimposes a plurality of image data,
The image superimposing unit superimposes additional image data from the decompression processing unit or other image data generated by the decompression processing unit on a part of the image data generated by the decompression processing unit and outputs the superimposed image data. An image receiving apparatus.
7. The image receiving apparatus according to claim 6, wherein the additional image generation unit adds a compression delay time among a plurality of image data generated by the expansion processing unit to a part of the image data generated by the expansion processing unit. An image receiving apparatus that superimposes and outputs short image data.
8. The image receiving device according to claim 7, wherein the additional image generation unit extracts an image feature, generates additional image data based on the result, and performs object detection, moving object as the image feature extraction. An image receiving apparatus comprising detection, state detection, edge extraction, and extraction of a pixel having a luminance within a predetermined threshold range.
A plurality of surveillance cameras installed at different points,
A network to which the plurality of surveillance cameras are connected;
A monitoring camera system comprising a monitoring center that receives images captured by the plurality of monitoring cameras via the network,
The multi-surveillance camera is equipped with the image transmission device according to claim 1,
The monitoring center is equipped with the image receiving device according to claim 6,
The surveillance camera system, wherein the surveillance center includes a display monitor for displaying an image generated by the image receiving device.
Multiple video conferencing devices installed at different points,
A video conferencing system comprising a network connected to the plurality of video conferencing devices,
The video conference device includes the image transmission device according to claim 1 and the image reception device according to claim 6.
The video conference apparatus includes a display monitor for receiving an image transferred from an image transmission apparatus of another video conference apparatus by the image receiving apparatus and displaying the generated image. .
Multiple in-vehicle cameras installed in different parts of the car,
A network arranged in the automobile and connected to the plurality of in-vehicle cameras;
An ECU having a function of receiving images taken by the plurality of monitoring cameras via the network;
An in-vehicle camera system provided with a display monitor arranged in the automobile and displaying an image generated by the ECU,
The in-vehicle camera is equipped with the image transmission device according to claim 1,
An in-vehicle camera system in which the image receiving device according to claim 6 is mounted on the ECU.