WO2017195844A1 - Moving image transmission device and moving image transmission method - Google Patents

Moving image transmission device and moving image transmission method Download PDF

Info

Publication number
WO2017195844A1
WO2017195844A1 PCT/JP2017/017762 JP2017017762W WO2017195844A1 WO 2017195844 A1 WO2017195844 A1 WO 2017195844A1 JP 2017017762 W JP2017017762 W JP 2017017762W WO 2017195844 A1 WO2017195844 A1 WO 2017195844A1
Authority
WO
WIPO (PCT)
Prior art keywords
stream data
moving image
data
processing
color difference
Prior art date
Application number
PCT/JP2017/017762
Other languages
French (fr)
Inventor
Takahiro Narumi
Nobuyasu Tanaka
Original Assignee
Systemk Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Systemk Corporation filed Critical Systemk Corporation
Publication of WO2017195844A1 publication Critical patent/WO2017195844A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234309—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 or from Quicktime to Realvideo
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00—Burglar, theft or intruder alarms
    • G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19665—Details related to the storage of video surveillance data
    • G08B13/19667—Details realated to data compression, encryption or encoding, e.g. resolution modes for reducing data volume to lower transmission bandwidth or memory requirements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/231—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234363—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the spatial resolution, e.g. for clients with a lower screen resolution
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/27—Server based end-user applications
    • H04N21/274—Storing end-user multimedia data in response to end-user request, e.g. network recorder
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/183—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a single remote source

Definitions

  • the present invention relates to a moving image transmission system which transmits a moving image via a network, and more particularly to a moving image transmission system and a moving image transmission method suitable for a case where a moving image is transmitted by communication means which uses a relatively narrow band as in wireless communication.
  • conventional technology can stably transmit high-quality and high-frame-rate moving image data in real time in an environment where the system can be connected with a stable broadband network which uses a broad band as with optical fibers
  • conventional technology has a problem that frame missing or transmission failure sometimes occur in an environment where the system can be connected only with a network which uses a relatively narrow band that fluctuates largely.
  • the present invention provides a moving image transmission device and a moving image transmission method capable of transmitting a fine and high-quality moving image, which is smooth in motion, in a condition where it is difficult to secure a broad band for transmission of a moving image via a network as with an IP camera.
  • the present invention provides a moving image transmission device which is connected with an image pickup device directly or via a network and transmits a moving image to a receiving terminal via a network
  • the image pickup device including image pickup means configured to pick up a moving image, encoding means configured to perform encode processing to a picked-up moving image so as to generate first stream data, and output means configured to output the generated first stream data
  • the moving image transmission device including: receiving means configured to receive the first stream data from the image pickup device; storage means configured to store the received or generated stream data; processing means configured to perform re-encode processing to the first stream data stored in the storage means so as to generate the compressed second stream data; and transmitting means configured to transmit the second stream data, wherein the re-encode processing includes thinning processing of thinning out a color difference signal of YUV image data.
  • the re-encode processing includes thinning processing of converting image data of the first stream data into YUV image data composed of a luminance signal and a color difference signal and thinning out the color difference signal.
  • the first stream data is of YUV 4:2:2 format or YUV 4:2:0 format which is obtained by converting the picked-up image data into YUV image data composed of a luminance signal and a color difference signal and thinning out the color difference signal, and the re-encode processing includes thinning processing of further thinning out the color difference signal from image data of the first stream data.
  • the second stream data is of YUV 4:2:0 format.
  • the re-encode processing includes reduction processing of reducing the size of the image data after the thinning processing.
  • the moving image transmission device of the present invention includes a router function.
  • the moving image transmission device of the present invention is controllably connected with a cloud system via the network, and the transmitting means includes means for uploading the second stream data to the cloud system.
  • the present invention provides a moving image transmission method of transmitting first stream data which is distributed from a moving image pickup device connected with a network, the moving image transmission method including: a reception step of receiving the first stream data from the moving image pickup device; a temporary storage step of temporarily storing the received first stream data; a processing step of performing re-encode processing to stored first stream data so as to generate the compressed second stream data; a storage step of storing the second stream data; and a transmission step of transmitting the second stream data, wherein the re-encode processing includes thinning processing of thinning out a color difference signal of YUV image data.
  • the first stream data is of YUV format constituted of image data which is obtained by converting the picked-up image data into YUV image data composed of a luminance signal and a color difference signal and thinning out the color difference signal
  • the processing step includes a step of further thinning out the color difference signal from image data of the first stream data and reconverting the image data into YUV format.
  • the processing step includes a step of reducing the size of image data of the first stream data after further thinning out the color difference signal from the image data.
  • the transmission step includes: a step of uploading the second stream data to a cloud system which is controllably connected via a network; and a step of transmitting the second stream data in response to a request from a receiving terminal.
  • the moving image transmission device of the present invention is a moving image transmission device which is connected with an image pickup device directly or via a network and transmits a moving image to a receiving terminal via a network
  • the image pickup device including image pickup means configured to pick up a moving image, encoding means configured to perform encode processing to a picked-up moving image so as to generate first stream data, and output means configured to output the generated first stream data
  • the moving image transmission device including: receiving means configured to receive the first stream data from the image pickup device; storage means configured to store the received or generated stream data; processing means configured to perform re-encode processing to the first stream data stored in the storage means so as to generate the compressed second stream data; and transmitting means configured to transmit the second stream data
  • the re-encode processing includes thinning processing of thinning out a color difference signal of YUV image data
  • second stream data is generated by further compressing the first stream data generated by an image pickup device such as an IP camera and is transmitted, and therefore there is an effect that frame missing is prevented and a moving
  • the re-encode processing in the moving image transmission device of the present invention includes thinning processing of converting image data of the first stream data into YUV image data composed of a luminance signal and a color difference signal and thinning out the color difference signal, the data volume is compressed by thinning out a color difference signal, which gives small influence on human visual sense, while maintaining the resolution, size and frame rate of Y image data, which give great influence on human visual sense, and therefore there is an effect that a fine and high-quality moving image which looks smooth in motion can be transmitted.
  • the first stream data in the moving image transmission device of the present invention is of YUV 4:2:2 format or YUV 4:2:0 format which is obtained by converting the picked-up image data into YUV image data composed of a luminance signal and a color difference signal and thinning out the color difference signal
  • the re-encode processing includes thinning processing of further thinning out the color difference signal from image data of the first stream data
  • the data volume is compressed by further applying thinning processing of a color difference signal to first stream data of YUV 4:2:2 format or YUV 4:2:0 format compressed largely, and therefore there is an effect that a fine moving image which is smooth in motion can be transmitted in real time while preventing frame missing even via an unstable network which uses a narrow band as in wireless communication.
  • the second stream data in the moving image transmission device of the present invention is of YUV 4:2:0 format, there is an effect that a high compression ratio can be obtained by utilizing human visual characteristics.
  • the re-encode processing in the moving image transmission device of the present invention includes reduction processing of reducing the size of the image data after the thinning processing, it is possible to obtain a further higher compression ratio while minimizing deterioration of image quality.
  • This moving image transmission device has an effect that a fine moving image which is smooth in motion can be transmitted in real time while surely preventing frame missing even via an unstable network which uses a narrow band as in wireless communication.
  • the moving image transmission device of the present invention includes a router function, it is possible to provide router equipment, which connects the image pickup device with a wide area network such as the Internet, with a function of compressing and converting the first stream data outputted from the image pickup device and transmitting second stream data.
  • This moving image transmission device has an effect that even an existing IP camera can be installed at a place, which can be connected only with an unstable network that uses a relatively narrow band, and a fine moving image which is smooth in motion can be transmitted in real time.
  • the moving image transmission device of the present invention is controllably connected with a cloud system via the network and the transmitting means includes means for uploading the second stream data to the cloud system, there is an effect that stream data is normally distributed from the moving image transmission device by live distribution or recording distribution while stream data is distributed from a cloud system by live distribution or recording distribution when bands at the moving image transmission device side decrease, and therefore a moving image can be always transmitted to a receiving terminal.
  • the moving image transmission method of the present invention is a moving image transmission method of transmitting first stream data which is distributed from a moving image pickup device connected with a network
  • the moving image transmission method including: a reception step of receiving the first stream data from the moving image pickup device; a temporary storage step of temporally storing the received first stream data; a processing step of performing re-encode processing to the stored first stream data so as to generate the compressed second stream data; a storage step of storing the second stream data; and a transmission step of transmitting the second stream data
  • the re-encode processing includes thinning processing of thinning out a color difference signal of YUV image data
  • second stream data can be generated by further compressing the first stream data generated by an image pickup device such as an IP camera and transmitted in real time.
  • this moving image transmission method makes it possible to distribute the stored second stream data by recording distribution when a necessary band width cannot be obtained, and therefore there is an effect that a moving image can be surely transmitted.
  • the first stream data in the moving image transmission method of the present invention is of YUV format constituted of image data which is obtained by converting the picked-up image data into YUV image data composed of a luminance signal and a color difference signal and thinning out the color difference signal
  • the processing step includes a step of further thinning out the color difference signal from image data of the first stream data and reconverting the image data into YUV format
  • the data volume is compressed by thinning out a color difference signal, which gives small influence on human visual sense, while maintaining the resolution, size and frame rate of Y image data, which give great influence on human visual sense, and therefore a fine and high-quality moving image which looks smooth in motion can be transmitted.
  • the data volume is compressed by further applying thinning processing of a color difference signal to first stream data of YUV format, and therefore this moving image transmission method has an effect that a fine moving image which is smooth in motion can be transmitted in real time while preventing frame missing even via an unstable network which uses a narrow band as in wireless communication.
  • the processing step in the moving image transmission method of the present invention includes a step of reducing the size of image data of the first stream data after further thinning out the color difference signal from the image data, it is possible to obtain a further higher compression ratio while minimizing deterioration of image quality.
  • This moving image transmission method has an effect that a fine moving image which is smooth in motion can be transmitted in real time while surely preventing frame missing even via an unstable network which uses a narrow band as in wireless communication.
  • the transmission step in the moving image transmission method of the present invention includes: a step of uploading the second stream data to a cloud system which is controllably connected via a network; and a step of transmitting the second stream data in response to a request from a receiving terminal, there is an effect that stream data is normally distributed directly by live distribution or recording distribution while stream data is distributed from a cloud system by live distribution or recording distribution when bands of a connected network decrease, and therefore a moving image can be always transmitted to a receiving terminal.
  • Fig. 1 is a block diagram illustrating an example of a moving image transmission device according to the present invention was shown.
  • Fig. 2 is a flowchart illustrating an example of a moving image transmission method according to the present invention was shown.
  • FIG. 1 is a block diagram illustrating an example of a moving image transmission device according to the present invention.
  • a moving image transmission device 1 of the present invention is a device which is connected with an image pickup device 3 directly or via a network and transmits a moving image to a receiving terminal 6 via a network 4, the image pickup device 3 including image pickup means (image pickup unit 31) configured to pick up a moving image, encoding means (encoding unit 32) configured to perform encode processing to a picked-up moving image so as to generate first stream data, and output means (output unit 33) configured to output the generated first stream data, the device including: receiving means (receiving unit 11) configured to receive the first stream data from the image pickup device 3; storage means (storage unit 12) configured to store the received or generated stream data; processing means (processing unit 13) configured to perform re-encode processing to the first stream data stored in the storage means (storage unit 12) so as to generate the compressed second stream data; and transmitting means (transmitting unit 14) configured to transmit the second stream data,
  • the present inventors have focused on the fact that a moving image which is picked up when an image pickup device 3 such as an IP camera is installed as a monitoring camera includes a large part, such as the background, which does not move and influence to be given on visual image quality is small even when color difference information is further thinned out from YUV format used by an IP camera or the like.
  • the present inventors have found that various advantages can be obtained when the compression ratio of image data, which has been already compressed at the image pickup device 3 by lossy compression, is further increased by thinning out color difference information and the image data is compressed by lossy compression.
  • the present invention realizes unconventional high compression processing by applying compression using color difference information of YUV image data to general transcode, which lowers the resolution or compresses inter-frame prediction information by lossy compression in few operations, only when moving image compression codes before and after conversion of moving image data are of the same type.
  • a transmission device 1 of the present application can reduce the size of moving image data to approximately 500 kbps while maintaining high-vision image quality and a general frame rate of 30 fps, and realizes live distribution of a moving image, which is visually fine, by retaining color information.
  • the image pickup device 3 has: the image pickup unit 31 which is an IP camera to be installed as a monitoring camera, for example, and picks up a moving image; the encoding unit 32 configured to perform encode processing to a picked-up moving image so as to generate first stream data; and the output unit 33 configured to output the generated first stream data.
  • the image pickup unit 31 which is an IP camera to be installed as a monitoring camera, for example, and picks up a moving image
  • the encoding unit 32 configured to perform encode processing to a picked-up moving image so as to generate first stream data
  • the output unit 33 configured to output the generated first stream data.
  • the image pickup unit 31 is provided with a lens and an image pickup element (such as CCD or CMOS). Light from an object travels through the lens, forms an image on the image pickup element, and is converted into RGB image data through photoelectric conversion by the image pickup element. According to setting, the image pickup unit 31 supplies the encoding unit 32 with RGB image data, or converts RGB image data into YUV image data composed of a luminance signal and a color difference signal and supplies the encoding unit 32 with the YUV image data.
  • the size of image data is 1980 ⁇ 1080 pixels, and the frame rate is 30 fps.
  • the encoding unit 32 performs thinning processing of a color difference signal into YUV 4:2:2 format or YUV 4:2:0 format without changing the size and frame rate of image data, and performs compression processing with H.264 standard so as to generate first stream data.
  • the generated first stream data is temporarily saved in an internal memory and outputted from the output unit 33.
  • the output unit 33 outputs stream data, which has been requested with RTSP or HTTP, to the transmission device 1.
  • the encoding unit 32 is not necessarily configured to perform compression processing with H.264 standard, and may perform compression processing with another standard such as MPEG-1, MPEG-2, MPEG-4 or H.263 standard.
  • the image pickup device 3 may be provided with an unillustrated storage device and have a structure wherein the encoding unit 32 generates moving image data of Motion JPEG standard in addition to first stream data and the storage device stores the moving image data.
  • the image pickup device 3 is only required to have a structure which can output the picked-up moving image data in real time, and various cameras, such as a widespread IP camera, which can photograph a moving image can be used as the image pickup device 3.
  • the transmission device 1 has the receiving unit 11 configured to receive first stream data from the image pickup device 3, the storage unit 12 configured to store the received or generated stream data, the processing unit 13 configured to perform re-encode processing to the first stream data stored in the storage unit 12 so as to generate the compressed second stream data, and the transmitting unit 14 configured to transmit the second stream data.
  • the transmission device 1 and the image pickup device 3 are connected with each other via a communication cable 2 as in PoE connection so that electric power can be supplied from the transmission device 1 to the image pickup device 3.
  • a communication cable 2 as in PoE connection
  • a plurality of image pickup devices 3 may be connected with a transmission device 1.
  • the transmission device 1 and the image pickup device 3 may be connected with each other via a general network such as wired or wireless LAN.
  • the transmission device 1 is provided with a router function and is wirelessly connected with a wide area network 4 such as the Internet.
  • the image pickup device 3 is connected with the network 4 via the transmission device 1 and can distribute a moving image to the receiving terminal 6 which can access the network 4.
  • various settings, such as photographing conditions, of the image pickup device 3 can be made from the receiving terminal 6.
  • the transmission device 1 may be wiredly connected with the network 4, it is preferable that the transmission device 1 can be wirelessly connected with the network 4 so that an area where the image pickup device 3 can be installed widens.
  • the receiving unit 11 receives first stream data from the image pickup device 3 via the communication cable 2 in real time, and the storage unit 12 temporarily saves stream profile data as it is.
  • the storage unit 12 is provided with a memory such as ROM or RAM and a hard disk. Allocated to the storage unit 12 are a program area where a program or the like to be used for executing various processing by the processing unit 13 is stored, a work area where work data or the like to be generated in the process of execution of this program is temporarily stored, and the like. Moreover, the storage unit 12 stores the first stream data received from the image pickup device 3, and the second stream data generated at the processing unit 13.
  • the processing unit 13 is an electronic circuit such as a processor, and performs control of the transmission device 1, re-encode processing of the first stream data stored in the storage unit 12, and the like. In this example, the processing unit 13 performs transcode processing of not decoding but re-encoding first stream data. The processing unit 13 performs thinning processing of further thinning out a color difference signal from YUV image data of first stream data, and next performs reduction processing of reducing the size of image data. The image data size is reduced to 1280 ⁇ 720 pixels, for example. In this example, the frame rate is not changed from 30 fps, in order to achieve smooth playback of a moving image.
  • Thinning processing of a color difference signal is processing of leaving a luminance signal (Y) of YUV image data as it is and thinning out only a color difference signal (U, V).
  • the image data is converted into 4:2:0 format by thinning out half of U and V respectively in the horizontal direction and the vertical direction.
  • the processing unit 13 performs re-encode processing to first stream data with H.264 standard, and performs thinning processing of a color difference signal in addition to general transcode processing so that the bit rate of second stream data to be generated becomes approximately 500 kbps.
  • moving image data is compressed at Level 3 of a baseline profile. It is to be noted that compression processing to moving image data is preferably processing of compressing moving image data with H.264 standard among current standards, though the standard or format is not limited to the example, and another standard or format may be used.
  • the processing unit 13 is provided with a YUV filter configured to change components of YUV image data.
  • a parameter for scale is set for each of YUV, that is, Y parameter is used for adjusting the luminance, U parameter is used for adjusting blue color balance, and V parameter is used for adjusting red color balance.
  • first stream data received at the receiving unit 11 is of RGB format
  • the processing unit 13 converts image data of first stream data into YUV image data composed of a luminance signal (Y) and a color difference signal (U, V), and then performs thinning processing of thinning out a color difference signal.
  • the processing unit 13 converts the image data into 4:2:0 format by thinning out half of U and V respectively in the horizontal direction and the vertical direction.
  • the processing unit 13 performs re-encode processing to first stream data with H.264 standard, and performs thinning processing of a color difference signal in addition to general transcode processing so that the bit rate of second stream data to be generated becomes approximately 500 kbps.
  • the receiving unit 11 preferably generates second stream data in a progressive form, regardless of the form of the first stream data.
  • the second stream data generated at the processing unit 13 is stored in the storage unit 12 and also transmitted from the transmitting unit 14 via the network 4 to a cloud system 5 and the receiving terminal 6.
  • the transmitting unit 14 is provided with a wireless communication function and is wirelessly connected with the wide area network 4 such as the Internet.
  • the transmitting unit 14 transmits the generated second stream data to the receiving terminal 6 in real time and, at the same time, uploads the generated second stream data to the cloud system.
  • the transmitting unit 14 distributes second stream data, which is stored in the storage unit 12, by recording distribution. It is to be noted that the transmitting unit 14 may be wiredly connected with the network 4.
  • the cloud system 5 is connected with the transmission device 1 via the network 4, and associates with the transmission device 1 using API (Application Programing Interface).
  • the cloud system 5 stores second stream data, which is uploaded from the transmission device 1, in the storage device.
  • the cloud system 5 can transmit the second stream data to a receiving terminal in real time, and distributes the second stream data by recording distribution in response to a request from a receiving terminal.
  • the receiving terminal 6 is a personal computer, a tablet terminal, a smartphone or the like which can be connected with the wide area network 4 such as the Internet.
  • the receiving terminal 6 can access the transmission device 1 or the cloud system 5 so as to view a moving image, which is picked up by the image pickup device 3, by live viewing or recording viewing.
  • the receiving terminal 6 can access the cloud system 5 so as to perform equipment management or remote operation of the cloud system 5, and management of setting change or the like of the transmission device 1.
  • Fig. 2 is a flowchart illustrating an example of a moving image transmission method according to the present invention.
  • a moving image transmission method of the present invention is a method of transmitting first stream data which is distributed from the moving image pickup device 3 connected with the network 4, the moving image transmission method including: a reception step of receiving first stream data from the image pickup device 3; a temporary storage step of temporarily storing the received first stream data; a processing step of performing re-encode processing to stored first stream data so as to generate the compressed second stream data; a storage step of storing second stream data; and a transmission step of transmitting second stream data.
  • the image pickup device 3 converts RGB image data, which is composed of the respective images of red (R), green (G) and blue (B), into YUV image data composed of a luminance signal (Y) and a color difference signal (U, V) (step S1). Pickup of a moving image by the image pickup device 3 can be controlled via the network 4.
  • the image pickup device 3 performs thinning processing of the color difference signal (U, V) into YUV 4:2:0 format at the encoding unit 32, and performs compression processing with H.264 standard so as to generate first stream data (step S2).
  • the size of image data is 1980 ⁇ 1080 pixels, and the frame rate is 30 fps, for example.
  • a progressive signal is preferably used with the maximum resolution of the image pickup device 3.
  • the image pickup device 3 temporarily saves the generated first stream data in the internal memory (step S3).
  • the image pickup device 3 waits for a request from the transmission device 1 (step S4). According to an indication from the receiving terminal 6 or the cloud system 5, the transmission device 1 requests the image pickup device 3 to output first stream data. Upon receiving the request from the transmission device 1, the output unit 33 outputs the first stream data in real time (step S5).
  • the transmission device 1 When the receiving unit 11 receives the first stream data, the transmission device 1 temporarily saves stream profile data in the storage unit 12 as it is (step S6).
  • the data is saved in MPEG form.
  • the transmission device 1 leaves the luminance signal (Y) of YUV image data as it is and performs thinning processing only to the color difference signal (U, V) at the processing unit 13 (step S7).
  • the processing unit 13 utilizes human visual characteristics to further perform thinning processing of a color difference signal to the first stream data, which has been subjected to thinning processing of a color difference signal.
  • the processing unit 13 partially modifies the pixel form of YUV using a lookup table YUV filter. Thinning processing of the color difference signal (U, V) can be performed by combining pixel thinning and tone compression.
  • the processing unit 13 performs reduction processing of reducing the size of image data, which has been subjected to thinning processing of a color difference signal (step S8).
  • the size of image data is reduced to 1280 ⁇ 720 pixels, for example. It is to be noted that the frame rate is not changed from 30 fps. Since first stream data to be generated at the image pickup device 3 includes an interlace signal and a progressive signal, the processing unit 13 uses a progressive signal to generate stream data.
  • the processing unit 13 performs re-encode processing to moving image data with H.264 standard so as to generate second stream data (step S9).
  • the processing unit 13 specifies the bit rate of second stream data to be generated so that the bit rate becomes 500 kbps or below, and also performs compression processing such as interframe prediction or entropy coding.
  • the transmission device 1 saves the generated second stream data in the storage unit 12 and, at the same time, transmits the second stream data from the transmitting unit 14 via the network 4 to the receiving terminal 6 or the cloud system 5 (step S10).
  • a moving image transmission method of the present invention realizes unconventional high compression processing while maintaining a moving image having color information which is also visually fine.
  • This moving image transmission method can reduce the size of moving image data to 500 kbps or below while maintaining high-vision image quality and a general frame rate. This makes it possible to stably distribute fine moving image data by live distribution even via a network having an uplink communication speed as low as approximately 1 Mbps.
  • a moving image transmission method of the present invention can reduce a load on the image pickup device 3 or the cloud system 5 by distributing compression processing of moving image data.
  • the transmission device 1 may be constituted of a cloud system.
  • the transmission device 1 and the image pickup device 3 are connected with each other via a network.
  • the other structure is the same as the structure of Example 1.
  • the transmission device 1 is constituted of a cloud system
  • live viewing from a receiving terminal 6, which exists at a place where a narrow band is used, without frame missing also becomes possible.

Abstract

To provide a moving image transmission device and a moving image transmission method capable of transmitting a high-quality moving image via a network even in a state where a broad band cannot be obtained as with an IP camera.　Provided is a transmission device 1 which is connected with an image pickup device 3 directly or via a network and transmits a moving image to a receiving terminal 6 via a network 4, the image pickup device 3 including an image pickup unit 31 configured to pick up a moving image, an encoding unit 32 configured to perform encode processing to a picked-up moving image so as to generate the first stream data, and an output unit 33 configured to output the generated first stream data, the transmission device 1 including: a receiving unit 11 configured to receive the first stream data from the image pickup device 3; a storage unit 12 configured to store the received or generated stream data; a processing unit 13 configured to perform re-encode processing to the first stream data stored in the storage unit 12 so as to generate the compressed second stream data; and a transmitting unit 14 configured to transmit the second stream data.

Description

MOVING IMAGE TRANSMISSION DEVICE AND MOVING IMAGE TRANSMISSION METHOD
　　The present invention relates to a moving image transmission system which transmits a moving image via a network, and more particularly to a moving image transmission system and a moving image transmission method suitable for a case where a moving image is transmitted by communication means which uses a relatively narrow band as in wireless communication.
　　In recent years, with the growth of broadband networks, a monitoring system wherein an IP camera is installed indoors or outdoors and transmits moving image data via a network so that the moving image data can be viewed or saved at a separated place (see Fig. 6 in Patent Literature 1, for example) has been widespread. In a conventional monitoring camera system, picked-up moving image data is distributed via a network in a compressed manner. Known compression methods for moving image data include MPEG2, MPEG4 or H.264, for example.
[PTL 1]　　Patent 2012-90172
　　Although conventional technology can stably transmit high-quality and high-frame-rate moving image data in real time in an environment where the system can be connected with a stable broadband network which uses a broad band as with optical fibers, conventional technology has a problem that frame missing or transmission failure sometimes occur in an environment where the system can be connected only with a network which uses a relatively narrow band that fluctuates largely.
　　As development of stable optical fiber networks which use broad bands among broadband networks concentrates on densely populated areas such as urban areas, there are still many other areas which depend on unstable ADSL or wireless communication which uses a relatively narrow band. Technology for transmitting a fine image at a high frame rate has been desired, since it is necessary to degrade the image quality, reduce the size, or lower the frame rate of image data in a case where moving image data having a large data capacity is transmitted in real time in areas provided with unstable networks which use relatively narrow bands.
　　In view of the situation, the present invention provides a moving image transmission device and a moving image transmission method capable of transmitting a fine and high-quality moving image, which is smooth in motion, in a condition where it is difficult to secure a broad band for transmission of a moving image via a network as with an IP camera.
　　In order to solve the above problem, the present invention provides a moving image transmission device which is connected with an image pickup device directly or via a network and transmits a moving image to a receiving terminal via a network, the image pickup device including image pickup means configured to pick up a moving image, encoding means configured to perform encode processing to a picked-up moving image so as to generate first stream data, and output means configured to output the generated first stream data, the moving image transmission device including: receiving means configured to receive the first stream data from the image pickup device; storage means configured to store the received or generated stream data; processing means configured to perform re-encode processing to the first stream data stored in the storage means so as to generate the compressed second stream data; and transmitting means configured to transmit the second stream data, wherein the re-encode processing includes thinning processing of thinning out a color difference signal of YUV image data.
　　Moreover, in the moving image transmission device of the present invention, the re-encode processing includes thinning processing of converting image data of the first stream data into YUV image data composed of a luminance signal and a color difference signal and thinning out the color difference signal.
　　Moreover, in the moving image transmission device of the present invention, the first stream data is of YUV 4:2:2 format or YUV 4:2:0 format which is obtained by converting the picked-up image data into YUV image data composed of a luminance signal and a color difference signal and thinning out the color difference signal, and the re-encode processing includes thinning processing of further thinning out the color difference signal from image data of the first stream data.
　　Moreover, in the moving image transmission device of the present invention, the second stream data is of YUV 4:2:0 format.
　　Moreover, in the moving image transmission device of the present invention, the re-encode processing includes reduction processing of reducing the size of the image data after the thinning processing.
　　Moreover, the moving image transmission device of the present invention includes a router function.
　　Moreover, the moving image transmission device of the present invention is controllably connected with a cloud system via the network, and the transmitting means includes means for uploading the second stream data to the cloud system.
　　Moreover, the present invention provides a moving image transmission method of transmitting first stream data which is distributed from a moving image pickup device connected with a network, the moving image transmission method including: a reception step of receiving the first stream data from the moving image pickup device; a temporary storage step of temporarily storing the received first stream data; a processing step of performing re-encode processing to stored first stream data so as to generate the compressed second stream data; a storage step of storing the second stream data; and a transmission step of transmitting the second stream data, wherein the re-encode processing includes thinning processing of thinning out a color difference signal of YUV image data.
　　Moreover, in the moving image transmission method of the present invention, the first stream data is of YUV format constituted of image data which is obtained by converting the picked-up image data into YUV image data composed of a luminance signal and a color difference signal and thinning out the color difference signal, and the processing step includes a step of further thinning out the color difference signal from image data of the first stream data and reconverting the image data into YUV format.
　　Moreover, in the moving image transmission method of the present invention, the processing step includes a step of reducing the size of image data of the first stream data after further thinning out the color difference signal from the image data.
　　Moreover, in the moving image transmission method of the present invention, the transmission step includes: a step of uploading the second stream data to a cloud system which is controllably connected via a network; and a step of transmitting the second stream data in response to a request from a receiving terminal.
　　Since the moving image transmission device of the present invention is a moving image transmission device which is connected with an image pickup device directly or via a network and transmits a moving image to a receiving terminal via a network, the image pickup device including image pickup means configured to pick up a moving image, encoding means configured to perform encode processing to a picked-up moving image so as to generate first stream data, and output means configured to output the generated first stream data, the moving image transmission device including: receiving means configured to receive the first stream data from the image pickup device; storage means configured to store the received or generated stream data; processing means configured to perform re-encode processing to the first stream data stored in the storage means so as to generate the compressed second stream data; and transmitting means configured to transmit the second stream data, wherein the re-encode processing includes thinning processing of thinning out a color difference signal of YUV image data, second stream data is generated by further compressing the first stream data generated by an image pickup device such as an IP camera and is transmitted, and therefore there is an effect that frame missing is prevented and a moving image which is smooth in motion can be transmitted in real time even via an unstable network which uses a relatively narrow band.
　　Moreover, since the re-encode processing in the moving image transmission device of the present invention includes thinning processing of converting image data of the first stream data into YUV image data composed of a luminance signal and a color difference signal and thinning out the color difference signal, the data volume is compressed by thinning out a color difference signal, which gives small influence on human visual sense, while maintaining the resolution, size and frame rate of Y image data, which give great influence on human visual sense, and therefore there is an effect that a fine and high-quality moving image which looks smooth in motion can be transmitted.
　　Moreover, since the first stream data in the moving image transmission device of the present invention is of YUV 4:2:2 format or YUV 4:2:0 format which is obtained by converting the picked-up image data into YUV image data composed of a luminance signal and a color difference signal and thinning out the color difference signal, and the re-encode processing includes thinning processing of further thinning out the color difference signal from image data of the first stream data, the data volume is compressed by further applying thinning processing of a color difference signal to first stream data of YUV 4:2:2 format or YUV 4:2:0 format compressed largely, and therefore there is an effect that a fine moving image which is smooth in motion can be transmitted in real time while preventing frame missing even via an unstable network which uses a narrow band as in wireless communication.
　　Moreover, since the second stream data in the moving image transmission device of the present invention is of YUV 4:2:0 format, there is an effect that a high compression ratio can be obtained by utilizing human visual characteristics.
　　Moreover, since the re-encode processing in the moving image transmission device of the present invention includes reduction processing of reducing the size of the image data after the thinning processing, it is possible to obtain a further higher compression ratio while minimizing deterioration of image quality. This moving image transmission device has an effect that a fine moving image which is smooth in motion can be transmitted in real time while surely preventing frame missing even via an unstable network which uses a narrow band as in wireless communication.
　　Moreover, since the moving image transmission device of the present invention includes a router function, it is possible to provide router equipment, which connects the image pickup device with a wide area network such as the Internet, with a function of compressing and converting the first stream data outputted from the image pickup device and transmitting second stream data. This moving image transmission device has an effect that even an existing IP camera can be installed at a place, which can be connected only with an unstable network that uses a relatively narrow band, and a fine moving image which is smooth in motion can be transmitted in real time.
　　Moreover, since the moving image transmission device of the present invention is controllably connected with a cloud system via the network and the transmitting means includes means for uploading the second stream data to the cloud system, there is an effect that stream data is normally distributed from the moving image transmission device by live distribution or recording distribution while stream data is distributed from a cloud system by live distribution or recording distribution when bands at the moving image transmission device side decrease, and therefore a moving image can be always transmitted to a receiving terminal.
　　Moreover, since the moving image transmission method of the present invention is a moving image transmission method of transmitting first stream data which is distributed from a moving image pickup device connected with a network, the moving image transmission method including: a reception step of receiving the first stream data from the moving image pickup device; a temporary storage step of temporally storing the received first stream data; a processing step of performing re-encode processing to the stored first stream data so as to generate the compressed second stream data; a storage step of storing the second stream data; and a transmission step of transmitting the second stream data, wherein the re-encode processing includes thinning processing of thinning out a color difference signal of YUV image data, second stream data can be generated by further compressing the first stream data generated by an image pickup device such as an IP camera and transmitted in real time. Moreover, this moving image transmission method makes it possible to distribute the stored second stream data by recording distribution when a necessary band width cannot be obtained, and therefore there is an effect that a moving image can be surely transmitted.
　　Moreover, since the first stream data in the moving image transmission method of the present invention is of YUV format constituted of image data which is obtained by converting the picked-up image data into YUV image data composed of a luminance signal and a color difference signal and thinning out the color difference signal, and the processing step includes a step of further thinning out the color difference signal from image data of the first stream data and reconverting the image data into YUV format, the data volume is compressed by thinning out a color difference signal, which gives small influence on human visual sense, while maintaining the resolution, size and frame rate of Y image data, which give great influence on human visual sense, and therefore a fine and high-quality moving image which looks smooth in motion can be transmitted. The data volume is compressed by further applying thinning processing of a color difference signal to first stream data of YUV format, and therefore this moving image transmission method has an effect that a fine moving image which is smooth in motion can be transmitted in real time while preventing frame missing even via an unstable network which uses a narrow band as in wireless communication.
　　Moreover, since the processing step in the moving image transmission method of the present invention includes a step of reducing the size of image data of the first stream data after further thinning out the color difference signal from the image data, it is possible to obtain a further higher compression ratio while minimizing deterioration of image quality. This moving image transmission method has an effect that a fine moving image which is smooth in motion can be transmitted in real time while surely preventing frame missing even via an unstable network which uses a narrow band as in wireless communication.
　　Moreover, since the transmission step in the moving image transmission method of the present invention includes: a step of uploading the second stream data to a cloud system which is controllably connected via a network; and a step of transmitting the second stream data in response to a request from a receiving terminal, there is an effect that stream data is normally distributed directly by live distribution or recording distribution while stream data is distributed from a cloud system by live distribution or recording distribution when bands of a connected network decrease, and therefore a moving image can be always transmitted to a receiving terminal.
Fig. 1 is a block diagram illustrating an example of a moving image transmission device according to the present invention was shown. Fig. 2 is a flowchart illustrating an example of a moving image transmission method according to the present invention was shown.
　　An embodiment of the present invention will be described based on an illustrated example. Fig. 1 is a block diagram illustrating an example of a moving image transmission device according to the present invention.
　　A moving image transmission device 1 of the present invention is a device which is connected with an image pickup device 3 directly or via a network and transmits a moving image to a receiving terminal 6 via a network 4, the image pickup device 3 including image pickup means (image pickup unit 31) configured to pick up a moving image, encoding means (encoding unit 32) configured to perform encode processing to a picked-up moving image so as to generate first stream data, and output means (output unit 33) configured to output the generated first stream data, the device including: receiving means (receiving unit 11) configured to receive the first stream data from the image pickup device 3; storage means (storage unit 12) configured to store the received or generated stream data; processing means (processing unit 13) configured to perform re-encode processing to the first stream data stored in the storage means (storage unit 12) so as to generate the compressed second stream data; and transmitting means (transmitting unit 14) configured to transmit the second stream data,
　　The present inventors have focused on the fact that a moving image which is picked up when an image pickup device 3 such as an IP camera is installed as a monitoring camera includes a large part, such as the background, which does not move and influence to be given on visual image quality is small even when color difference information is further thinned out from YUV format used by an IP camera or the like. In addition, the present inventors have found that various advantages can be obtained when the compression ratio of image data, which has been already compressed at the image pickup device 3 by lossy compression, is further increased by thinning out color difference information and the image data is compressed by lossy compression.
　　The present invention realizes unconventional high compression processing by applying compression using color difference information of YUV image data to general transcode, which lowers the resolution or compresses inter-frame prediction information by lossy compression in few operations, only when moving image compression codes before and after conversion of moving image data are of the same type. A transmission device 1 of the present application can reduce the size of moving image data to approximately 500 kbps while maintaining high-vision image quality and a general frame rate of 30 fps, and realizes live distribution of a moving image, which is visually fine, by retaining color information.
Image Pickup Device
　　In this example, the image pickup device 3 has: the image pickup unit 31 which is an IP camera to be installed as a monitoring camera, for example, and picks up a moving image; the encoding unit 32 configured to perform encode processing to a picked-up moving image so as to generate first stream data; and the output unit 33 configured to output the generated first stream data.
　　The image pickup unit 31 is provided with a lens and an image pickup element (such as CCD or CMOS). Light from an object travels through the lens, forms an image on the image pickup element, and is converted into RGB image data through photoelectric conversion by the image pickup element. According to setting, the image pickup unit 31 supplies the encoding unit 32 with RGB image data, or converts RGB image data into YUV image data composed of a luminance signal and a color difference signal and supplies the encoding unit 32 with the YUV image data. In this example, the size of image data is 1980×1080 pixels, and the frame rate is 30 fps.
　　In this example, the encoding unit 32 performs thinning processing of a color difference signal into YUV 4:2:2 format or YUV 4:2:0 format without changing the size and frame rate of image data, and performs compression processing with H.264 standard so as to generate first stream data. The generated first stream data is temporarily saved in an internal memory and outputted from the output unit 33. The output unit 33 outputs stream data, which has been requested with RTSP or HTTP, to the transmission device 1.
　　The encoding unit 32 is not necessarily configured to perform compression processing with H.264 standard, and may perform compression processing with another standard such as MPEG-1, MPEG-2, MPEG-4 or H.263 standard. Moreover, the image pickup device 3 may be provided with an unillustrated storage device and have a structure wherein the encoding unit 32 generates moving image data of Motion JPEG standard in addition to first stream data and the storage device stores the moving image data.
　　The image pickup device 3 is only required to have a structure which can output the picked-up moving image data in real time, and various cameras, such as a widespread IP camera, which can photograph a moving image can be used as the image pickup device 3.
Transmission Device
　　The transmission device 1 has the receiving unit 11 configured to receive first stream data from the image pickup device 3, the storage unit 12 configured to store the received or generated stream data, the processing unit 13 configured to perform re-encode processing to the first stream data stored in the storage unit 12 so as to generate the compressed second stream data, and the transmitting unit 14 configured to transmit the second stream data.
　　The transmission device 1 and the image pickup device 3 are connected with each other via a communication cable 2 as in PoE connection so that electric power can be supplied from the transmission device 1 to the image pickup device 3. Although one image pickup device 3 is connected with one transmission device 1 in the illustrated example, a plurality of image pickup devices 3 may be connected with a transmission device 1. Moreover, when a power source for the image pickup device 3 can be secured, the transmission device 1 and the image pickup device 3 may be connected with each other via a general network such as wired or wireless LAN.
　　In the example, the transmission device 1 is provided with a router function and is wirelessly connected with a wide area network 4 such as the Internet. The image pickup device 3 is connected with the network 4 via the transmission device 1 and can distribute a moving image to the receiving terminal 6 which can access the network 4. Moreover, various settings, such as photographing conditions, of the image pickup device 3 can be made from the receiving terminal 6. Although the transmission device 1 may be wiredly connected with the network 4, it is preferable that the transmission device 1 can be wirelessly connected with the network 4 so that an area where the image pickup device 3 can be installed widens.
　　The receiving unit 11 receives first stream data from the image pickup device 3 via the communication cable 2 in real time, and the storage unit 12 temporarily saves stream profile data as it is.
　　The storage unit 12 is provided with a memory such as ROM or RAM and a hard disk. Allocated to the storage unit 12 are a program area where a program or the like to be used for executing various processing by the processing unit 13 is stored, a work area where work data or the like to be generated in the process of execution of this program is temporarily stored, and the like. Moreover, the storage unit 12 stores the first stream data received from the image pickup device 3, and the second stream data generated at the processing unit 13.
　　The processing unit 13 is an electronic circuit such as a processor, and performs control of the transmission device 1, re-encode processing of the first stream data stored in the storage unit 12, and the like. In this example, the processing unit 13 performs transcode processing of not decoding but re-encoding first stream data. The processing unit 13 performs thinning processing of further thinning out a color difference signal from YUV image data of first stream data, and next performs reduction processing of reducing the size of image data. The image data size is reduced to 1280×720 pixels, for example. In this example, the frame rate is not changed from 30 fps, in order to achieve smooth playback of a moving image.
　　Thinning processing of a color difference signal is processing of leaving a luminance signal (Y) of YUV image data as it is and thinning out only a color difference signal (U, V). In this example, the image data is converted into 4:2:0 format by thinning out half of U and V respectively in the horizontal direction and the vertical direction. The processing unit 13 performs re-encode processing to first stream data with H.264 standard, and performs thinning processing of a color difference signal in addition to general transcode processing so that the bit rate of second stream data to be generated becomes approximately 500 kbps. For example, moving image data is compressed at Level 3 of a baseline profile. It is to be noted that compression processing to moving image data is preferably processing of compressing moving image data with H.264 standard among current standards, though the standard or format is not limited to the example, and another standard or format may be used.
　　The processing unit 13 is provided with a YUV filter configured to change components of YUV image data. Regarding the YUV filter, a parameter for scale is set for each of YUV, that is, Y parameter is used for adjusting the luminance, U parameter is used for adjusting blue color balance, and V parameter is used for adjusting red color balance.
　　When first stream data received at the receiving unit 11 is of RGB format, the processing unit 13 converts image data of first stream data into YUV image data composed of a luminance signal (Y) and a color difference signal (U, V), and then performs thinning processing of thinning out a color difference signal. The processing unit 13 converts the image data into 4:2:0 format by thinning out half of U and V respectively in the horizontal direction and the vertical direction. The processing unit 13 performs re-encode processing to first stream data with H.264 standard, and performs thinning processing of a color difference signal in addition to general transcode processing so that the bit rate of second stream data to be generated becomes approximately 500 kbps. The receiving unit 11 preferably generates second stream data in a progressive form, regardless of the form of the first stream data.
　　The second stream data generated at the processing unit 13 is stored in the storage unit 12 and also transmitted from the transmitting unit 14 via the network 4 to a cloud system 5 and the receiving terminal 6.
　　The transmitting unit 14 is provided with a wireless communication function and is wirelessly connected with the wide area network 4 such as the Internet. The transmitting unit 14 transmits the generated second stream data to the receiving terminal 6 in real time and, at the same time, uploads the generated second stream data to the cloud system. Moreover, in response to a request from a receiving terminal, the transmitting unit 14 distributes second stream data, which is stored in the storage unit 12, by recording distribution. It is to be noted that the transmitting unit 14 may be wiredly connected with the network 4.
Receiving Side Device
　　The cloud system 5 is connected with the transmission device 1 via the network 4, and associates with the transmission device 1 using API (Application Programing Interface). The cloud system 5 stores second stream data, which is uploaded from the transmission device 1, in the storage device. The cloud system 5 can transmit the second stream data to a receiving terminal in real time, and distributes the second stream data by recording distribution in response to a request from a receiving terminal.
　　The receiving terminal 6 is a personal computer, a tablet terminal, a smartphone or the like which can be connected with the wide area network 4 such as the Internet. The receiving terminal 6 can access the transmission device 1 or the cloud system 5 so as to view a moving image, which is picked up by the image pickup device 3, by live viewing or recording viewing. Moreover, the receiving terminal 6 can access the cloud system 5 so as to perform equipment management or remote operation of the cloud system 5, and management of setting change or the like of the transmission device 1.
Operation of Transmission Device
　　The operation of the transmission device 1 according to Example 1 will be described with reference to Fig. 2. Fig. 2 is a flowchart illustrating an example of a moving image transmission method according to the present invention.
　　A moving image transmission method of the present invention is a method of transmitting first stream data which is distributed from the moving image pickup device 3 connected with the network 4, the moving image transmission method including: a reception step of receiving first stream data from the image pickup device 3; a temporary storage step of temporarily storing the received first stream data; a processing step of performing re-encode processing to stored first stream data so as to generate the compressed second stream data; a storage step of storing second stream data; and a transmission step of transmitting second stream data.
　　When a moving image is picked up at the image pickup unit 31, the image pickup device 3 converts RGB image data, which is composed of the respective images of red (R), green (G) and blue (B), into YUV image data composed of a luminance signal (Y) and a color difference signal (U, V) (step S1). Pickup of a moving image by the image pickup device 3 can be controlled via the network 4.
　　The image pickup device 3 performs thinning processing of the color difference signal (U, V) into YUV 4:2:0 format at the encoding unit 32, and performs compression processing with H.264 standard so as to generate first stream data (step S2). Regarding the first stream data, the size of image data is 1980×1080 pixels, and the frame rate is 30 fps, for example. Regarding the first stream data, a progressive signal is preferably used with the maximum resolution of the image pickup device 3.
　　The image pickup device 3 temporarily saves the generated first stream data in the internal memory (step S3).
　　The image pickup device 3 waits for a request from the transmission device 1 (step S4). According to an indication from the receiving terminal 6 or the cloud system 5, the transmission device 1 requests the image pickup device 3 to output first stream data. Upon receiving the request from the transmission device 1, the output unit 33 outputs the first stream data in real time (step S5).
　　When the receiving unit 11 receives the first stream data, the transmission device 1 temporarily saves stream profile data in the storage unit 12 as it is (step S6). Here, the data is saved in MPEG form.
　　The transmission device 1 leaves the luminance signal (Y) of YUV image data as it is and performs thinning processing only to the color difference signal (U, V) at the processing unit 13 (step S7). The processing unit 13 utilizes human visual characteristics to further perform thinning processing of a color difference signal to the first stream data, which has been subjected to thinning processing of a color difference signal.
　　The processing unit 13 partially modifies the pixel form of YUV using a lookup table YUV filter. Thinning processing of the color difference signal (U, V) can be performed by combining pixel thinning and tone compression.
　　The processing unit 13 performs reduction processing of reducing the size of image data, which has been subjected to thinning processing of a color difference signal (step S8). The size of image data is reduced to 1280×720 pixels, for example. It is to be noted that the frame rate is not changed from 30 fps. Since first stream data to be generated at the image pickup device 3 includes an interlace signal and a progressive signal, the processing unit 13 uses a progressive signal to generate stream data.
　　The processing unit 13 performs re-encode processing to moving image data with H.264 standard so as to generate second stream data (step S9). The processing unit 13 specifies the bit rate of second stream data to be generated so that the bit rate becomes 500 kbps or below, and also performs compression processing such as interframe prediction or entropy coding.
　　The transmission device 1 saves the generated second stream data in the storage unit 12 and, at the same time, transmits the second stream data from the transmitting unit 14 via the network 4 to the receiving terminal 6 or the cloud system 5 (step S10).
　　A moving image transmission method of the present invention realizes unconventional high compression processing while maintaining a moving image having color information which is also visually fine. This moving image transmission method can reduce the size of moving image data to 500 kbps or below while maintaining high-vision image quality and a general frame rate. This makes it possible to stably distribute fine moving image data by live distribution even via a network having an uplink communication speed as low as approximately 1 Mbps. Moreover, a moving image transmission method of the present invention can reduce a load on the image pickup device 3 or the cloud system 5 by distributing compression processing of moving image data.
　　As another example of the present invention, the transmission device 1 may be constituted of a cloud system. In such a case, the transmission device 1 and the image pickup device 3 are connected with each other via a network. The other structure is the same as the structure of Example 1.
　　In a case where the transmission device 1 is constituted of a cloud system, it is possible to achieve reduction of storage capacity and enhancement of storage speed in the cloud system, though it is impossible to reduce a transmission load of first stream data from the image pickup device 3 to the transmission device 1. Moreover, live viewing from a receiving terminal 6, which exists at a place where a narrow band is used, without frame missing also becomes possible.
1　　Transmission Device
2　　Communication Cable
3　　Image Pickup Device
4　　Network
5　　Cloud System
6　　Receiving Terminal
11　　Receiving Unit
12　　Storage Unit
13　　Processing Unit
14　　Transmitting Unit
31　　Image Pickup Unit
32　　Encoding Unit
33　　Output Unit

Claims (11)

  1. 　　A moving image transmission device which is connected with an image pickup device directly or via a network and transmits a moving image to a receiving terminal via a network, the image pickup device including image pickup means configured to pick up a moving image, encoding means configured to perform encode processing to a picked-up moving image so as to generate first stream data, and output means configured to output the generated first stream data, the moving image transmission device comprising:
    　　receiving means configured to receive the first stream data from the image pickup device;
    　　storage means configured to store the received or generated stream data;
    　　processing means configured to perform re-encode processing to the first stream data stored in the storage means so as to generate the compressed second stream data; and
    　　transmitting means configured to transmit the second stream data,
    　　wherein the re-encode processing includes thinning processing of thinning out a color difference signal of YUV image data.
  2. 　　The moving image transmission device according to Claim 1, wherein the re-encode processing includes thinning processing of converting image data of the first stream data into YUV image data composed of a luminance signal and a color difference signal and thinning out the color difference signal.
  3. 　　The moving image transmission device according to Claim 1,
    　　wherein the first stream data is of YUV 4:2:2 format or YUV 4:2:0 format which is obtained by converting the picked-up image data into YUV image data composed of a luminance signal and a color difference signal and thinning out the color difference signal, and
    　　the re-encode processing includes thinning processing of further thinning out the color difference signal from image data of the first stream data.
  4. 　　The moving image transmission device according to Claim 2 or 3, wherein the second stream data is of YUV 4:2:0 format.
  5. 　　The moving image transmission device according to any one of Claims 2 to 4, wherein the re-encode processing includes reduction processing of reducing a size of the image data after the thinning processing.
  6. 　　The moving image transmission device according to any one of Claims 1 to 5, comprising a router function.
  7. 　　The moving image transmission device according to Claim 6, the moving image transmission device being controllably connected with a cloud system via the network, wherein the transmitting means comprises means for uploading the second stream data to the cloud system.
  8. 　　A moving image transmission method of transmitting first stream data which is distributed from a moving image pickup device connected with a network, the moving image transmission method comprising:
    　　a reception step of receiving the first stream data from the moving image pickup device;
    　　a temporary storage step of temporarily storing the received first stream data;
    　　a processing step of performing re-encode processing to the stored first stream data so as to generate the compressed second stream data;
    　　a storage step of storing the second stream data; and
    　　a transmission step of transmitting the second stream data,
    　　wherein the re-encode processing includes thinning processing of thinning out a color difference signal of YUV image data.
  9. 　　The moving image transmission method according to Claim 8,
    　　wherein the first stream data is of YUV format constituted of image data which is obtained by converting the picked-up image data into YUV image data composed of a luminance signal and a color difference signal and thinning out the color difference signal, and
    　　the processing step includes a step of further thinning out the color difference signal from image data of the first stream data and reconverting the image data into YUV format.
  10. 　　The moving image transmission method according to Claim 9, wherein the processing step includes a step of reducing a size of image data of the first stream data after further thinning out the color difference signal from the image data.
  11. 　　The moving image transmission method according to Claim 10, wherein the transmission step includes:
    　　a step of uploading the second stream data to a cloud system which is controllably connected via a network; and
    　　a step of transmitting the second stream data in response to a request from a receiving terminal.
PCT/JP2017/017762 2016-05-12 2017-05-10 Moving image transmission device and moving image transmission method WO2017195844A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2016096139A JP6045051B1 (en) 2016-05-12 2016-05-12 Moving picture transmission apparatus and moving picture transmission method
JP2016-096139 2016-05-12

Publications (1)

Publication Number Publication Date
WO2017195844A1 true WO2017195844A1 (en) 2017-11-16

Family

ID=57543846

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2017/017762 WO2017195844A1 (en) 2016-05-12 2017-05-10 Moving image transmission device and moving image transmission method

Country Status (2)

Country Link
JP (1) JP6045051B1 (en)
WO (1) WO2017195844A1 (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060268300A1 (en) * 2005-05-27 2006-11-30 Fuji Photo Film Co., Ltd. Moving picture real-time communications terminal, control method for moving picture real-time communications terminal, and control program for moving picture real-time communications terminal
US20090174811A1 (en) * 2007-12-11 2009-07-09 Chih-Ta Star Sung Method and apparatus of bayer pattern direct video compression
US20090189981A1 (en) * 2008-01-24 2009-07-30 Jon Siann Video Delivery Systems Using Wireless Cameras
US20110007159A1 (en) * 2009-06-06 2011-01-13 Camp David M Video surveillance system and associated methods
US20110292287A1 (en) * 2003-03-20 2011-12-01 Utc Fire & Security Americas Corporation, Inc. Systems and methods for multi-stream image processing
JP2012090172A (en) 2010-10-21 2012-05-10 Canon Inc Monitoring camera, monitoring camera control method, and network camera system

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4251859B2 (en) * 2002-01-15 2009-04-08 ソニー株式会社 Data processing apparatus and method
JP4861723B2 (en) * 2006-02-27 2012-01-25 池上通信機株式会社 Monitoring system

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110292287A1 (en) * 2003-03-20 2011-12-01 Utc Fire & Security Americas Corporation, Inc. Systems and methods for multi-stream image processing
US20060268300A1 (en) * 2005-05-27 2006-11-30 Fuji Photo Film Co., Ltd. Moving picture real-time communications terminal, control method for moving picture real-time communications terminal, and control program for moving picture real-time communications terminal
US20090174811A1 (en) * 2007-12-11 2009-07-09 Chih-Ta Star Sung Method and apparatus of bayer pattern direct video compression
US20090189981A1 (en) * 2008-01-24 2009-07-30 Jon Siann Video Delivery Systems Using Wireless Cameras
US20110007159A1 (en) * 2009-06-06 2011-01-13 Camp David M Video surveillance system and associated methods
JP2012090172A (en) 2010-10-21 2012-05-10 Canon Inc Monitoring camera, monitoring camera control method, and network camera system

Also Published As

Publication number Publication date
JP2017204767A (en) 2017-11-16
JP6045051B1 (en) 2016-12-14

Similar Documents

Publication Publication Date Title
US10574990B2 (en) Image processing device and method with a scalable quantization matrix
KR102393150B1 (en) Image processing apparatus and method
US11706448B2 (en) Image processing apparatus and image processing method
JP6780761B2 (en) Image coding device and method
US20220394299A1 (en) Image processing apparatus and method
RU2641259C2 (en) Image processing device and method
US20230308646A1 (en) Image encoding device and method and image decoding device and method
WO2015104891A1 (en) Image processing device and image processing method
RU2663341C2 (en) Image processing device and method
KR20220104311A (en) Image encoding device, image encoding method and recording medium
KR102549670B1 (en) Chroma block prediction method and device
JP6891895B2 (en) Image processing device and image processing method
US20150030068A1 (en) Image processing device and method
Lin et al. Novel chroma subsampling strategy based on mathematical optimization for compressing mosaic videos with arbitrary RGB color filter arrays in H. 264/AVC and HEVC
US20150043638A1 (en) Image processing apparatus and image processing method
JP4799191B2 (en) Communication terminal, communication system, and communication method
WO2017195844A1 (en) Moving image transmission device and moving image transmission method
US20150139310A1 (en) Image processing apparatus and image processing method

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17727392

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 17727392

Country of ref document: EP

Kind code of ref document: A1