JP2001145073A - Moving picture transmission method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method to transmit a moving picture with excellent display quality of less missing frames through a mobile phone even under a condition where a transmission speed of data is limited. SOLUTION: Interleave processing is applied to image data expanded in a transmission frame memory 45a on the basis of a plurality of preset interleave patterns so as to transmit the data in a way that the data quantity to be sent per frame is minimized. On the other hand, a receiver side expands the received data in a reception frame memory 45b while applying interpolation processing to the interleaved data according to a procedure corresponding to the interleave patterns used for interleaving the data by a transmitter side.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for transmitting a moving image via a portable telephone.

[0002]

2. Description of the Related Art Conventionally, this kind of moving image transmission is provided as an additional function in a portable telephone after a video signal input through a video camera is compressed using a predetermined data format such as MPEG. In general, the communication is performed using data communication in a non-talking mode or a packet transmission mode.

[0003]

However, the data transmission speed of a portable telephone is limited to a maximum of 64 Kbps even for a PHS type, and is too small compared to the amount of data to be transmitted. As described above, there is a disadvantage that the transmission of data by simply compressing image data and transmitting data as described above results in severe frame drop images.

That is, the conventional image transmission is based on a method of transmitting each frame image accurately, and is a method of simply compressing and transmitting image data of each frame.
In the case of an image close to a still image, the number of dropped frames is small. However, when the image is changed to an image with little redundancy and intense movement, the image quality is extremely deteriorated.

The present inventor has conducted an experiment in view of such inconveniences. As a result, the amount of data per frame is reduced as much as possible irrespective of the redundancy of image data, while the number of frames that can be transmitted is maintained at, for example, 15 frames or more per second. It has been found that by making it possible, transmission of a moving image with excellent image quality without discomfort is performed.

The present invention has been made on the basis of the above-described knowledge, and thins out image data in accordance with a preset procedure so as to reduce the amount of data to be transmitted per frame as much as possible. It is an object of the present invention to provide a moving image transmission method excellent in display quality by complementing drawn data with a preset reference.

[0007]

SUMMARY OF THE INVENTION According to the present invention, there is provided a method for transmitting a moving image, comprising the steps of: loading a directly input video signal on a frame memory for transmission; Performing a thinning process on the image data in accordance with a preset procedure, and then transmitting the data to the outside via the mobile phone; and transmitting the data to the image data input from the outside via the mobile phone. After performing a complementation process of data corresponding to the data thinning process performed on the side, the process of expanding the data in the frame memory for reception, and the image data expanded in the frame memory corresponding to the scanning position on the display means. Reading and displaying.

There are a plurality of procedures for the above-described image data thinning processing. Prior to data transmission, one of the procedures with the highest thinning efficiency in the image data currently being processed is automatically selected. It is preferable to configure.

The above-mentioned procedure for the image data thinning process includes a plurality of types of pattern data for specifying the dot positions to be transmitted on the display screen. While transmitting transmission data including data created using the pattern data corresponding to the pattern data and data specifying the type of the pattern, the receiving side of the transmission data performs a complementary procedure specified from the type of the pattern. The data is complemented using the data.

[0010]

As described above, according to the present invention, the image data is decimated according to a preset procedure, and the amount of data to be transmitted per frame is reduced as much as possible. By complementing with the set criteria, a moving image with excellent display quality is enabled.

[0011]

BEST MODE FOR CARRYING OUT THE INVENTION A moving picture transmission method according to the present invention will be described in more detail based on an example implemented in a moving picture transmission system 10 shown in FIG. It is needless to say that the present invention can be implemented in substantially the same manner for various systems that transmit and receive moving image data.

The moving picture transmission system 10 includes first and second two sets of moving picture transmission devices 12 whose configuration will be described in detail later.
a and 12b, and by connecting the two moving image transmission devices 12a and 12b wirelessly via at least one mobile phone 11, it is possible to display a moving image at a remote location without limitation of the imaging position. It is assumed that.

As shown in FIGS. 2 and 3, the moving image transmitting apparatus 12 has a moving image transmitting unit 15 in which a data processing means 13 is housed in a main body case 14 as a basic unit. One or more of the means 16, the operation means 17, and the display means 18 may be connected as necessary via the connection terminal group 19 or the connector, or may be connected to the body case 14 integrally or via the connector. By connecting the circuit boards and permanently disposing them, a set of moving image transmission devices 12 is configured. Further, as shown in FIG. 1B, a secondary battery 20 is built in the main body case 14 so as to be portable and portable.
As it can be configured as a fixed type driven only by one,
It has a flexible configuration.

With such a configuration, for example, in the system of FIG. 1A, the first and second moving image transmission devices 1
Both 2a and 12b are equipped with all of the above-described means for enabling connection, so that they can be used as a videophone or videoconference system based on two-way equal data communication.

On the other hand, in the system shown in FIG.
While the first moving image transmission device 12a is provided at a position where neither the wired public line 22 nor the commercial AC power supply 21 is provided, the second moving image transmission device 12b provided at a base station equipped with these facilities is used. A moving image can be displayed while remotely controlling the imaging state of the first moving image transmission device 12a.

In this case, the moving image transmission unit 15a on the first moving image transmission device 12a side has a minimum necessary configuration, and the capacity and power consumption of the main body case 14 are reduced to enable driving by the secondary battery 20. As enhances portability and mobility. Further, it has a plurality of imaging means 16 and transmits data via the mobile phone 11.

On the other hand, on the side of the second moving image transmitting apparatus 12b, the image pickup means 1 is provided in the moving image transmitting unit 15b.
As many as necessary functions except for 6 are housed together as much as possible, and driving by a commercial AC power supply 21 and data transmission via a wired public line 22 are performed.

In the following, as shown in FIG. 3, a moving image transmission unit 15 constructed in the form of a handy personal computer having the above-mentioned display means and operation means integrated in a main body case 14 is exemplified.

However, the present invention is not limited to this, and as shown in FIG. 7, the image pickup means is also incorporated in the main body case 14 so as to be a handy type video camera, or conversely, the above three means are excluded from the main body case 14. The capacity of the main body case 14 and the moving image transmission unit 1
5 It is also possible to suppress the power consumption of the device itself to a necessary minimum.

The moving image transmission unit 15 includes a data processing unit 13 inside a flat main body case 14 having a substantially rectangular cross section.
To store. Further, a display 24 made of a liquid crystal display panel is provided as a display means on the front side of the main body case 14, and the supporting legs 60 are provided on the back side so as to be able to freely come and go.

A keyboard 23 provided as operating means is swingably attached to the main body case 14 so as to close the display screen 61 of the display 24, and data can be input / output to / from the data processing means 13 on the side surface. Are arranged.

With this configuration, when stored, the keyboard 23 and the support legs 60 come into close contact with the main body case 14 to reduce the number of projections, thereby improving the storage efficiency. On the other hand, at the time of use, the main body case 14 can be held in an inclined state by the support legs 60, and the keyboard 23 is opened, so that the display screen 61 of the display 24 is tilted when the main body case 14 is placed on a desk. The keyboard 23 is opened horizontally so that the keyboard 23 can be operated while looking in from above.

In FIG. 3, a touch panel 25 is further provided as the operation means 17 so as to cover the entire display screen 61 of the display 24, thereby enabling direct instruction operation with a fingertip and driving the camera. Means 5
6 shows an example in which a joystick 26 specially configured for the operation 6 is connected, but the present invention is not limited to this, and various operation means such as a mouse and a trackball can be used.

The data processing means 13 has basically the same construction as that of a conventional personal computer as shown in FIG. 4, and connects a CPU 28, an internal memory 29 or various controllers 30 via a bus 27. CPU according to a program stored in internal memory 29
Reference numeral 28 performs basic data processing operations necessary for image processing.

Further, data that needs to be permanently stored or data that exceeds the capacity of the internal memory 29 is stored as a memory means 32 using a memory card 31 which can be inserted into and removed from the main body case 14.

An extension board is detachably connected to the bus 27 via a general-purpose or dedicated connector (not shown), so that image information can be input / output through a terminal group 19 provided on the extension board. An extended data processing operation corresponding to input / output data is enabled.

The extension board illustrated in FIG.
3 input terminal 19a and image signal 33 output terminal 19b
The input terminal 19c of the audio signal 34a, and the audio signal 34
b, an input / output terminal 19e for transmission data 35 transferred to and from the mobile phone 11, a connection terminal 19f for the joystick 26, and an input / output terminal 19g for the control signal 36. The connector of the shape corresponding to them is exposed from the peripheral surface position.

Here, the input terminal 19a of the image signal 33a
And a plurality of sets (four sets in the present embodiment) of an audio signal 34a and an input terminal 19c of the audio signal 34a. As shown in FIG.
Are connected, the moving image and the sound at the four locations are input in parallel to the multiplexer 41 in the video signal conversion circuit 40 and the multiplexer 42 in the sound data processing unit 47 constituting the image data processing unit 39, respectively. I am trying to.

From the four sets of image signals 33a and sound signals 34a input in parallel, only one set of signals is selectively extracted according to designation by the control unit 43 and the multiplexers 41 and 42. In the following, for simplicity, the operation will be described for the case where one set of the image signal 33a and the sound signal 34a is input, but the same applies to the case of a plurality of sets.

The NT input via the video camera 37
The analog image signal 33a such as the SC system is sequentially converted from analog to digital by a video decoder 44 provided in a video signal conversion circuit 40, and stored in a transmission frame memory 45a. Circuit 5
5 stores the image data 4 from the frame memory 45 as necessary.
6 is read out, digitally displayed as a moving image on a dedicated display 24, or converted again from digital to analog, and an analog display is displayed as a moving image on a monitor (not shown) connected to the output terminal 19b of the image signal 33. Make it possible. It should be noted that the video camera 37 is not limited to the analog type as described above, and it is a matter of course that the digital camera can be implemented in substantially the same manner.

The audio signal 34a input via the microphone 38 is converted from analog to digital by the A / D conversion circuit 48 of the audio data
a, and the multi-data simultaneous communication processing unit 50 shown in FIG.
After synthesizing with the image data 58 stored in 5a, the data is converted into a data format corresponding to the communication mode of the mobile phone 11 by the modem 51, and the data is transmitted through the mobile phone 11.

On the other hand, the transmission data 35 transmitted from the other moving image transmission device 12 via the mobile phone 11 as described above is the same as the transmission of the transmission data 35 described above. The processing unit 50 separates the image data 58 into sound data 49 and the inputted sound data 49b is subjected to digital / analog conversion in a D / A conversion circuit 52, so that sound is generated by sound generation means 53 such as an earphone or a speaker. Can be reproduced as

The image data 58 separated from the transmission data 35 is subjected to predetermined data processing, which will be described later, in a transmission / reception image data processing circuit 54 of the image data processing unit 39, and is developed in a reception frame memory 45b. The necessary image data 46b is extracted to the synthesizing circuit 55, so that the transmission frame memory 4 described above is obtained.
The image data 46a extracted from the image data 5a and the image data 46a and 46b alone are combined with the display 2
4 is displayed.

Each of the video cameras 37 is provided with a camera driving means 56, and the imaging direction or range of the video cameras 37 corresponding to the instruction operation by the push button or lever operation of the joystick 26 provided as a dedicated operation means. It is also possible to configure such that control of imaging conditions such as is performed.

In this case, the operation state of the operation unit 17 is determined by the control unit 43, and if it is determined that the operation is the operation of the video camera 37, the control signal 36 is directly transmitted to the camera driving unit 56. And the necessary driving operation is performed.

Conversely, if it is determined that the operation is for another moving image transmission device 12, the control data 43 corresponding to the operation data is transmitted from the control unit 43 to the multi-data simultaneous communication processing unit 50, and the modem 51 and the portable telephone 11 are transmitted. The moving image is transmitted to the other moving image transmission device 12 via the other terminal. Then, as in the case described above, the control data 57 separated from the transmission data 35 in the multi-data simultaneous communication processing unit 50 is input to the control unit 43. Then, a control signal 36 corresponding to the control data 57 is sent from the control unit 43 to the camera driving unit 56, and a necessary driving operation for the video camera 37 is executed.

A moving image during a period in which active image monitoring is not performed, such as a period during which moving image transmission and display are stopped, and a video camera that is not selected among a plurality of video cameras 37. For a captured moving image or a moving image selected by the operator to save, a video is generated each time a predetermined trigger such as detection by a sensor or a switch operation or a predetermined condition such as a set time interval is satisfied. The moving image input from the camera 37 is captured as a still image and can be stored in the memory card 31.

It is preferable that the data can be transmitted in addition to or instead of the above-mentioned moving image in accordance with an instruction using the operating means 17 from the moving image transmitting apparatus 12 at a remote position.

The method of synthesizing the transmission / reception image data 58, the sound data 49, and the control data 57 via the multi-data simultaneous communication processing unit 50 is performed in substantially the same manner as in the prior art. Here, the detailed description is omitted.

According to the present invention, the transmission / reception image data processing circuit 54 further transmits the image data 46 expanded in the transmission frame memory 45a via the mobile phone 11 before the data transmission. It is characterized in that data is thinned out and compressed, and conversely, data is developed in the frame memory 45b for reception.

That is, when reading out the image data 46 expanded in the transmission frame memory 45a, not all data is read out but read out in a state where data is thinned out according to a predetermined algorithm described later and transmitted per frame. After the amount of data to be reduced is reduced as much as possible, data compression is performed in substantially the same manner as before.

On the other hand, on the receiving side of the image data 58,
After decompressing the data compressed as described above, the data decimated at the time of transmission is expanded in the reception frame memory 45b while complementing the data using a predetermined algorithm, thereby preventing the deterioration of the image quality for each frame as much as possible. However, image transmission with few dropped frames is realized.

Here, as the above-described data thinning procedure, the present invention employs a combination of the following methods.

The first method utilizes the fact that images are generally continuous in many cases. If the direction in which images tend to be continuous is in the horizontal direction of the display screen 61, data in dot units is used. If the unit data is oblique, the dot data in both the vertical and horizontal directions are made to correspond to the inclination angle, or if no continuous direction is specified, the dot data in the vertical and horizontal directions are evenly distributed, for example, every other dot data. The image data 58 is transmitted in a state of being thinned out according to a predetermined standard such as

On the other hand, at the time of reception, it is assumed that the data at the position decimated at the time of transmission changes at a predetermined rate such as a linear change or a logarithmic change, and the data is complemented using these criteria. Is what you do.

In the second method, the positions of dot data or line data to be thinned out differ from frame to frame as described above, so that while data for a plurality of frames is received, the data thinned during transmission is automatically converted. It is based on the fact that it is complemented by

Further, as a third method, instead of transmitting the data of the entire screen evenly, in the range where detailed information is required, the above-described thinning frequency is reduced to transmit the moving image as faithfully as possible. Conversely, for a range that does not require much attention, the thinning data is increased as much as possible to reduce the amount of transmission data.

The selection of the range in which the difference in transmission image quality is made can be performed by automatically selecting a range in which image data changes little between frames or a range that is not important as an observation target, for example, by taking a difference in inter-frame data. Alternatively, it is specified by a range specification on the display screen of the operator.

In the present invention, instead of using the above-mentioned three thinning methods alternatively, all the thinning procedures are integrated, and one or three methods can be performed simultaneously in parallel. For this purpose, a plurality of thinning-out patterns are created which enable to specify the position of data in all dot data constituting the display screen to be transmitted.

Then, before the start of data transmission, every set time, every time a predetermined number of frames are transmitted, or automatically at a predetermined timing such as immediately after the imaging range is changed by the camera driving means 56, It is determined whether it is appropriate to apply the thinning method. Further, a thinning pattern corresponding to the determination result is selected to perform a data thinning operation, and data is transmitted to the receiving side together with selection data for specifying which pattern has been selected.

On the other hand, on the receiving side, there is provided data for complementary processing individually corresponding to each of the above-described thinning patterns. Then, from the selection data received from the transmitting side, it is determined which thinning pattern is used to perform the data thinning processing, and the complementary processing of the received data is performed based on the complementary processing data corresponding to the thinning pattern. , The moving image is restored.

For example, in a frame immediately after the apparatus is started, all image data is transmitted. At a predetermined determination time such as the next frame, the first method is used to determine in which direction on the display screen continuity is recognized, and a thinning pattern corresponding to the determination is selected to transmit data. do.

In the next frame, a thinning pattern having a complementary relationship with the thinning pattern is selected and transmitted using the second method described above. At the same time, if a display range in which there is no or little change in data between frames is determined, a thinning pattern from which data in that area is selectively excluded is selected and used for data transmission.

[Brief description of the drawings]

FIG. 1 is a schematic diagram showing an example in which the present invention is applied to a moving image transmission system.

FIG. 2 is a block diagram illustrating an overall configuration of a moving image transmission device.

FIG. 3 is a partially broken perspective view showing an example in which the moving image transmission unit is configured in a handy personal computer format.

FIG. 4 is a block diagram illustrating an example of a data processing unit.

FIG. 5 is a block diagram illustrating a specific configuration of a video signal conversion circuit and an audio data processing unit.

FIG. 6 is a block diagram illustrating a specific configuration of an image data processing unit.

FIG. 7 is a perspective view illustrating an example in which the moving image transmission device is configured in a video camera format.

[Explanation of symbols]

 REFERENCE SIGNS LIST 10 moving image transmission system 11 mobile phone 12 moving image transmission device 13 data processing means 14 body case 15 moving image transmission unit 16 imaging means 17 operation means 18 display means 19 connection terminal 20 secondary battery 21 commercial AC power supply 22 public line 23 keyboard 24 display 25 touch panel 26 joystick 27 bus 28 CPU 29 internal memory 30 controller 31 memory card 32 storage means 33 image signal 34 acoustic signal 35 transmission data 36 control signal 37 video camera 38 microphone 39 image data processing unit 40 video signal conversion circuit 41 multiplexer 42 multiplexer 43 control unit 44 video decoder 45 frame memory 46 image data 47 sound data processing unit 48 A / D conversion circuit 49 sound data 50 various kinds of data Data simultaneous communication processing unit 51 modem 52 D / A converter 53 sound generating means 54 transmits and receives the image data processing circuit 55 the image synthesizing circuit 56 camera drive unit 57 control data 58 the image data 60 support legs 61 display screen

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) H04N 5/225 H04B 7/26 109M 5K067 7/24 H04N 7/13 Z 5K101 F-term (reference) 5C022 AA01 AA12 AB62 AB66 AC01 AC27 AC54 AC69 AC73 5C059 KK01 LB05 LB07 LB12 LB15 MA00 MA05 PP04 RC11 SS07 SS10 TA06 TA07 TA09 TA23 TB04 TC02 TC34 TC37 UA02 UA05 UA38 5C064 AA01 AA02 AA06 AB04 AC04 AC06 AC09 AC12 AD01 AC06 AD07 AC12 AD01 A06 BD02 5K027 AA11 BB03 CC08 HH29 5K067 AA13 BB04 BB21 CC04 CC08 DD52 EE02 EE10 FF02 FF31 5K101 KK04 LL12 NN06

Claims (3)

[Claims] 1. A step of expanding a video signal directly input on a frame memory for transmission, and performing a thinning process on image data expanded on the frame memory for transmission in accordance with a preset procedure. Then, a step of transmitting data to the outside via the mobile phone, and a process of complementing data corresponding to the above-described data thinning-out processing performed on the data transmission side with respect to image data input from the outside via the mobile phone. And transmitting the image data expanded in the frame memory to a scanning position on a display means, and reading and displaying the image data expanded in the frame memory. 2. There are a plurality of procedures for the above-described image data thinning processing. Prior to data transmission, one of the procedures with the highest thinning efficiency in the image data currently being processed is automatically selected. The moving image transmission method according to claim 1. 3. The above-described procedure for thinning image data includes a plurality of types of pattern data for specifying a dot position to be transmitted on a display screen. Data created using pattern data corresponding to the procedure,
3. The transmission data including data specifying the type of the pattern is transmitted, and on the receiving side of the transmission data, the data is complemented by using a complementing procedure specified by the type of the pattern. Video transmission method.