JP2003299062A - Video converter and its method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To properly distribute video images even in a terminal on a different network. <P>SOLUTION: An image conversion method which distributes an image data picked up by an imaging device in real time is as follows: the image data of a first image data system is received from a server connected to the imaging device through a first network; the received image data of the first image data system is converted to a second image data system corresponding to a second network; and the converted image data is distributed to a terminal on the second network. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video distribution technique for distributing video data via a network.

[0002]

2. Description of the Related Art The applicant of the present invention has proposed a system for delivering a captured live image using a communication infrastructure such as the Internet and for instructing a camera setting and a camera operation for image capturing.

[0003] Further, by providing a higher radio wave utilization efficiency and communication band than the conventional mobile phone service, a service that enables data communication such as Internet access while making a telephone call is becoming widespread. For such services, a connection form called multi-access is prepared, and by using this, telephone communication is possible while performing data communication such as web browsing.

Further, the processing capability of the portable terminal itself has been strengthened, and the work which has been performed by a PC (personal computer) up to now can be processed by the portable telephone terminal. For example, mobile phone terminals equipped with functions such as mail, web browsing, and video transmission / reception are provided. In addition, 3G-PP (Third Generator)
ation Partnership Project
In t), 3G-324M (for circuit switching) is established as a video transmission standard for third-generation mobile phones, and the video transmission technology for mobile phones is being standardized.

[0005]

However, when video information provided on the Internet is provided to mobile phone terminals, conversion of video information is performed by grasping characteristics such as narrow communication band and screen size of the mobile phone service. In addition, there is a problem that an appropriate video service cannot be provided to the mobile phone terminal unless the transmission control is performed.

The present invention has been made in view of the above problems, and appropriately provides video information distributed on the Internet for mobile phones.

[0007]

In order to achieve the above-mentioned object, according to the present invention, in a video conversion device for delivering video data imaged by an imaging device in real time, a server connected to the imaging device. Receiving means for receiving the video data of the first video data method from the first network via the first network, and the video data of the first video data method received by the receiving means corresponding to the second network. Control means for converting the video data into the second video data system and delivering the converted video data to the terminals on the second network.

Further, in a video conversion method for delivering video data picked up by an image pickup device in real time,
Video data of a first video data system is received from a server connected to the imaging device via a first network, and the received video data of the first video data system is compatible with a second network. And a video conversion method for converting the converted video data to a terminal on the second network.

[0009]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the accompanying drawings. Here, terms used in the description of the present embodiment will be described in advance below.

Compression in the MPEG-4 codec frame and prediction of the moving direction of the object in the image by searching the motion vector,
A method of compressing data by saving only the vector. In other words, it is a method in which the image of the frame itself is decomposed into objects (for example, a background, a person, and a building that make up the image), and the signals are compressed using a signal that is different between the frames according to each object.

In addition, due to the spread of video transmission / reception terminals from a portable information terminal connected to a mobile communication network to a PC connected to a broadband Internet, high compression capable of covering a wide bit rate of several tens of kpbs to several tens of Mbps. MPEG-4 was established by ISO in 1999 as a moving image compression coding method having coding efficiency and strong resistance to transmission path errors such as wireless communication and the Internet. This video distribution service using MPEG-4 is provided for personal information terminals PDA and mobile phone terminals. For example, a service is provided between mobile phone terminals for mutually transmitting and receiving video using MPEG-4.

RTP (Real-) as a transmission protocol for an application that delivers information (especially assuming video and audio etc.) in real time on the RTP / RTCP network.
(time Transport Protocol)
However, according to IETF RFC1889 ¨ RTP: AT
transport Protocol for Rea
It is recommended as l-Time Applications. RTP is a protocol that includes a mechanism for observing the state of a communication path using a time stamp of a terminal that delivers information and adjusting so that information can be delivered in an appropriate state.

RTSP (Real-time Str)
aiming Protocol) IETF by RFC2326-Real Tim
e Streaming Protocol (RTS
P) ¨ is recommended. When transmitting information in real time, RTP is supposed to unidirectionally broadcast (or multicast) from the server to the client, whereas RTSP allows unicast and bidirectional transmission between the server and client. I am assuming. In other words, in RTSP, the client communicates with the server to rewind the video content,
It is possible to skip to another chapter.

Further, in RTSP, R is used in the transmission layer.
It may be used as a control protocol for an application that delivers information (especially assuming video and audio etc.) in real time using TP or the like.

(First Embodiment) A form of an image distribution system for converting a live image sent from a camera server arranged on a network to a mobile phone terminal and relaying it will be described.

In this embodiment, a live image relayed by an image conversion server for converting and relaying an image is used as an image format (codec, screen size, etc.) for a mobile phone terminal.
And conversion of communication method (protocol). And, in the video conversion server of the present embodiment, as its characteristic,
(1) Use of source video with camera control, (2)
The point is to convert continuous JPEG data (Motion JPEG method: a method of compressing video data within a frame (compressing each frame as a still image)) into MPEG-4 data.

FIG. 1 is a diagram showing an example of a video distribution system according to the present embodiment.

In FIG. 1, the camera servers 101, 10
2 delivers the video imaged by the connected camera as a moving image in real time on the network by the Motion JPEG method. A PC (personal computer) 200 is a client for receiving an image distributed by the Motion JPEG method.

The video conversion server 400 is the camera server 1.
The video data provided by 01 and 102 are converted to mobile phone terminals and distributed to the mobile phone terminals. The gateway 500 is a network and a mobile telephone network (64 kbp) whose data communication speed is limited by the network.
s) is a gateway that mediates between
Reference numerals 01 and 602 denote clients that receive the video converted by the video conversion server. The mobile phone terminal 6
The viewer program on 01,602 typically
The program is installed at the time of shipment from the factory, but the Java program or the like may be downloaded and executed at the time of execution (at the time of use).

The camera servers 101 and 102, and the PC 200 or mobile phone terminal 601 as a client
With respect to 602, a request is sent from the client to the camera server via the network, and when the request is accepted, the camera server delivers the video data to the client so that the client can view the camera video. In addition, a camera control command is sent from the client to the camera server, which enables operations such as zooming, panning, and tilting of the camera.

The network shown in FIG. 1 may be an intranet operated within a company or an organization, or may be the Internet that connects the world widely. In addition, the video conversion server is an Internet exchange (IX) or data center (IDC: Internet).
t Data Center) and is designed to reduce the communication load.

FIG. 2 shows an example of a hardware configuration for operating the video conversion server 400. More specifically, a CPU that executes various processes by programs
201, a RAM 202 serving as a main storage device, an FD device 203 for loading a program from an external recording medium,
An HD device 204 as a secondary storage device, a network I / F 205 for connecting to a network, and the like. Further, although not shown, in order to improve operability, an input device such as a controller such as a keyboard or a mouse for performing setting or the like, or a display device such as a display may be provided.

FIG. 3 shows an example of the hardware configuration of the camera servers 101 and 102. More specifically, the CPU 30 that executes various processes by the program
1, a RAM 302 serving as a main storage device, an FD device 303 for loading a program from an external recording medium, an HD device 304 as a secondary storage device, a network I / F 305 for connecting to a network, and a camera for imaging a subject 305, a video capture board 306 that acquires a video from the camera, and a serial I / F 307 that outputs a control command to the camera 305. Further, although not shown, in order to improve operability, an input device such as a controller such as a keyboard or a mouse for performing setting or the like, or a display device such as a display may be provided.

FIG. 4 shows an example of the hardware configuration of the mobile phone terminals 601 and 602 that operate as clients. More specifically, a wireless communication I / F 401 for connecting to a mobile phone network, a RAM for storing programs and data, a storage device 402 including a flash memory, and a C for executing various processes by the programs.
The PU 403, operation members such as buttons and switches arranged on the mobile phone terminal, a display 404, a display controller 405 for controlling display of the display, and a voice input / output device 408 including a microphone and a speaker.

FIG. 5 is a diagram schematically showing an outline of a software-like process flow of the mobile phone terminals 601, 602, the video conversion server 400, and the camera servers 101, 102 in the image delivery system of the present embodiment. . In the image distribution system of the present embodiment, the mobile phone terminals 601 and 6
02 for controlling the cameras connected to the camera servers 101, 102, and the camera servers 101, 1
From 02 to the mobile phone terminals 601 and 602.

The camera servers 101 and 102 include two modules, a camera control server 503 that controls the control of the camera and a video server 508 that controls the delivery of the video.

The mobile phone terminals 601 and 602 include two modules, a camera operation unit 501 corresponding to a camera control command and a camera status notification, and an image display unit 504 for displaying a received camera image.

Further, the video conversion server 400 acquires a video from the camera control unit 502 that relays camera control or issues a camera control command, and a video system of the acquired video for a mobile phone terminal. Modules for converting and transmitting the same to the mobile phone network (video acquisition unit 507, video conversion unit 506, video transmission unit 505) are included.

FIG. 6 is a flow chart of operation processing as a client in the mobile phone terminals 601 and 602.

In step S601, first, the identifier of the video conversion server 400 is obtained. For this, the user may directly input the identifier by operating the key of the operation unit 407, or may select the identifier included in the mail or the web page. The identifier in this case is typically the video conversion server 400 within the mobile phone network.
However, it may be a URL for identifying the video conversion server 400.

Next, in step S602, the video transmission unit 505 of the video conversion server 400 is accessed via the gateway 500 based on the obtained identifier.

Then, in step S603, the attribute information of the video data provided by the video conversion server 400 is acquired. This is, for example, RTSP-DESCRIBE
¨ By sending and receiving messages.

In step S604, the video conversion server 400 is requested to prepare for video transmission, and information necessary for video reception is obtained. This is, for example, RTSP-SETUP
¨ By sending and receiving messages. The reception information obtained here includes a video transmission protocol and a transmission level parameter.

In step S604, a video display thread is created. The video display thread opens the video data receiving port based on the obtained reception information and requests the video conversion server to start video transmission (step S).
611). Then, in step S612, the video data is received from the video conversion server 400 via the video data receiving port, the video data is decoded in step S613, and displayed on the display device 404 of the mobile phone terminal in step S614. Then, in order to continuously display the video from the video conversion server, the operations from step S611 to step S614 are repeated during the period from step S607 to step S610.

At step S605, the display of the image is started, and at the same time, via the gateway 500,
It connects to the camera control unit 502 of the video conversion server 400 based on the obtained identifier. Then, the operation processing thereafter is continued to the main loop which receives and executes the operation request from the user.

First, as the processing of the main loop, a user's command is received from the operation unit 407 in step S606. If this relates to camera control, step S
In 607, a command is issued to the camera control unit 502 of the video conversion server 400, and when it is related to video data, a command is issued to the video transmission unit 505 of the video conversion server 400 in step S608. If the user's operation changes the state of the mobile phone terminals 601, 602 itself (for example, turning on the backlight or terminating the client), the internal state is updated in step S609. When the processing of S609 to S611 is completed, S608
And returns to the operation input by the user.

Then, in S607, if the user operation is the end of the video receiving operation as the client,
The programs related to the operation are sequentially ended.

FIG. 7 is a flowchart of the operation processing of the camera control server 503 in the camera servers 101 and 102.

In step S701, the camera control server 503 first reads the operation setting information of the camera control server 503 from a specific file (system database such as a registry depending on the OS) at the time of startup, and starts the operation based on it. Here, the client PC 200 and the video conversion server 400 (the mobile phone terminal 60)
1, 602) to open a port for accepting a request from the client, and then proceed to step S702 to enter a request acceptance state from a client (PC 200 or video conversion server 400).

When a request (connection request or operation command request) is received, if the request is a connection request, the flow advances to step S703 to determine whether or not connection is possible. If not, a connection refusal error code is returned, and the process returns to step S702. If yes, go to step S704,
As the connection processing, a reception thread for receiving the command from the client is generated, and the client requesting the connection is registered, and then the process returns to step S702.

The operation receiving thread generated as described above receives a command from the corresponding client (step S707). When a command arrives, it receives it and passes it to the main program that operates the camera. The main program receives this in step S702, proceeds to step S705 in response to a camera operation command, performs camera control (camera pan, tilt, zoom, etc.) corresponding to the command, and outputs the result (whether the operation is successful or not). A code indicating failure or the like) to the operation reception thread that has received the camera operation request. Step S708
Then, the thread corresponding to this client sends back the control result to the client that has output the camera operation command.

The operation acceptance thread is step S7.
It is generated for each client registered in 04. Then, in the operation process of the main program, the state (for example, the pan / tilt / zoom value) changed by the control of the camera in step S706 is transmitted to all reception threads and output to all connected clients (step). S709). Upon receiving the connection end command from the client, each operation reception thread corresponding to the client notifies the main program of the connection end command and proceeds to step S710 to end its own reception thread.

In the handling of the operation command, it is possible to request the assignment of the camera operation right before the specific operation command is issued. This eliminates confusion in situations where multiple people require camera operation.

In this case, first, the client issues a camera operation right acquisition request command, and the camera control server 503 selects refusal, allocation, or waiting from the current camera control right allocation state. Reply to the client. Camera control is revoked at a specific pre-determined time or the shorter time it takes for the client to terminate the connection and be assigned to the next person waiting in line. The number of people waiting in line is also limited to a predetermined number (for example, 5 people), and more requests are rejected. The client can issue the operation command only after the camera control right is acquired and is deprived. Then, the camera control server accepts only the operation command from the client to which the camera control right is given. By such processing, a camera operation right allocation request can be issued.

Next, the operation processing of the video server 508 in the camera servers 101 and 102 will be described with reference to the flowchart of FIG.

In step S801, first, the operation setting information of the video server 508 is read out from a specific file (system database such as registry depending on the OS) at the time of startup, and the operation is started based on the information. Here, a thread that acquires, encodes, and stores video is generated (this thread is initially in a sleep state), and the PC 200 and the video conversion server 400 (mobile phone terminals 601 and 60).
The port that receives the request from 2) is opened, and then the request receiving state of step S802 is entered.

When the request (connection request or command request) is accepted, it is determined whether the request is a connection request. If the request is a connection request, it is determined in step S803 whether or not connection is possible.
If not, a connection refusal error code is returned and step S8
Return to 02. If yes, as a connection process in step S804, a session identifier for identifying a session for each client is generated, a video reception thread for receiving a command related to a video from the client is generated, and the client is registered. It returns to S802. At this time, if the request content is a connection request to the live video and the threads for acquiring and encoding the video are in the sleep state, the operation start of these threads is instructed before returning to S802.

In the generated client-compatible video reception thread, first, in step S807, a command is received from the corresponding client. When a command arrives, it receives it and passes it to the main program that performs video processing.

In the main program, step S802
In step S805, a command to perform video processing is received, and a setting change operation relating to video acquisition, encoding / transmission, etc. is performed. The changed result (a code indicating the success or failure of the operation) is transmitted to the image reception thread corresponding to the client that has received the command request. The thread corresponding to the client sends the result back to the client in step S808.

In the main program, step S804
In step S806, the image capture board 306 captures image data at a preset time interval according to an instruction to start an operation from a thread that acquires and encodes an image from the image capture board 306.
, And convert it to compressed data. This compressed data is then transmitted to all client-compatible threads connected to the live video.

In step S809, the thread corresponding to each client determines whether or not there is a next video frame transmission request from the client, and if there is a request, delivers the compressed data to the client. At this time, the video data to be delivered is the motion of QVGA size (320x240).
This is a JPEG compression method. When the thread corresponding to the client connected to the live video receives the request to send the next video frame from the client (this is generally sent back when the client completes receiving the compressed video data). A video frame transmission request flag is set to.

If a connection end command is received from the client, it is notified to the main program, and the thread of its own is ended in step S810.

FIG. 9 is a diagram exemplifying allocation of the operation units 407 of the mobile phone terminals 601 and 602 when operating as one of the clients of the camera servers 101 and 102.

When the client is operating, functions such as panning, tilting, zooming for the camera, requesting camera control right, backlight compensation, and connection termination are assigned to the operation unit 407 of the mobile phone terminal.

This screen is a UI for the user to perform an input operation in the operation processing of the mobile terminals 601 and 602 shown in FIG. 6, and the video data displayed in step S614 is displayed on the display device 404 of the mobile phone terminal. Displayed, the instruction from the operation unit 407 is received in S606, commands are generated in S609 to S611, and these commands are sent to the video conversion server 404.

FIG. 10 shows setting values used by the camera servers 101 and 102, that is, the camera control server 503 and the video server 50 in steps S701 and S801.
8 is a diagram showing an example of a display screen of a camera server setting program for setting the operation setting information read when 8 starts in a specific file (a system database such as a registry depending on the OS).
03, video server 508, various parameters (described later) relating to video quality, connection restrictions, etc. can be set. When the OK button is pressed, the set value is written to a specific file or registry, and if cancelled, the process ends without being written.

FIG. 11 is a flow chart showing the operation of the camera server setting program of FIG.

When the setting program is started, the first step S
1101 camera control server 503 and video server 5
08 specific file (OS)
Depending on the system database such as registry), it reads the setting information and sets it as internal data. Or later,
The loop of receiving the operation input of the user and executing the operation is repeated. In step 1102, the operation input by the user is waited for, and if there is input, it is received. Then, in step 1103, it is determined whether or not the input value is within an appropriate range. If not, an error message is output in step 1104. The output is performed, the value is returned, and the process returns to S1102 where the user waits for input.
If it is within the proper range, the internal data is updated to S1.
Return to 102 (the values that can be set here include the following items: TCP port number for communication for camera control, COM (serial) port connected to the camera, shutter speed, presence / absence of log information related to camera control, and log. File name, TCP port number for communication related to video, presence / absence of log information and log file name, frame rate that defines time interval for capturing video, and Q-F that determines compression quality
actor, screen size of original data for compression, maximum connection time for one client / client, number of people waiting for control right for camera control, occupied time for holding control right for one client, maximum number of connectable clients for video and camera control Such).

If the input from the user is the OK button, the process advances from step S1102 to step S1105 to write the updated internal data to a specific file or the like that stores the setting information regarding the camera control server and the video server, and in step S1106. Brings up a panel asking if you want to restart the camera server for the changes to take effect. When restarting, the camera control server, the video server, etc. are restarted in step S1107, and the setting program is ended in step S1108. If not restarted, S110
The process directly proceeds from S6 to S1108 and ends. In addition, S11
When the user's input in 02 is the cancel button, the process directly proceeds from S1102 to S1108 and ends.

FIG. 12 is a diagram schematically showing conversion of video data in the video conversion server 400. Video data (Motionio) transmitted from the camera servers 101 and 102
The nJPEG, QVGA size 320 × 240) is received via the communication stack for the camera servers 101 and 102 of the video conversion server 400, and orthogonal transformation processing is performed in the JPEG decoder. Then, the video data subjected to the orthogonal transform processing is MPEG set for the mobile phone.
Video data for mobile phones (MP
EG-4 simple-profile, QCIF size 176x144, 64 Kbps). Then, it is transmitted to the mobile phone client via the communication stack for the mobile phone network.

FIG. 13 is a flow chart showing the operation processing of the video conversion server 400. In the present embodiment, the PC 200 that does not require conversion processing can be connected to the video server 508 as a client in order to integrate the management of client registration.

In step S1301, the video conversion server 400 first reads the operation setting information of the video conversion server from a specific file (system database such as a registry depending on the OS) at the time of startup, and starts the operation based on it. Here, a communication port that receives requests from the mobile phone terminals 601 and 602 that are clients is opened, and the process advances to step S1302 to enter the request acceptance state.

When the request (RTSP message or the like) is accepted in step S1302a, the process advances to step S1303 to determine whether the requesting client (PC terminal or mobile phone terminal) can be connected. If not, the connection refusal error code is returned, and the process returns to step S1302. If yes, step S1304
Then, as a connection process, a client corresponding thread that exchanges information such as video data with the connected client is generated, the client is registered, and the process returns to step S1302.

In the generated client corresponding thread, if the request accepted in step 1302a is other than the connection request, step S13
At 10, the content of the request from the corresponding client is analyzed.

The request content is a request for requesting attribute information of video data (for example, RTSP-DESC).
If it is a RIBE message), step S1
In step 311, information such as the name and identifier of the video data attached to the request is extracted, the attribute information of the video data corresponding to the information is extracted from the database unit (HD device 204) in the video conversion server 400, and the appropriate information is extracted. Format (for example, SDP: Session Des)
After editing it into a description protocol, reply to the client.

Further, the request content is a request for requesting preparation for video transmission (for example, RTSP-SETU).
P. message), step S1321
First, the source video information of the video data requested by the request is retrieved from the database in the video conversion server 400. The source video information is, for example, communication attribute information such as the network address and port number of a camera server that provides a live video as shown in FIG. 14, and parameter information for camera control.

Subsequently, the flow advances to step S1322 to initialize the video acquisition unit 507 according to the source video information.
Specifically, the camera server 101 that provides the source video
Or, connect to 102 and start source image acquisition. Then, proceeding to step S1323, the video conversion parameters are retrieved from the database in the video conversion server 400. The video conversion parameters describe selection of a conversion codec, parameters for the codec, data format for codec input / output, and the like.

Succeedingly, in a step S1324, the video converting unit 506 is initialized in accordance with the video converting parameters. The video converting unit 506 includes a JPEG decoder and an MPEG-4 encoder. further,
In step S1325, the video transmission parameter information is retrieved from the database in the video conversion server 400.
The video transmission parameter information describes a transmission timeout, a transmission bit rate range, and the like.

In step S1326, the video transmission unit 505 for mobile phone terminals is set according to the video transmission parameter information, and information necessary for the client to receive the video information distributed from the video transmission unit 505 (communication port information). Etc.) to the client (mobile phone terminal) that requested the request.

Further, the content of the request is a request for requesting the start of video transmission (for example, RTSP.PLAY.
If the message is a message), the flow advances to step S1331 to generate a video conversion drive thread, and returns to the client whether the video conversion drive thread can be executed. In this video conversion drive thread, the video acquisition unit 507, the video conversion unit 506, and the video transmission unit 50 that have been prepared in advance.
5 to drive.

If the video acquisition unit 507 and the video conversion unit 5
06, if either of the video transmission units 505 is not ready, the video conversion drive thread is terminated, and the reason why the video conversion drive thread cannot be executed is returned to the client.

When the preparation is completed, the video conversion drive thread is transferred from the video acquisition unit 507 to the video conversion unit 506.
Is set so that the video data processed by the video conversion unit 506 is transferred to the video transmission unit 505. After making such settings, the camera server 1
The video conversion unit 506 receives the video from 01 or 102.
It is repeated to send the result converted by the method from the video transmitting unit 505 to the client. Note that the video conversion drive thread is a portable terminal (6) that requires a registered (connected) client to perform video conversion processing in the video conversion unit 506.
01, 602) or the PC 200, and if it is the PC 200, the video conversion unit 506 does not perform the conversion process and delivers it to the client (PC 200) in the Motion-JPEG video data format as it is. By this processing, the processing speed can be increased for the PC 200.

Further, the request content is a request for requesting temporary suspension of video transmission (for example, RTSP section PAU).
SE message), step S134
Go to 1 and suspend the video conversion drive thread.

Further, the content of the request is a request for requesting interruption of video transmission (for example, RTSP-TEARD).
OWN message), step S13
Proceeding to 51, the video conversion drive thread is terminated. Then, the processing proceeds to step S1352, and the video acquisition unit 507
Post-processing the video conversion unit 506 and the video transmission unit 505,
The process advances to step S1353, the main program is notified of the end of connection with the client, and the client corresponding thread is ended.

Next, the video acquisition unit 507, the video conversion unit 506, and the video transmission unit 505 that are functioning in the video conversion server 400 will be sequentially described.

The video acquisition unit 507 first connects to the camera server which is the target for acquiring the live video according to the source video information received at the initialization. Then, in response to the request of the video conversion drive thread, the video data is acquired from the camera server, the time stamp at the time of acquisition is added, and the video data is passed to the video conversion unit. Since the camera server in the present embodiment provides the video data in the Motion JPEG format, the time stamp is added to each JPEG data.

Next, the image conversion unit 506 will be described.
First, the parameters to the codec received at initialization,
Also, the codec input / output data format and the like are set in the MPEG-4 encoder. Then, in response to a request from the video conversion drive thread, the source video data received from the video acquisition unit 507 is adjusted to a data format and image size for codec input, and then input to an MPEG-4 encoder, and the processing result is Video transmission unit 505
Hand over to.

Then, the image conversion unit 506 converts the source image data in the JPEG format into the QCIF size and Y in advance.
Converted to UV411 format and input to MPEG-4 codec MPEG-4 data (I-frame or P-fr)
After generating (name), it is passed to the video transmission unit 505.
At this time, the time stamp added by the video acquisition unit 507 is also input to the MPEG-4 codec.

The video conversion unit 5 in this embodiment
In 06, the time stamp given to the source video of each frame is observed, and if the reception interval of the source video is longer than a predetermined time, the motion compensation mechanism of MPEG-4 is suppressed (for example, motion vector search is performed). Prohibit and diff the frame itself without breaking it down into objects)
To the MPEG-4 encoder. This is because when the reception interval is longer than a predetermined time (for example, 1 second or longer), the correlation with the previous frame becomes low. The predetermined time is set as the setting information of the video conversion server 400.

Further, in the video converting section 400, the MP
When the output of the EG-4 encoder is monitored and the output data is excessively generated according to the specifications of the mobile phone terminal, the input source video is thinned out in frame units to determine the information amount of the entire video stream. Processing for instructing the video acquisition unit 507 to reduce the number may be performed.

Next, the video transmission unit 505 will be described. First, the communication channel is prepared according to the transmission parameter information received at the initialization, and the information necessary for the client to connect is provided. Then, in response to the request of the video conversion drive thread, the video data generated by the video conversion unit 506 is transmitted to the client (mobile phone terminal). In the present embodiment, it is assumed that the client is a mobile phone terminal that does not have high processing capability, and the video data is MPEG-4 data whose data rate varies depending on the video content and I / P-frame. There is. Therefore, the video transmission unit monitors the transmission state of the transmission to the client and notifies the video conversion unit of the transmission state so that the transmission is performed at an appropriate frame rate.

With the above configuration, the portable telephone terminal (601, 6
02), the server (10
1, 102) to install the video conversion server for converting the Motion-JPEG video data output to the MPEG mobile phone video data, and display live video from the camera on the mobile phone terminal as well. Can be made. The function of the video conversion unit of the video conversion server absorbs fluctuations in the source video content generated by the camera server placed on the network and the communication status of the network, and stabilizes the appropriate video stream on the mobile phone terminal. Can be supplied.

In the present embodiment, an example in which a video conversion server is installed on the network independently of the gateway connecting the mobile phone network and the network has been described. However, the video conversion server is a gateway. It may be implemented as a part. In addition, the video conversion server and the gateway are connected to each other through a VPN (Virtual PrivateN).
It is possible to easily imagine a connection form in which a dedicated line is connected, including an "etwork".

Further, in the present embodiment, the case where the same video data is distributed to all mobile phone users has been described. However, the contract status of the users, the presence or absence of billing, the performance of the mobile phone terminals, etc. Accordingly, it may be designed to use video conversion parameters having different resolutions, for example.

(Modification of the First Embodiment) A part of the operation of the video conversion server 400 in the first embodiment may be changed as follows. That is, the video conversion server 400 actively requests the camera servers 101 and 102.

In the modification of this embodiment, the video acquisition unit 507 of the video conversion server 400 operates as follows.

The video acquisition unit 507 first connects to the camera server 101 or 102 that provides a live video according to the source video information received at the time of initialization. And
In response to a request from the video conversion drive thread, the video data is acquired from the camera server, a time stamp for the video acquisition is added, and the video data is output to the video conversion unit 506. At this time, the video acquisition unit 507 observes the image size and the data size of the video data acquired from the camera server of the connection destination, the added time stamp, and the like, and, for example, is less than or equal to the frame rate initially set in the video relay server. If only the source video data of 1 is received, the camera server of the connection destination is requested to deliver exclusive video data by a privileged connection (a connection in which video delivery to other terminals such as the PC 200 is excluded).

As described above, the live image sent from the camera server arranged on the Internet can be relayed with priority to the mobile phone terminal. The video conversion server 400 is characterized in that a better quality source video is acquired by actively controlling the camera server of the connection destination.

The timing of requesting the privileged connection may be temporarily changed from a blur of the image when a request for pan / tilt control of the camera to the camera server is made by another client. Since the frame rate may decrease, the privileged connection request may be controlled before and after the camera control.

Further, in the above description, exclusive video delivery by privileged connection is described, but video delivery may be provided to clients other than the video conversion server 400. As a result, the video conversion server 400
Although it will not be possible to use all the resources of the camera server, it will be possible to continue the video distribution service to other clients, although limited.

In addition, in the above, in addition to obtaining a high quality source video by exclusive video data delivery by privileged connection of the camera server, for example, when it is desired to prioritize the motion of the video over the image quality, the camera server supplies the video. The camera server may be requested to reduce the screen size of the image to be displayed by one level (from 320 × 240 of QVGA to 160 × 120 of subQVGA). If it is desired to prioritize image quality over video movement, the camera server may be requested to increase the image quality value (Q value) when the JPEG data is created by the camera server.

In this embodiment, the preset information of the camera server is not used, but if the camera server is inquired about the service status to other clients and the video conversion server of this embodiment is the only client, It is also possible to sequentially or selectively tour the home position and presets of the camera server.

As described above, the present invention supplies the program code of the software that realizes the functions of the above-described embodiments to a video conversion device via a network such as the Internet, and the computer (or CPU or CPU of the video conversion device). This can be achieved by the MPU) reading and executing the program code stored in the storage medium.

In this case, the program code itself read from the storage medium realizes the function of the CPU 50 of the above-described embodiment, and the storage medium storing the program code constitutes the present invention.

As a storage medium for supplying the program code, for example, a floppy (registered trademark) disk, a hard disk, an optical disk, a magneto-optical disk, C
A D-ROM, a CD-R, a magnetic tape, a non-volatile memory card, a ROM, etc. can be used.

Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also the OS (operating system) running on the computer based on the instruction of the program code. ) And the like perform some or all of the actual processing, and the processing realizes the functions of the above-described embodiments.

Furthermore, after the program code read from the storage medium is written in the memory provided in the function expansion board inserted into the computer or the function expansion unit connected to the computer, based on the instruction of the program code, A CPU or the like included in the function expansion board or the function expansion unit performs a part or all of the actual processing, and the processing realizes the functions of the above-described embodiments.

When the present invention is applied to the above-mentioned storage medium, the storage medium stores the program code corresponding to the above-mentioned flow chart, but in brief description, it is indispensable for the image pickup apparatus of the present invention. Various modules
It will be stored in a storage medium.

[0099]

As described above, according to the present invention, the video displayed on the client on the portable information terminal can be provided to the user in an appropriate state.

[Brief description of drawings]

FIG. 1 is a diagram showing an example of a video distribution system.

FIG. 2 is a diagram showing an example of a hardware configuration for operating a video conversion server 400.

FIG. 3 is a diagram showing an example of a hardware configuration of camera servers 101 and 102.

FIG. 4 is a diagram showing an example of a hardware configuration of mobile phone terminals 601 and 602.

FIG. 5 is a diagram schematically showing an outline of a software-like process flow of the image distribution system.

FIG. 6 is an operation process flowchart of the mobile phone terminals 601 and 602.

7 is an operation processing flowchart of a camera control server 503 in the camera servers 101 and 102. FIG.

FIG. 8 is an operation processing flowchart of the video server 508 in the camera servers 101 and 102.

FIG. 9 is a diagram exemplifying allocation of operation units 407 of mobile phone terminals 601 and 602.

FIG. 10 is a diagram showing an example of a display screen of a camera server setting program.

FIG. 11 is a flowchart showing the operation of a camera server setting program.

FIG. 12 is a diagram schematically showing conversion of video data in the video conversion server 400.

FIG. 13 is a flowchart showing an operation process of the video conversion server 400.

FIG. 14 is a diagram showing an example of source video information.

[Explanation of symbols]

101,102 Camera server 200 personal computer 201 CPU 202 RAM 204 HD device 400 Video conversion server 500 gateway 601,602 Mobile phone terminals

   ─────────────────────────────────────────────────── ─── Continued front page    F-term (reference) 5C064 BA01 BA07 BB03 BB05 BC11                       BC23 BD02 BD08                 5K024 AA74 BB04 CC07 CC11 DD01                       DD04 FF04 GG01 GG03                 5K101 KK04 KK15 LL12 MM07 NN06                       NN18 NN21 PP03 RR05 SS07

Claims (22)

[Claims] 1. A video conversion device for delivering video data imaged by an imaging device in real time, wherein video data of a first video data system is received from a server connected to the imaging device via a first network. And a second means for transmitting the video data of the first video data system received by the receiving means to a second network.
And a control unit for distributing the converted video data to the terminal on the second network. 2. The video conversion processing device according to claim 1, wherein the second network is limited in data communication speed as compared with the first network. 3. The image data format according to claim 2, wherein the first video data format is a video data format compressed using intraframe compression, and the second video data format uses intraframe compression and interframe compression. A video conversion device characterized by the conventional video data system. 4. The compression processing according to claim 3, wherein the second video data format decomposes an image in one frame into a plurality of objects, and performs a decomposition process to obtain a difference between frames in each object. An image conversion device characterized by performing. 5. The control device according to claim 4, wherein the control means is 1
A video conversion device, wherein the disassembly processing is prohibited when a frame interval has passed a predetermined time or more. 6. The control unit according to claim 1, wherein the control unit determines whether the terminal to which the video data is to be delivered is on the first or second network, and is connected to the first network. Video data of the first video data system to terminals that are connected, and video data of the second video data system to terminals that are connected to the second network. Characteristic video conversion device. 7. The control means according to claim 1, wherein the control means is R.
A video conversion device, which distributes video data to a terminal on the second network by TSP or RTP. 8. The control unit according to claim 1, wherein the control unit requests the server to preferentially distribute the image when the video data received by the receiving unit is equal to or lower than a predetermined frame rate. Video conversion device characterized by. 9. The video conversion device according to claim 1, wherein the first video data format is a Motion-JPEG format, and the second video data format is an MPEG-4 format. 10. The video conversion device according to claim 1, wherein the video conversion device is a gateway that mediates between the first network and the second network. 11. A video conversion method for delivering video data imaged by an imaging device in real time, comprising receiving video data of a first video data system from a server connected to the imaging device via a first network. However, the received video data of the first video data system is converted into the second video data system corresponding to the second network, and the converted video data is distributed to the terminal on the second network. Video conversion method. 12. The video conversion processing device according to claim 11, wherein the second network is limited in data communication speed as compared with the first network. 13. The video data format according to claim 12, wherein the first video data format is a video data format compressed using intraframe compression, and the second video data format uses intraframe compression and interframe compression. The video conversion method is characterized by the conventional video data system. 14. The compression processing according to claim 13, wherein the second video data format decomposes an image in one frame into a plurality of objects, and performs decomposition processing for obtaining a difference between frames in each object. A video conversion method characterized by performing. 15. The video conversion method according to claim 14, wherein the disassembling process is prohibited when one frame interval has passed a predetermined time or more. 16. The terminal according to claim 11, wherein it is determined whether the terminal to which the video data is delivered is on the first or second network, and the terminal connected to the first network is determined. Distributes video data of the first video data system, and distributes video data of the second video data system to terminals connected on the second network. Method. 17. The video conversion method according to claim 11, wherein video data is distributed to a terminal on the second network by RTSP or RTP. 18. The video conversion method according to claim 11, wherein when the received video data has a predetermined frame rate or less, the server is requested to preferentially distribute the image. 19. The video conversion method according to claim 11, wherein the first video data format is a Motion-JPEG format, and the second video data format is an MPEG-4 format. 20. The video conversion method according to claim 11, wherein the conversion processing is executed in a gateway which mediates between the first network and the second network. 21. An operation processing program for executing the processing of the flat storage conversion method according to claim 11. 22. A storage medium storing the operation processing program according to claim 21.