CN100534174C - Information distribution device, information distribution system, and information distribution system method - Google Patents
Information distribution device, information distribution system, and information distribution system method Download PDFInfo
- Publication number
- CN100534174C CN100534174C CNB2006100074797A CN200610007479A CN100534174C CN 100534174 C CN100534174 C CN 100534174C CN B2006100074797 A CNB2006100074797 A CN B2006100074797A CN 200610007479 A CN200610007479 A CN 200610007479A CN 100534174 C CN100534174 C CN 100534174C
- Authority
- CN
- China
- Prior art keywords
- image
- mentioned
- data
- voice
- speech
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Landscapes
- Telephonic Communication Services (AREA)
- Television Systems (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention relates to information distributor, information distributing system, and relative information distributing method, wherein said invention can avoid camera server and portable terminal, to avoid increasing the cost. And the information distributor is characterized in that: it comprises an image data receiving unit for receiving image data from several image emitters, a voice data receiving unit for receiving voice data from several voice emitters, a coding unit for selectively combing received image data and received voice data, to be coded as voice image data, and a distributing unit for distributing voice image data generated by coding unit to the receiving device.
Description
The present invention be that March 28, application number in 2003 are 03121487.8 the applying date, denomination of invention is the division of the application for a patent for invention of " view data distribution method, view data dispensing device and system ".
Technical field
The present invention relates to the view data that provides from the camera that is connected to network is applied the technology that changes and be distributed to image display device.
Background technology
Drive the video camera of the function that discloses a kind of image that the video camera that can watch a long way off being disposed via networks such as the Internets is provided in the flat 10-040185 communique the Japanese Patent Application Publication spy.The video camera that will have this network function below is called the camera server device.In this existing example, in a plurality of terminal installations such as personal computer, not only can watch image simultaneously from the camera server device, also can use a plurality of terminal installations from the panning of Long-distance Control video camera, angle of inclination and convergent-divergent multiplying power.
In this camera server apparatus system of also permitting video camera control, on a plurality of terminal installations, during 1 video camera of permission control, physically need to mediate the authority of only controlling 1 video camera.About this point,, just can only be possessed of control power and carry out video camera control between time limit the user by importing the notion of opening disclosed so-called control authority in the flat 10-042278 communique the Japanese Patent Application Publication spy.On the other hand, the technology of overlay information is opened in the flat 11-196404 communique the Japanese Patent Application Publication spy and is disclosed on from the image of this camera server device.
In recent years, along with portable phone, portable terminal device development of technology, also camera review is watched or operated to slave unit like this.But, when the image from this camera server device not only is distributed to terminals such as personal computer, when also being distributed to terminal installations such as portable phone etc., because image transmission method and picture format etc. are different in terminals such as personal computer and the portable terminal device, so the camera server device need be held both sides' interface, just the problem that the camera server installation cost increases appears.This at the special purpose interface that the camera server device side need have the video camera control usefulness of portable terminal device in addition, just makes the complicated and cost of camera server device increase for video camera control too.
On the other hand, for image from the camera server device of the function that does not have on image overlapping advertisement etc., later on just can not overlapping neatly advertisement.In addition, when thinking that overlapping information is very big, keeping overlay information on the camera server device is to distribute the different function of this purpose with image in the past, sees unrealistic from the cost aspect.And then, can not accomplish for example overlapping advertising message in portable phone in the prior art, and in such in the past terminal not overlapping advertising message.
Via network control at a distance video camera and obtain in the technology of display image, have the panning of video camera, inclination, convergent-divergent, backlight correction etc. and the relevant high feature of the degree of freedom of video camera control.General use with image and voice as one group, receive and dispatch via network in the video conference system of image in a plurality of places and voice.The technology of reproducing on one side via network one side download images, voice is called fluidization technique, utilizes the live distribution technology of coding, net distribution, reception and the reproduction of carrying out image and voice simultaneously.
About the correspondence of image and voice, open in the flat 11-305318 communique the Japanese Patent Application Publication spy, have the be mapped camera of output image and voice of camera parameters and voice.Open the device that discloses selection and output image and voice in the flat 08-56326 communique the Japanese Patent Application Publication spy.Japanese Patent Application Publication spy opens flat 10-93941 communique and discloses the example that connects a plurality of places, switches the video conference system that uses image and voice.
In via the so-called web camera of network control video camera at a distance, generally only can obtain image and do not have voice.On the other hand, except that video camera control, can also receive and dispatch image and voice in the video conference system, but the purposes epigraph adopts the mode that is input to identical bidirectional communication apparatus in same place with voice.It generally is the linking objective of image and voice is specially carried out appointment by the terminal use using method.
The image fluidization technique is that a band phonetic image is distributed to a plurality of receiving systems, and making up arbitrarily, image and voice do not carry out usually.In the selection of disclosed image and voice in the past, composite set, can not on network, make up arbitrarily.
Image distribution system by continuous distribution of images of data transfer medium such as the Internet, in-house networks has been popularized in the whole society, is used for various fields such as the transmission of live image, the supervision of indoor and outdoor, vegeto-animal observation.
In these image distribution systems, be used for the image Distributor of distribution of images, this image Distributor is most to adopt JPEG coded system (by the image encoding mode of the world standard of ISO/IEC 10918 defineds) as the image encoding mode.
On the other hand, the coded image data of following the JPEG coded system (jpeg encoded data) that sends from the image Distributor is received by client terminal, and the decoding back shows on picture.Because current PC (personal computer), PDA (personal digital assistant) majority of just popularizing has the decoding function as the jpeg encoded data of standard feature, so PC and PDA are utilized as client terminal.
But portable phone is popularized rapidly in recent years, and as the portable terminal device that in Japan uses, the popularity rate of portable phone is more taller than notebook type PC and PDA.And then, the function of portable phone improves rapidly, is equipped with recently the decoding function of the coded data (MPEG4 coded data) of following MPEG4 coded system (by the phonetic image coded system of the world standard of ISO/IEC14496 defined) in the portable phone of the corresponding third generation communication mode of practicability as standard feature in Japan.But, because the decoding function of not carrying jpeg encoded data usually in the portable phone, so can not directly send jpeg encoded data to portable phone from above-mentioned image Distributor.
At this problem, consider 2 solutions.First solution is to improve the image Distributor, can send the MPEG4 method of coding data.But, in this method, need existing image Distributor is replaced into new image Distributor, replace the cost of usefulness pro rata and will spend huge with the platform number that is provided with of image Distributor.
Second solution is in the way of the communication path of image Distributor and portable phone Relay Server to be set, and undertaken from the method for jpeg encoded data to the coded data converting of MPEG4 coded data by this Relay Server.The advantage of this method is by many image Distributors are connected to 1 Relay Server, and that can reduce Relay Server greatly is provided with the platform number, and the cost that usefulness is set thus also can be suppressed greatly.
But the method that Relay Server is set also has shortcoming.This is because the image size of handling usually with respect to image Distributor in the past is QVGA (Quarter VGA) size (horizontal 320 pixels, vertical 240 pixels) or 1/16VGA size (horizontal 160 pixels, vertical 120 pixels), the common decodable image size of portable phone is QCIF (QuarterCIF) size (horizontal 176 pixels, vertical 144 pixels), so the jpeg encoded data of QVGA size or 1/16VGA size must be transformed to the MPEG4 coded data of QCIF size, with regard to worrying thus coded data converting and cause deterioration of image quality.
For example, the resolution conversion mode of jpeg encoded data in the past as the Japanese Patent Application Publication spy open flat 4-229382 number disclosed, constitute in the orthogonal transform data among 1 by in the jpeg image decoding processing, obtaining, the coefficient composition that only takes out low order carries out the anti-quadrature conversion, with the image size reduction be horizontal m/8 doubly, n/8 doubly (m, n are smaller or equal to 7 integer more than or equal to 1) longitudinally.But, because from the QVGA size to the conversion of QCIF size be horizontal 0.55 times (4.4/8), vertical 0.6 (4.8/8) doubly, from the 1/16VGA size to the conversion of QCIF size be horizontal 1.1 times (8.8/8), vertical 1.2 (9.6/8) doubly, so m, n do not become integer, can not carry out from the conversion of QVGA size or 1/16VGA size to the QCIF size.
And then, as general image resolution ratio transform method in the past, have image by dredging pixel (dwindling) between certain ratio or inserting the method for same pixel (expansion) repeatedly, the weighted average of a plurality of pixels adjacent with calculating generates the method for new pixel value etc., and arbitrary method all can be come the changing image size by any multiplying power.But, produce use in these prior arts and from Figure 44 to Figure 47, be described below such problem points.
Figure 44 is before the conversion of expression when according to prior art the image transform of QVGA size being the QCIF size and the figure of the corresponding relation of the image range after the conversion.As shown in the drawing, the image range of horizontal 320 pixels, vertical 240 pixels is reduced into the image range of horizontal 176 pixels, vertical 144 pixels.As mentioned above, this is horizontal 0.55 times (4.4/8), vertical 0.6 (4.8/8) conversion multiplying power doubly.
Figure 45 is the figure that block boundary line that the image size conversion of explanation by Figure 44 causes moves.In same figure, solid line is represented the position of the boundary line of horizontal 8 pixel separation, vertical 8 pixel separation, and dotted line is represented the position of the boundary line of horizontal 4.4 (=8 * 0.55) pixel separation, vertical 4.8 (=8 * 0.6) pixel separation.Promptly by the image size conversion of Figure 44, the block boundary line position that is present in the preceding image of conversion moves to dotted line position from solid line position.Image after the conversion is cut apart along the block boundary line of solid line position once more and is carried out the MPEG4 image encoding then, and therefore there are the block boundary line in dotted line position and solid line position both sides in the resulting image of MPEG4 picture decoding.
But the block boundary line of dotted line position is generated by the coding of the JPEG in the image Distributor, and therefore the compression ratio along with the JPEG coding improves, and occurs the piece distortion in dotted line position.In addition, the block boundary line of solid line position is that the MPEG4 image encoding by Relay Server is generated, and therefore the compression ratio along with the MPEG4 image encoding improves, and also occurs the piece distortion in solid line position.
Current, the message capacity between image Distributor and the portable phone is that per second is tens of to about hundreds of kilobits, and in order to transmit the level and smooth image of action, message capacity is insufficient, therefore sets the compression ratio of image higher usually.For this reason, the piece distortion can obviously occur among above-mentioned dotted line position shown in Figure 45 and the solid line position both sides, the image quality that the portable phone user sees just reduces greatly.
Figure 46 is before the conversion of expression when by prior art the image transform of 1/16VGA size being the QCIF size and the figure of the corresponding relation of the image range after the conversion.As shown in the drawing, the image range of horizontal 160 pixels, vertical 120 pixels is enlarged into the image range of horizontal 176 pixels, vertical 144 pixels.As mentioned above, this is horizontal 1.1 times (8.8/8), vertical 1.2 (9.6/8) conversion multiplying power doubly.
Figure 47 is the figure that the block boundary line that caused by the image size conversion of Figure 46 of explanation moves.In same figure, solid line is represented the position of the boundary line of horizontal 8 pixel separation, vertical 8 pixel separation, and dotted line is represented the position of the boundary line of horizontal 8.8 (=8 * 1.1) pixel separation, vertical 9.6 (=8 * 1.2) pixel separation.Promptly by the image size conversion of Figure 46, the block boundary line position that is present in the preceding image of conversion moves to dotted line position from solid line position.Image after the conversion is cut apart along the block boundary line of solid line position once more and is carried out the MPEG4 image encoding then, and therefore there are the block boundary line in dotted line position and solid line position both sides in the resulting image of MPEG4 picture decoding.
Just, under the situation of the image of 1/16VGA size, also the piece distortion can occur among dotted line position and the solid line position both sides, the image quality that the portable phone user sees just reduces greatly.
Summary of the invention
The present invention finishes just in view of the above problems, and purpose is the interface that does not need to be provided with in addition the usefulness of communicating by letter such as camera server and portable terminal device, just can avoid the cost of camera server device to increase.
In addition, second purpose is the special purpose interface that does not need to be provided with in addition camera server control usefulness, just can avoid the cost of camera server device to increase.
And then the 3rd purpose is to make the camera server device not have the redundancy function of information overlap processing etc., just can avoid the cost of camera server device to increase.
In order to achieve the above object, the invention provides a kind of information distribution device, this information distribution device can be connected with a plurality of speech detection devices with a plurality of image-pickup devices, it is characterized in that, comprise: the view data receiving element receives described view data from the picture transmitter device that can send view data; The speech data receiving element receives speech data from the voice dispensing device; The information holding unit keeps the corresponding informance between view data and the speech data; Coding unit, based on the corresponding informance that remains in the above-mentioned information holding unit, above-mentioned view data that will be received by above-mentioned view data receiving element and the above-mentioned speech data that is received by above-mentioned speech data receiving element make up, and encode as band phonetic image data; And Dispatching Unit, will be distributed to receiving system by the above-mentioned band phonetic image data that above-mentioned coding unit generated; Wherein, under a plurality of speech datas and the corresponding situation of above-mentioned view data, above-mentioned coding unit makes up above-mentioned view data and above-mentioned a plurality of speech data successively, and under the situation that the arbitrary speech data in above-mentioned a plurality of speech datas can not be received by above-mentioned speech data receiving element, above-mentioned coding unit makes up the speech data except that the above-mentioned speech data that can not receive in above-mentioned view data and the described a plurality of speech data successively.
In order to achieve the above object, the present invention also provides a kind of information dispensing method, this information dispensing method be used for can with a plurality of image-pickup devices and the information distribution device that a plurality of speech detection devices are connected, it is characterized in that: receive described view data from image-pickup device that can the pickup image data; Based on remaining on view data in the information holding unit and the corresponding informance between the speech data, receive speech data from the voice dispensing device; Received view data and received speech data are made up, and encode as band phonetic image data; Above-mentioned band phonetic image data behind the coding are distributed to receiving system, wherein, under a plurality of speech datas and the corresponding situation of above-mentioned view data, above-mentioned coding step makes up above-mentioned view data and above-mentioned a plurality of speech data successively, and under the situation that the arbitrary speech data in above-mentioned a plurality of speech datas can not be received by above-mentioned receiving step, above-mentioned coding step makes up the speech data except that the above-mentioned speech data that can not receive in above-mentioned view data and the above-mentioned a plurality of speech data successively.
Other features of the present invention and advantage are by being that the following explanation of reference will be come to understand with the accompanying drawing.In addition, in the accompanying drawings, additional identical with reference to label to same or analogous structure.
Description of drawings
Accompanying drawing is included in the specification, constitutes its part, represents form of implementation of the present invention, and is used from explanation principle of the present invention with the record one of specification.
Fig. 1 is the synoptic diagram of physical structure of the information distribution system of the expression first embodiment of the present invention;
Fig. 2 is the block diagram of structure of the camera server device of the expression first embodiment of the present invention;
Fig. 3 is the figure of an example of the user interface images of expression display operation terminal;
Fig. 4 is the figure of an example of the outward appearance of the portable display terminal of expression;
Fig. 5 is the figure of logical construction of the information distribution system of the expression data flow that is conceived to the first embodiment of the present invention;
Fig. 6 is the flow chart of action of image transforming unit of the conversion server of the expression first embodiment of the present invention;
Fig. 7 is the figure of database structure of the Advertisement Server of the expression first embodiment of the present invention:
Fig. 8 is expression obtains the flow process of sending the video camera control command from the video camera control authority of the first embodiment of the present invention a flow chart;
Fig. 9 is the figure of the data format of expression video camera control authority request of the first embodiment of the present invention and video camera control command;
Figure 10 is that expression is replied the figure of the flow process of voice from the control command voice that send the first embodiment of the present invention to reception;
Figure 11 A and 11B be the key button of the first embodiment of the present invention and video camera control command, from the correspondence table of reproducing with portable display terminal of replying voice of replying of camera server device;
Figure 12 is expression and the flow chart of the flow process of the corresponding responder action of control command of the camera server device of the first embodiment of the present invention;
Figure 13 is the figure of data format of the video camera control command of the expression first embodiment of the present invention;
Figure 14 is the figure that is illustrated in the form of the speech data that exchanges between the distribution services apparatus of the first embodiment of the present invention and the conversion server unit;
Figure 15 is the figure that is illustrated in the form of the speech data that exchanges between the distribution services apparatus of the first embodiment of the present invention and the conversion server unit;
The flow chart of the overview of the flow process when Figure 16 is the control authority request of control authority Managing speech converter unit of conversion server unit of the expression first embodiment of the present invention;
Figure 17 is the flow chart of flow process of the camera switching control of the expression second embodiment of the present invention;
Figure 18 is video camera sequence number, video camera name, the video camera address correspondence table of the second embodiment of the present invention;
Figure 19 is the formation of the advertisement information list of the third embodiment of the present invention;
Figure 20 is the block diagram of structure of the information distribution system of the expression fourth embodiment of the present invention;
Figure 21 is the flow chart of action of the conversion server unit of the expression fifth embodiment of the present invention;
Figure 22 is the block diagram of brief configuration of the information distribution system of the expression sixth embodiment of the present invention;
Figure 23 is the block diagram of the hardware configuration of expression image server of the sixth embodiment of the present invention and voice server;
Figure 24 is the block diagram of software configuration of the information distribution system of the expression sixth embodiment of the present invention;
Figure 25 is the figure of course of action of software module of the information distribution system of the expression sixth embodiment of the present invention;
Figure 26 A～26C is repeater management server image information, voice messaging and the image voice of the expression sixth embodiment of the present invention figure to the list structure of application;
Figure 27 is the flow chart of processing sequence of request processing procedure of the Relay Server of the expression sixth embodiment of the present invention;
Figure 28 is the flow chart of processing sequence of image receiving course of the Relay Server of the expression sixth embodiment of the present invention;
Figure 29 is the flow chart of processing sequence of voice receiving course of the Relay Server of the expression sixth embodiment of the present invention;
Figure 30 is the flow chart of processing sequence of image phonetic synthesis process of transmitting of the Relay Server of the expression sixth embodiment of the present invention;
Figure 31 A and 31B are the expression repeater management server conditional information of the seventh embodiment of the present invention and image voice to the figure of the list structure of using;
Figure 32 is the flow chart of processing sequence of request processing procedure of the Relay Server of the expression seventh embodiment of the present invention;
Figure 33 is the block diagram of brief configuration of the information distribution system of the expression eighth embodiment of the present invention;
Figure 34 is the pie graph of the information distribution system of the expression tenth embodiment of the present invention;
Figure 35 is the flow chart of the conversion process of the expression tenth embodiment of the present invention;
Figure 36 is the figure that is used to illustrate the image size conversion of the tenth embodiment of the present invention;
Figure 37 is the flow chart of the conversion process of the expression 11st embodiment of the present invention;
Figure 38 is the figure that is used to illustrate the image size conversion of the 11st embodiment of the present invention;
Figure 39 is the flow chart of the conversion process of the expression 12nd embodiment of the present invention;
Figure 40 is the flow chart of the conversion process of the expression 13rd embodiment of the present invention;
Figure 41 is the flow chart of the conversion process of the expression 14th embodiment of the present invention;
Figure 42 is expression the figure of the image size of the 14th embodiment of the present invention in the situation of the increase that laterally narrows down to the piece distortion that produced longitudinally on the either direction at 1/2 o'clock;
Figure 43 is the flow chart of the conversion process of the expression 15th embodiment of the present invention;
Figure 44 is the figure that is used to illustrate image size conversion in the past;
Figure 45 is the figure that is used to illustrate the piece distortion when changing image was big or small in the past;
Figure 46 is the figure that is used to illustrate image size conversion in the past;
Figure 47 is the figure that is used to illustrate the piece distortion when changing image was big or small in the past.
Embodiment
Describe the preferred embodiments of the present invention in detail below with reference to accompanying drawing.
＜the first embodiment 〉
This first embodiment is the image that is taken into remote controlled camera server device, information with the overlapping advertisement etc. on image of the conversion server unit in the distribution way, image is distributed to portable display terminal, and then can carries out video camera control from portable display terminal.
Fig. 1 is the sketch of structure of the physical image dissemination system of this first embodiment.As shown in Figure 1, with the camera server apparatus system of forming by camera server device 111, display operation terminal 112 and first network 113, also have conversion server unit 114, advertising service apparatus 115, second network 116, distribution services apparatus 117, the 3rd network 118, portable display terminal 119 to constitute image distribution system.
In the camera server apparatus system, specify the address of camera server devices 111 and connect from display operation terminal 112 via first network 113, obtain the realtime graphic that camera server device 111 is photographed, obtain the video camera control authority simultaneously as required and carry out video camera control.If display operation terminal 112, camera server device 111 can be discerned on network mutually, then can exist a plurality of.
Fig. 2 is the block diagram of the structure of expression camera server device 111, be taken into compression unit 122 with camera unit 121 with image, be that the image that video camera is photographed is taken into as numerical data, generate the compressed image of Motion JPEG form, image is distributed to the display operation terminal of connection request by image communication unit 125.If connection request is from a plurality of display operation terminals, then simultaneously to a plurality of display operation terminal distribution of images.The display operation terminal that obtains the authority (control authority) of control video camera is sent the video camera control command to camera control unit 123, pans, the video camera control of inclination, convergent-divergent etc.Video camera is controlled communication unit 126 and is controlled the generation of such video camera control command, explains, replys.The display operation terminal that control authority administrative unit 124 management obtains current control authority can be controlled the information relevant with control authority of tabulation, priority etc. of display operation terminal of remaining time, the control right request limit of camera unit 121.Communication control unit 127 controls and image communication unit 125, video camera control communication unit 126 and outside communicating by letter.
In the distribution services apparatus 117, image is distributed to a plurality of portable display terminal 119 in the connection, in each portable display terminal 119, the MPEG4 image that receives is carried out decoding and displaying.Suppose portable display terminal 119, for example for example can receive the digital picture about 64kbps at a high speed and show with portable phone or portable information terminal (PDA).Fig. 4 represents the example of portable display terminal.The 141st, image information display unit, the 142nd, key pushbutton unit.
Among this first embodiment, conversion server unit 114 carries out from the conversion of Motion jpeg image to the MPEG4 image.Therefore, as employed picture format in the system, suppose have from camera server device 111 to conversion server unit 114 or the system of display operation terminal 112 for Motion JPEG, the system from conversion server unit 114 to portable display terminal 119 is the simple profile of the viewable portion of MPEG4.
But, among this first embodiment, be not limited to the compressed format of image, the compressed image format conversion that receives from the camera server device system for the condition of the compressed image form that can show at portable display terminal 119 can be got final product so long as satisfy in conversion server unit 114.Not necessarily will carry out format conversion, needing only image can correctly distribute and show, can get final product by overlay information.If from handling and the consideration of network burden aspect, it is also passable not carry out image compression, also can be non-compressed image.
Control about video camera, obtain control authority from portable display terminal 119 to 111 requests of camera server device, send control command, control command is sent to camera server device 111 via distribution services apparatus 117, conversion server unit 114.From portable display terminal 119 to distribution services apparatus 117, the system of conversion server unit 114, in this first embodiment, two-way sound channel distribution control signal and its situation of replying of using conversation usefulness are described.Describe in detail in the back about this point.
If from camera server device 111, conversion server unit 114 just can regard that to remove a part outer and the display operation terminal is equal as.Except that portable display terminal 119, all distributed IP address (below be called the address) in each device described in this first embodiment, as mutually can well-determined identification address on network.But the identification of the portable display terminal 119 on the network can be used the distinctive scheme of portable phone, even discern with telephone number.But, also can communicate by letter as long as can discern mutual device, terminal, any identifying schemes all can use.
The 3rd network 118 is assumed in first embodiment and is used for having enough frequency bands from distribution services apparatus 117 to portable display terminal 119 transitive graph pictures and video camera control command, in portable display terminal 119 1 sides are wireless portable phone nets, as long as can guarantee enough frequency bands that the communication between distribution services apparatus 117 and the portable display terminal 119 needs in theory, and be not limited to its physical structure.Are images through the image of the 3rd network 118 among this first embodiment with the MPEG4 image packetsization, the video camera control command and reply be as described later like that in the second and the 3rd network 116,118 voice as the two-way sound channel of conversation usefulness transmit.Have transmitting the network connection of the enough frequency bands of advertising message as long as use between conversion server unit 114 and the advertising service apparatus 115.
Fig. 5 represents to be conceived to the theoretical construct of data flow.Among Fig. 5, the formation identical with Fig. 1 added same reference numerals.Operation display terminal 112 is client computer for camera server device 111.Camera server device 111 is made of camera operation unit 161, display unit 162.In the operation screen of Fig. 3, camera operation unit 161 correspondences 133～135, display unit 162 correspondences 131.Control communication unit 127, image communication unit 125 swap datas with the video camera of each camera server device 111 respectively,, and use camera operation unit 161 to carry out video camera control at display unit 162 display images.In addition, such as mentioned above, a plurality of operation display terminals 112 can be connected to 1 camera server device 111 simultaneously.
The flow process that at first is conceived to image a little at length describes the action of Fig. 5.111 pairs of All Clients that are connected to camera server device 111 of camera server device are promptly operated the image that is compressed by Motion JPEG that 114 fens hair camera units 121 of display terminal 112 and conversion server unit are taken into.Record and narrate like that although operation display terminal 112 and conversion server unit 114 are 1 in Fig. 5, can certainly be respectively a plurality of.
The flow process of the image transforming unit 164 of expression conversion server unit 114 among Fig. 6.At first among the step S111, obtain image from camera server 111, (step S112) decompresses on the Motion jpeg compressed image one frame one frame ground that receives with 164 pairs of image transforming unit at once, when the needs advertisement is overlapping (step S113 is for being), carry out to the overlapping processing of image ad (step S114), MPEG4 compressed image (step S115) sends to distribution services apparatus 117 (step S116) once more.At this moment, in the overlapping processing of advertisement in step S114, be delivered in the PTZ value (angle of panning, angle of inclination, convergent-divergent multiplying power) that keeps in the control authority Managing speech converter unit 163 of conversion server unit 114 to advertising service apparatus 115, obtain the necessary advertising message corresponding from advertising database 170, use this advertising message to overlap on the image with this PTZ value.In the distribution services apparatus 117, the MPEG4 image stream that is received is distributed to simultaneously many portable display terminals (being 119 among Fig. 5) that are connected to distribution services apparatus 117.Because the advertising message that obtains of Fig. 7 of explanation has sleazy situation according to the PTZ value of time period and video camera from behind, so in the case, with regard to not overlapping advertisement.Just, advertisement is overlapping for denying among the step S113.
Advertising message is the combination of promotional literature and lap position.Have database in the advertising service apparatus 115, when the PTZ value that present moment and video camera are provided is inquired, just obtain promotional literature and lap position.In the database,, obtain the promotional literature and the lap position of the initial project of finding of the suitable corresponding moment, PTZ scope from the little such sheet form of side retrieval Fig. 7 of project sequence number.* mean no scope appointment (always being fit to).As resulting promotional literature, opaque telop (telop) character string, rest image, live image montage etc. are arranged.
About rest image, live image montage, have the α plane information, if necessary, can carry out overlapping so that as seen image section as a setting transparent.Under the situation of live image, in each frame, make synchronously carry out on the time overlapping.In addition, although in this first embodiment, be advertising message, being not limited to advertising message, can be the information of thinking to append to halfway image.For example, also can be overlapping the state of a control information of the camera server device that control authority is waited for number, control authority stand-by period, the inclination scale value of panning etc. obtains from linking objective camera server device.In the overlapping processing of the advertisement of step S114, serve as that basis decision lap position (central authorities etc.) up and down and advertising message show big or small (large, medium and small) with lap position information.
Then be conceived to the action that control flow describes Fig. 5 in detail.Expression obtains from the video camera control authority and begins the flow process sent to the video camera control command among Fig. 8.Each all sends the request (step S121) that control authority obtains to camera server device 111 portable display terminal 119, conversion server unit 114, display operation terminal 112 when video camera is controlled, after the acquire the right of control limit (step S122 is for being), before authority out of hand (because step S124 in for not), all repeatedly for transmission and its transmitting-receiving of replying (step S125, S126) of the control command of the operation of camera server device 111.The flow process of relevant with video camera control like this part is substantially the same.But, in a plurality of portable display terminal 119 in connection, use double-directional speech data channel this point difference.
Fig. 9 represents request of video camera control authority and video camera control command.The order of the request of expression control right, the angle of panning, angle of inclination, convergent-divergent change etc. and its are replied among Fig. 9.But, under the situation of a plurality of portable display terminals 119, be not directly to send these orders, and the various key buttons that are to use the key pushbutton unit 142 of Fig. 4 send the speech data corresponding with control command, be transformed to the video camera control command of Fig. 9 by conversion server unit 114, carry out video camera control.The request of control right too.
The flow process of expression control command among Figure 10.When the various key button of the control operation of the key pushbutton unit 142 of pressing corresponding portable display terminal, produce tone signal voice (control command voice).It is transformed to coded data speech datas such as GSM AMR by operation control unit 171, is delivered in the voice Dispatching Unit 166 of distribution services apparatus 117.In the voice Dispatching Unit 166, the speech data of former state is sent to the control authority Managing speech converter unit 163 of conversion server unit 114.
In control authority Managing speech converter unit 163, obtain this speech data, it is transformed to corresponding video camera control command, be issued to camera server device 111 and carry out video camera control.Replying of video camera control is flow process in contrast to this.The correspondence table of expression key button and video camera control command among Figure 11 A.By the combination operation of key button, generate the control command voice.Pan, inclination, scale value import with numerical key.Figure 12 is expression and the figure of the flow process of the corresponding responder action of control command of camera server device 111.When camera server device 111 is judged as object apparatus when control authority is arranged (step S131), accept control command (step S132), send response message (step S133).On the other hand, when never the device of control authority receives control command, send reply (the step S134) that does not accept to control the meaning.
Expression is from the correspondence table of reproducing with portable display terminal 119 of replying voice of replying of camera server device 111 among Figure 11 B.In the time can replying, produce the voice of reading the statement in the correspondence table.θ among Figure 11 B,
, z is that expression is panned, the numeral of angle of inclination, convergent-divergent multiplying power.
Enumerate the example of video camera control.Under the state that can obtain control authority, press key button 4,2,0 when continuous three times, generate the conversion server unit 114 20 units video camera control command as shown in Figure 13 of panning left, and be delivered to camera server device 111.The-20th, from pan the left implication of 20 units of current location.If+20, then be the order of 20 units of panning to the right.
The video camera control command of Fig. 9 and replying, as the transmission source address of first, second project, send destination address and represent the transmission source apparatus of ordering, replying, the address that sends destination apparatus respectively, the 3rd project is the identification text strings of the order or the kind of replying.Pan, angle of inclination change, the change of convergent-divergent multiplying power specify the change angle with relative value respectively.Numerical value+expression is panned to the right, is tilted, zoom enlarges ,-then be opposite.It is 1 unit that numerical value itself is established minimum controlled quentity controlled variable.During video camera control is replied, return the panning of the result that controlled, angle of inclination, convergent-divergent multiplying power with numerical value.
Reply and only turn back to the portable display terminal that sends the control command voice respectively, but only video camera control is replied all portable display terminals of turning back in the connection with the panning of verbal announcement video camera, angle of inclination, zoom state.The data mode of the voice of exchange illustrates at Figure 14 between distribution services apparatus 117 and the conversion server unit 114.
Although can be divided into little grouping during actual the transmission, the data of above-mentioned such form are two-way the exchanges.When distribution services apparatus 117 is delivered to conversion server unit 114 with speech data, the portable display terminal identifier (telephone number) of being sent the source by digital voice data corresponding with the control command voice and control command voice constitutes, on the contrary when conversion server unit 114 is delivered to distribution services apparatus 117 with speech data, constitute by the digitized portable display terminal identifier (telephone number) of reading voice and sending the target of voice.Control is replied as video camera, when speech data is returned the portable display terminal 119 of in the connection all, as portable display terminal identifier (telephone number), establish whole numbers as shown in Figure 15 and be 0 etc., expression is not delivered to the speech data of specific portable display terminal uniquely.
The then overview of the flow process during the control authority request of the control authority Managing speech converter unit 163 of expression conversion server unit 114 in Figure 16.Conversion server unit 114 has the waiting list of the identifier (telephone number) of portable display terminal 119, when when portable display terminal 119 has the control authority request of voice again, be transformed to corresponding video camera control command and send to camera server device 111 (step S171).Then, prescribe a time limit (step S172 is for being), enter the step S178 that illustrates later in acquire the right of control at once.On the other hand, in the time can not obtaining control authority at once (step S172 is for denying), will discern last (the step S173) of telephone number login at waiting list.Among the step S174, when can appending maybe can not append the time to the control authority waiting list from camera server device 111 notice, among the step S175 the voice transmission corresponding with this notice to portable display terminal 119.Afterwards, when reply when camera server device 111 returns (step S176 is for being) of giving control authority, take out identification telephone number (step S177) from the beginning of waiting list, corresponding voice are turned back to corresponding portable display terminal 119 (step S178).Among the step S179, accept control from the telephone number of the portable display terminal 119 that has obtained control authority.
When in step S180 when the control authority that finishes etc. effective time of camera server device 111 notice control authorities finishes, the portable display terminal 119 to the limit that is possessed of control power in step S181 sends the voice that the notice control authorities finish.
About using the function of conversion server unit 114, distribution services apparatus 117,115 realizations of advertising service apparatus, as long as can realize function separately, then be not limited to the apparatus structure of physics, for example repertoire can be realized on identical device.
According to this first embodiment, by using conversion server unit 114 in the path in the distribution way, just can be only to necessary terminal distribution overlapping the different images of the such additional information of advertising message, camera server device 111 does not just need to keep the content of portable display terminal 119 usefulness.By with the co-operating of advertising service apparatus 115, this additional information changeable overlapping because of constantly, the different information of video camera controlling value (PTZ value).Not only use rest image can also use live image, text in the additional information.
And then, among first embodiment, to directly not accepting from the such control device of the camera server device 111 of the control of the such key button of portable phone, by the speech data of key button being transformed to control command, carrying out the video camera control authority with regard to available key button and obtain and the video camera control operation by conversion server unit 114.Owing to will be transformed to voice from replying of camera server device 111, therefore control content confirmed in available voice.The also available voice of state of the panning of video camera, inclination, convergent-divergent are confirmed.
＜the second embodiment 〉
Among first embodiment, linking objective video camera decision when starting in distribution services apparatus 117, the conversion server unit 114.Second embodiment switches the camera server device 111 that conversion server unit 114 connects from the outside.Here, the method for switching from portable display terminal 119 is described.
Basically identical with above-mentioned first embodiment, but the action of conversion server unit 114 has some differences, and therefore the difference with first embodiment only is described.Figure 17 represents the flow process of the camera switching control seen from portable display terminal 119.In being connected to the portable display terminal 119 of distribution services apparatus 117, send the camera switching order.The camera switching order is specified by the combination of the key button of key pushbutton unit 142 shown in Figure 4.Here, be to press # (step S191).So, the control authority Managing speech converter unit 163 that similarly digital speech is delivered to conversion server unit 114 with control authority request and the video camera control command of first embodiment.
In the control authority Managing speech converter unit 163, when it being interpreted as camera server device switching command, carry out voice answer-back, inquiry password (step S192).Here enter password with portable display terminal 119, if password correct (step S193) then returns voice answer-back (step S194), the video camera sequence number of input switching target.Control authority Managing speech converter unit 163 has the such video camera sequence number of Figure 18-video camera name (speech data is arranged)-video camera address correspondence table, uses this information to obtain replying the address of voice and camera server device 111.
When input video camera sequence number (step S195), in the control authority Managing speech converter unit 163, use Figure 18 to be transformed to the address (for example being made as 100.20.30.102) of corresponding camera server device 111, in case finish to be connected so far with being connected of camera server device 111, newly be connected to the camera server device 111 of address with 100.20.30.102.Just can switch camera server device 111 thus from portable display terminal 119.Last in step S196, the video camera of switching target is notified to portable display terminal 119 with voice.
By prepare additional connector in conversion server unit 113, change connects the address of camera server device, is connected to different camera server devices 111, can realize that also the camera server device switches.
＜the three embodiment 〉
The 3rd embodiment when switching the camera server device, switches advertisement according to the camera server device that connects as second embodiment.
Except that the different this point of advertisement information list of the linking objective video camera this point of switching conversion server unit 114 from the outside and advertising service apparatus 115, basic identical with first embodiment.The camera switching method can realize with the method for second embodiment explanation.At this moment, database as advertising service apparatus 115, if alternate figures 7 has the such advertisement information list of Figure 19, then as the data that are delivered to from conversion server unit 114 advertising database 170, PTZ value except current time and video camera, the address of the camera server device in also transmit connecting, will be in the table of Figure 19 the advertising message of initial consistent project with the address of camera server device, promptly lap position and promotional literature are taken conversion server unit 114.Thus, just can switch the advertising message of demonstration according to the camera server device 111 that connects.
＜the four embodiment 〉
When the 4th embodiment is the connecting path of considering from portable information terminal 119 to camera server device 111 in the structure of first embodiment, can a plurality of paths select the combination of distribution services apparatuses 117, conversion server unit 114, advertising service apparatus 115.
The structure of expression the 4th embodiment among Figure 20.Each device, terminal all are present on a plurality of networks, can discern uniquely mutually equally with first embodiment.Because the action of each basic device is identical with first embodiment, so about each structure, the additional same reference sequence number to identical with Fig. 1 only illustrates the difference as system here.
The telephone number that portable display terminal 119 is phoned the distribution services apparatus connects, and carries out image demonstration and video camera control, but to there being the different connection telephone number of each distribution of a plurality of distribution services apparatuses 117.Therefore, if be connected to different distribution services apparatuses 117, just be connected to different conversion server units 114, different advertising service apparatuses 115.For example among Figure 20, when being connected to distribution services apparatus 117a, use conversion server unit 114a, advertising service apparatus 115a, when being connected to distribution services apparatus 117b, use conversion server unit 114b, advertising service apparatus 115b.If conversion server unit 114a is connected to identical camera server device 111 with 114b, the image of then seeing is identical, and video camera is controlled too.
But if the advertisement information list content difference of Fig. 7 that advertising service apparatus 115 has, even if then connect identical video camera, overlapping information is also with difference.
Thus, for example under the advertisement that hope the is published situation too much, by adopting this structure, even if identical camera server installation drawing picture also can switch ad content for the camera server device.
＜the five embodiment 〉
The 5th embodiment is in first embodiment, not overlay information but switch image and show on image.Different with first embodiment are described.
In conversion server unit 114, not with the step S113 of Fig. 6, it is overlapping that S114 is made as advertisement, but temporarily interrupt camera review, state of a control information that image switching is obtained for the image that retrieves from advertising database, image, text or from the camera server device etc. gets final product.As the timing that switches to such advertising message, because the situation that picture roll is difficult to watch in the video camera control is a lot, so carry out switching to advertising message and getting final product in the control period according to control information at camera server device 111.Therefore, newly append flow process shown in Figure 21.Just, beginning PTZ control and do not stop the state of PTZ action, i.e. state in the video camera action turns back to conversion server unit 114 (step S201) from camera server device 111.State is included in each frame headers of Motion jpeg image in the video camera action.Then, in the flow process of Figure 21, detect state (step S202) in this video camera action, the input advertisement gets final product (step S203) in the video camera action.
Although among the 5th embodiment, in video camera action, switch to advertising message and show, also can be in addition
1), switches to advertising message and show at video camera control waiting time.
2) the conversion server unit is connected to the camera server device, and view data arrives and to switch to advertising message during the conversion server unit and show.
3) regularly switch to advertising message and showing.
Deng in image, inserting advertising message and show.
In above first to the 5th all embodiment, display message may not be an advertising message, since data volume greatly, best reason such as insertion midway, also can be the information that should not place the information of camera server device maybe can not be placed in one, so long as should carry out overlapping information halfway, then can be any information.
According to above embodiment, by using the conversion server unit on the path in the distribution way, just can be only to the such additional information of the overlapping advertising message of the terminal of necessity, perhaps comprised the image of different additional informations in the specific timing distribution, camera server device 111 does not just need to keep the content of portable display terminal 119 usefulness.By the co-operating of advertising service apparatus, the changeable overlapping switching of this additional information shows because of constantly, the different information of video camera controlling value (PTZ value).Not only use rest image can also use live image, text etc. in the additional information.
＜the six embodiment 〉
Among the 6th embodiment, constitute a kind of information distribution system, on network configuration control video camera and send image image server, send the voice server of voice, the digital coding of image server and voice server for the band phonetic image and send to the Relay Server of receiving terminal, when receiving terminal during to the specific camera review of relaying server requests, desirable camera review and the encoded speech data that in Relay Server, determined in advance for the band phonetic image and return.
The overall structure of the information distribution system of the 6th embodiment as shown in figure 22.For on network 218, connecting the structure of Relay Server 211 and image server 212, voice server 213, client computer 219.
Connect the voice archives 216 of microphone 215 and voice data on the voice server 213, voice are sent on the network 218.The speech data of voice archives 216 also can be positioned in the internal storage device of voice server 213.Also carrying out with the same mode of image server 212, when sending request, voice server 213 returns the speech data of certain hour length to the order of voice server 213.Although G.711 the coded system of speech data has here, G.726, G.729, multiple mode such as GSM-AMR, much less the present invention and do not rely on coded system.
Then use Figure 23 that the hardware of server structure is described.Image server 212, voice server 213, Relay Server 211 are connected to network 250 among Figure 23.
CPU221, RAM222, ROM223, secondary storage device 226, VRAM225, peripheral interface 224, network interface 227 are connected to internal bus.More than the structure of illustrated such image server can use the personal computer of market sale to realize simply, but also can be via network from peripheral operation, therefore adopting does not have the so-called set-top box form of VRAM225, monitor 231, keyboard 232, mouse 233 without any problem yet.
The software configuration example of then in Figure 24, representing the 6th embodiment.Image server process 261 actions in the image server 212, voice server process 262 actions in the voice server 213, request processing procedure 265, image receiving course 263, voice receiving course 264,266 actions of image voice process of transmitting in the Relay Server 211, client's process 267 is moved respectively in the client computer.Here so-called process means the program unit that moves in the multiple task operating system.
Use Figure 25 that the action summary of each process is described.Client's process 267 is when starting, to the request processing procedure 265 requested image tabulations (S211) of Relay Server 211.Request processing procedure 265 is returned image list (S212).Image list has information such shown in Figure 26 A, describes in the back about content.Receive the guide look of the client computer display image of tabulation, the user is from one of selection operation wherein.So 267 pairs of requests of client's process processing procedure, 265 requested image are (S213) in succession.The user directly image target is input to client computer 219 in succession the time, step S211 and S212 just do not need.
The request processing procedure 265 of the Relay Server 211 that the acceptance pattern picture is asked is in succession carried out the selection (S214) of voice server 213 and voice with reference to the correspondence table 217 of voice and image.Specify image server 212 and video camera 214 and start image receiving course 263 then, specified speech server 213 and microphone 215 or voice document name etc. start voice receiving course 264 in addition.Startup will receive image and encoded speech data is an image voice process of transmitting 266 that band phonetic image data send.263 pairs of image server 212 requested image of image receiving course (S215).264 pairs of voice servers of voice receiving course, 213 request voice (S216).
The image server process 261 that accepts request obtains image (S217) from this video camera 214, and the image receiving course 263 of Relay Server 211 is returned.Voice receiving course 264 is obtained this speech data and is returned voice receiving course 264 (S218) from microphone 215 and voice archives 216 too in addition.Image that is returned and speech data are encoded to band phonetic image data (S219) in image voice process of transmitting 266, return client's process 267 (S220).Decode after client's process 267 receiving belt phonetic image data and reproduce (S221).
Then use Figure 26 A～26C explanation information relevant with voice with the image of Relay Server maintenance and with the corresponding relevant information of image and voice.Information has 273 3 kinds of the correspondence table shown in the voice table 272 shown in the image table 271 shown in Figure 26 A, Figure 26 B and Figure 26 C, in the image table 271 each video camera 214 that is connected to image server 212 is distributed picture numbers and Image Name, and IP address, port sequence number, the video camera name of image server 212 managed as attribute.Client computer 219 is selected Image Name and is specified the image of desirable video camera.Voice table 272 is too to each microphone 215 or file allocation voice sequence number and voice name, and IP address, port sequence number, microphone name or the filename of voice server 213 managed as attribute.
The corresponding relation of correspondence table 273 presentation video sequence numbers and voice sequence number keeps a plurality of voice sequence numbers corresponding with each picture numbers.When the user had asked to specify the image of Image Name, Relay Server 211 was obtained picture numbers from image table 271, then in correspondence table 273 with reference to this picture numbers, with reference to voice table 272, determine the position of voice on network from groups voice sequence number.Here, voice can be logined a plurality of, when user's long-time continuous is watched image, distribute these a plurality of voice successively.In the time voice can not being connected for some reason, switch to other voice of distributing to identical image.N/A represents there are not data among the figure.
More than be the action summary of the server group of the 6th embodiment, the flow chart of use Figure 27～33 is described in detail in the course of action of each process of the Relay Server 211 of the effect at performance center among the 6th embodiment.Relay Server 211 is made of request processing procedure 265, image receiving course 263, voice receiving course 264, image voice process of transmitting 266, and 3 processes beyond the request processing procedure 265 generate 1 to 1 client computer, independently move separately.
Figure 27 is the flow chart of processing procedure of the request processing procedure 265 of expression Relay Server 211.After the beginning, carry out initialization among the step S231, waiting event among the step S232.When the incident of generation, carry out event handling.Here incident only describes about the incident from client's process 267, the explanation of the incident of omission dependence OS etc.
(step S232 is for being) judges whether the client computer for having connected among the step S234 when incident is image request.(step S234 is for denying) sends image request incident and voice request incident to image server 212 and voice server 213 respectively among the step S235 under situation about having connected, returns step S602 and waits for next event.Under the situation of the client computer that does not connect (step S234 is for being), enter step S236 and confirm whether to be equal to or less than maximum number of connections.(step S236 is for denying) connects refusal to the client computer notice among the step S237 when surpassing maximum number of connections, returns step S232, waits for next event.In addition, maximum number of connections is that the disposal ability of considering Relay Server 211 pre-determines.
Be equal to or less than (step S236 is for being) under the situation of maximum number of connections, as the login process of client computer 219, logining the IP address of client computer among the step S238.Under the situation that the personal information of client computer 219 is sent here simultaneously, also it is logined in addition.Then, obtain the voice corresponding with image, respectively in step S239 with among image receiving course, the step S240 with among voice receiving course, the step S241 image voice process of transmitting being started, return step step S232, wait for next event.
In step S233, do not have to enter step S242 under the situation of connection request incident, judge whether to connecting End Event.The situation that this incident has client computer 219 to send, perhaps can not be under the situation that client computer sends in image voice process of transmitting 266 situation that takes place of incident by way of exception.(step S242 is for being) under these circumstances enters step S243 and connects end process.Connect end is started in the end process image receiving course 263, voice receiving course 264, image voice process of transmitting 266 when connecting beginning.Then enter step S244, delete this client computer, return step S232, wait for next event from the list that connects client computer.
(step S242 is for denying) enters step S245 under the situation that is not the connection End Event, judges whether to be video camera control request incident.Under the situation that is, enter step S246, the video camera control command from client computer is sent to image server 212, carry out step S232 after the end, wait for next event.
(step S245 is for denying) enters step S247 under the situation that is not video camera control request incident, judges whether to be the image list request event.Under the situation that is, among the step S248 image list returned client computer after, return step S232, wait for next event.(step S246 is for denying) returns step S232 under the situation that is not the image list request event, waits for next event.
The image receiving course 263 in the Relay Server 211 and the course of action of voice receiving course 264 then are described.Figure 28 is that course of action, Figure 29 of image receiving course is the course of action of voice receiving course.
Then in step S254, whether unusually successfully do not obtain image among determining step S252 and the S253.What is called is to receive the situation that can not receive fully owing to reasons such as network cut-outs in the way unusually.In that (step S254 for not) arranged when unusual, enter step S257, if be equal to or less than maximum attempts, then return step S252, attempt obtaining once again image.If surpass maximum attempts, then enter step S258, send exception generation incident and end.
In step S254, do not have to enter step S255 when unusual, the image that receives is kept in the buffer.Then in step S256, confirm whether just sending the finish command.This is the order that takes place among the step S243 of Figure 27.Under the situation of just sending this order, end process.In the absence about sending, return step S251 and continue and handle.
Then in step S265, whether unusually successfully do not obtain image among determining step S261 and the S264.Here what is called is to receive the situation that can not receive fully owing to reasons such as network cut-outs in the way unusually.Have when unusual, enter step S268, when being equal to or less than maximum attempts, return step S261, trial once again obtains voice.If surpass maximum attempts, then enter step S269, send exception generation incident, and finish.
There not being when unusual (among the step S265 for being), enter step S266, the voice that receive are kept in the buffer.Then in step S267, confirm whether to send the finish command.This is the order that takes place among the step S243 of Figure 27.Sending under the situation of this order end process.In the absence about sending, return step S260 and continue and handle.
Then use the course of action of Figure 30 key diagram as voice process of transmitting 266.After handling beginning, whether among the step S271, judging has the image speech data in frame buffer and speech buffer.Under all non-existent situation of which data, enter step S272.In step S272, surpass maximum attempts, be judged as when not having the image speech data, enter step S278, send mistake, exceptional cast and end take place among the step S279 to client computer 219.If be equal to or less than maximum attempts, through after the stand-by period, execution in step S271 once more.
Have among the step S271 under the situation of image speech data, enter step S273, each image and voice are generated coded data as the band phonetic image.Although here, there are multiple modes such as MPEG, RealVideo, Windows (R) Media in coded system, and the present invention does not also rely on coded system.Under the situation of the arbitrary existence of image voice, also can encode.The coding back sends coded data to client computer 219 in step S274.
Then judgement has no abnormal when sending in step S275.Have when unusual, judge whether to surpass the maximum attempts of the transmission that is predetermined among the step S277.When surpassing, enter step S278 after having carried out wrong transmission, exceptional cast and end take place in step S279.When being equal to or less than maximum attempts, returning step S274 and also send once more.
Being judged as among the step S275 does not have when sending to judge among the step S276 to have or not the finish command when unusual.This has situation about taking place in the step S243 of Figure 27, perhaps the situation of incident generation by way of exception in image receiving course 263 or voice receiving course 264.End process when the finish command is arranged.Return step S273 when not having the finish command, encoding then sends.
As seen from the above description, according to the 6th embodiment, can construct the internet protocol camera system and the information distribution system of additional picture specification of available voice and advertisement.
＜the seven embodiment 〉
Then the seventh embodiment of the present invention is described.Correspondence table 217 multifunctions that the 7th embodiment has Relay Server 211, can handle more detailed corresponding relation in the 6th embodiment.So-called detailed corresponding relation is when correspondence image and voice, utilize pan, personal data such as the camera parameters of inclination, convergent-divergent etc., time period, age of user, sex, residence wait and limit correspondence.The hardware configuration of the 7th embodiment is identical with the 6th embodiment with software configuration, and the correspondence table 217 of Relay Server 211 management is different with the action of request processing procedure 265.Therefore the part different with the 6th embodiment only be described below.
Figure 31 represents the correspondence table of Relay Server maintenance in the present embodiment and the example of condition table.Figure 31 A is the example of condition table 281, and Figure 31 B is the example of correspondence table 282.Condition table 281 with delegation as a condition Allotment Serial Number, to each condition sequence number with the time period, pan, condition that camera parameters of inclination, convergent-divergent etc., age, sex, residence etc. are relevant with user's personal information keeps as the scope of value and value.
The example of correspondence table 282 is compared with the correspondence table 282 of the 6th embodiment, each picture numbers is added the row that condition of contact is kept as the condition sequence number.When condition stub is None, means and unconditionally carry out correspondence.When the user has specified certain image, only under the situation of the whole unanimities of incident, permit the correspondence of image and voice then.Under inconsistent situation, can not send voice, also can be predetermined the voice corresponding with this situation.
Follow the course of action of the request processing procedure 265 of the Relay Server 211 enterprising action works that are illustrated in the 7th embodiment among Figure 32.In Figure 32,, only different parts is described the additional same steps as sequence number of the action identical with Figure 27.
For the processing of the image request incident below the step S233, in condition table 281, have under the situation of the condition relevant with camera parameters, the table 271 with reference to Figure 26 A among the step S280 is obtained camera status from this image server 212.This is the processing that obtains the camera parameters of the video camera corresponding with the desirable image of client computer.Then, will come out from the retrieval of condition table 281, select the voice of corresponding condition sequence number from the correspondence table 282 of Figure 31 B with group with the term harmonization headed by the camera parameters with reference to the condition of condition table 281.Then, 211 pairs of Relay Servers send the request that obtains of this speech data with the voice server 213 that selected voice are corresponding in step S239, and receive this speech data.
When in condition table 281 userspersonal information being arranged, client computer 219 needs to send user's personal information.At this moment, send personal information simultaneously with the image connection request that sends from 219 pairs of Relay Servers of client computer 211.Relay Server 211 comes out from the retrieval of condition table 281 based on the personal data handle that receives and the group of term harmonization, selects the voice of corresponding condition sequence number from correspondence table 282.Then, Relay Server 211 to sending the request that obtains of this speech data with the voice server 213 that selected voice are corresponding in step S239, and receives this speech data.
When in condition table 281, time information being arranged, Relay Server 211 be included in from client computer 219 have view data obtain request the time the moment the group of time period come out from the retrieval of condition table 281, select the voice of corresponding condition sequence numbers from correspondence table 282.Then, 211 pairs of Relay Servers send the request that obtains of this speech data with the voice server 213 that selected voice are corresponding in step S239, and receive this speech data.
When the video camera control request being arranged (step S245 is for being), among the step S282 image server 212 is sent the control command of this video camera from client computer.Then obtain the parameter information of video camera among the step S283.Whether judge among the step S284 then needs to connect once more about voice.Whether this is for by confirm the condition when permitting current connection referring again to the condition table 281 of Figure 31 A, continue to set up after video camera control.If set up, enter step S232, wait for next event.If be false, owing to need to connect once more,, in step S285, carry out connection processing once more so the condition table 281 that refers again to Figure 31 A is checked the voice sequence number of corresponding condition sequence number from the correspondence table 281 of Figure 31 B.The processing that this is meant attribute sound server 212 and microphone 215 or file and starts the voice receiving course once more.
Such as mentioned above, according to the 7th embodiment, condition of personal information by fixed time, camera parameters, user etc. etc., can more detailed decision corresponding relation, consequently can when carrying out picture specification, offer some clarification on thing shown in the picture with voice with voice, perhaps in speech advertising etc. to the additional resultful voice of image.
＜the eight embodiment 〉
The eighth embodiment of the present invention then is described.The 8th embodiment also can utilize the portable terminal of portable phone etc. except that the PC client computer in the 6th or the 7th embodiment.The system configuration of the 8th embodiment as shown in figure 33.
Figure 33 has increased mobile radio communication and portable phone client computer in Figure 22.In Figure 33, the additional same reference numeral of the structure identical with Figure 22 is omitted explanation, difference only is described.Portable terminal device client computer 292 is connected to the gateway of the Distribution Center 290 of mobile communication carrier wave via mobile radio communication 291.Gateway is transformed to communication mode on the network 218, exchange message to the communication mode of mobile radio communication then.Mode and the mode of utilizing the packet communication mode that the circuit switching mode utilized is arranged in the communication mode of portable terminal device client computer 292 and gateway.
Therefore, when using portable phone as terminal, in the gateway in Distribution Center 290, to each image assign telephone numbers of being photographed by each video camera 214, having from terminal during to the calling of the telephone number corresponding with image, the gateways in the Distribution Center 290 are to the corresponding image of Relay Server 211 requests.If then the band phonetic image from Relay Server 211 is transformed to image list at mobile communication in gateway, just can reproduces by the terminal reception.
In the connection that utilizes the packet communication mode, if use the known service of reproducing the live image montage, then when Relay Server 211 is specified video camera 214, generate and return by Relay Server 211 image has been carried out the video clipping that synthesizes with corresponding voice, so can receive it with terminal and reproduce via gateway.
Under circuit switched and situation that packet switching can be connected simultaneously, can on the picture on the mobile telephone, carry out camera operation, receive speech data while obtain rest image.At this moment, the band phonetic image data of returning from Relay Server 211 are divided into the voice data concurrency that Still image data that packet communication uses and circuit switched use and deliver to terminal gateway.
Such as mentioned above, according to the 8th embodiment, can in the 6th embodiment, carry out the network shooting machine operation that will utilize the portable terminal device of mobile radio communication as the band voice of client computer.
＜the nine embodiment 〉
The ninth embodiment of the present invention then is described.The correspondence table 217 (273 or 282) of the 9th embodiment image that variable Relay Server 211 has in the 6th or the 7th embodiment and voice, condition table 281.This appends by sending to Relay Server 211, upgrade, the request of deletion etc. realizes.
For example, consider the change of the correspondence table 273 of Figure 26 C.The kind of change order is appended two kinds of renewal, deletions.Appending with the difference of upgrading is to append under the non-existent situation of data relevant with specified picture numbers, upgrades under situation about existing.The request of Relay Server 211 as described in the 6th embodiment, is for example adopted the HTTP request of URL coding and the form of replying.Request of enumerating below and the example of replying.
A) upgrade 1 request to appending of correspondence table: http://host-address:port/addctbl? video=id﹠amp; Amp; Sound=id[﹠amp; Amp; Id was voice sequence number (can specify a plurality of) when sound=id...] wherein id was picture numbers, sound=id during video=id.Reply: HTTP/1.0 200OKContent-Type text/plain$r$nOKvideo_id wherein video_id is a picture numbers.
B) from the removal request of correspondence table: http://host-address:port/delctbl? video=id[﹠amp; Amp; Video=id...] wherein, id is picture numbers (can specify a plurality of) during video=id.Reply: HTTP/1.0 200OKContent-Type text/plain$r$nOK.
To appending in the update request of correspondence table, specify image sequence number and the voice sequence number corresponding with image.The voice sequence number can be specified a plurality of.Specify picture numbers in the removal request and delete corresponding data.The picture numbers of deletion can be specified a plurality of.When client computer connects deleted picture numbers request, relayed images or be predetermined the voice of corresponding regulation only.
Then about to the appending, upgrade, delete of condition table 281 shown in Figure 31 A, can stipulate following request and reply and carry out.
C) to the update request of appending of condition table: http://host-address:port/addqtbl? qid=num﹠amp; Amp; Attr=val1+val2[﹠amp; Amp; Attr=val1+val2...] wherein the num of qid=num is the condition sequence number.The attr of attr=val1+val2 is an attribute-name, and val1 and val2 are lower limit and higher limit.The example of attr is pan, tilt, zoom, time, age, sex etc.Reply: HTTP/1.0 200OKContent-Typetext/plain$r$nOKqid=qualify_id, wherein qualify_id is the condition sequence number.
D) from the removal request of condition table: http://host-address:port/delqtbl? qid=num[﹠amp; Amp; Qid=num...] wherein, the id of qid=id is condition sequence number (can specify a plurality of).Reply (during success): HTTP/1.0200 OK Content-Type text/plain$r$nOK.
When appending of condition table 281 upgraded, if the condition of having specified the condition sequence number is arranged, then upgrade, then do not append.When not having the specified requirements sequence number, additional new condition sequence number is also returned.When having specified the deletion of condition table, if the condition suitable with the condition sequence number arranged then delete.
Then when in the appending/upgrades of the correspondence table 282 of Figure 31 B, when carrying out the renewal of incident, if append the attribute relevant with condition in the above-mentioned expression formula of appending update request, that is, following correction is appended renewal 1 to above-mentioned correspondence table 273.
E) upgrade 2 requests to appending of correspondence table: http://host-address:port/addctbl? video=id[﹠amp; Amp; Qid=id] [﹠amp; Amp; Sound=id[﹠amp; Amp; Sound=id...]] wherein, the id of video=id is that the id of picture numbers, sound=id is voice sequence number (can specify a plurality of).The id of qid=id is the condition sequence number.
Be used to realize the action of the Relay Server 211 that upgrades, the renewal formality that increases correspondence table and condition table to the course of action of the Relay Server of Figure 32 gets final product.That is,, have from above-mentioned a) to e when in event handling) request the time, carry out correspondence table 273 or 282 and/or the change of appending renewal, deletion etc. of condition table 281 handle, and the wait of carrying out next event gets final product.
Such as mentioned above, according to the 9th embodiment, by in the 6th and the 7th embodiment,, just can dynamically change correspondence and respective conditions to adding the change processing of appending renewal, deletion etc. by the image of Relay Server use and correspondence table, the condition top application of voice.
As mentioned above, can construct and to receive view data, speech data respectively from picture transmitter device and voice dispensing device, and the band phonetic image data that will make up them are distributed to the system of receiving system.
＜the ten embodiment 〉
Figure 34 represents that use is as the figure of an example of the image distribution system of the Relay Server of the converting means of the coded system of changing image data among the tenth embodiment.
In Figure 34, video camera 301 is obtained image in real time, view data is transformed to QVGA or 1/16VGA size in image Distributor 302, and then coded image data is become the JPEG mode.Relay Server 303 is view data the image size of QCIF from QVGA or 1/16VGA size conversion for to portable phone line network 306 distribution of images by the aftermentioned method, and the coded system with view data becomes the MPEG mode from the JPEG mode conversion simultaneously.By top system, just can be to taking phone 304a, 304b, 304c .... the image of middle distribution video camera.
In addition, although below among Shuo Ming the tenth embodiment, if the coded system of the image before the transcoding, coding transform is the JPEG coded system, the coded system of the image behind the transcoding, coding transform is the mpeg image coded system, but piece is cut apart for comprising, the combination of other coded systems of the variety of processes of orthogonal transform and entropy coding also is effective, and then the image before and after the transcoding, coding transform also can be identical coded system.
Figure 35 is illustrated in the tenth embodiment of the present invention from the image of QVGA to show that size (horizontal 320 pixels, vertical 240 pixels) is transformed to the flow chart of processing procedure that image than its also little QCIF shows the coded data converting mode of utilizing Relay Server 303 of big or small (horizontal 176 pixels, vertical 144 pixels).
In addition, among the embodiment that the following describes, as long as the image of the image size behind the transcoding, coding transform before than transcoding, coding transform is slight greatly, then also effective to the combination of other image sizes.
In the step S311 of Figure 35, carry out JPEG entropy decoding (Hofmann decoding or arithmetic decoding) by jpeg encoded data, generate the orthogonal transform data (more correctly say so generate resulting each piece of each MCU that in the image range of QVGA size, comprises (Minimum Coding Unit) orthogonal transform data) of the image of QVGA size the image of QVGA size.
Among the step S312 as described in using Figure 36 back, by (part of more generally saying so block boundary line) cuts out the scope of the view data of QCIF size along part MCU boundary line from the scope of the view data of QVGA size, obtain the orthogonal transform data (more correctly say so and obtain the orthogonal transform data that each are included in resulting of MCU in the image range of QCIF size respectively) of QCIF size.
Here use Figure 36 that the view data that cuts out the QCIF size from the view data of QVGA size is described.
The figure of the corresponding relation of the image range that Figure 36 is expression when the image range of QVGA size cuts out the image range of QCIF size.When the coordinate in the upper left corner of the image range of establishing the QVGA size is the coordinate in (0,0), the lower right corner when being (319,239), from the upper left corner coordinate of the image range that cuts out here for (x1, in the time of y1), the coordinate in the lower right corner be exactly (x1+175, y1+143).Wherein, x1 and y1 must be respectively the multiples of minimum treat piece MCU (the Minimum Coding Unit) size of JPEG coding.For example when the image range of corresponding horizontal 16 pixels of MCU, vertical 16 pixels, x1 and y1 must be 16 multiples, and at this moment, the candidate value of x1 is 0,16,32,48, one of 64,80,96,112,128,155, the candidate value of y1 is 0, one of 16,32,48,64,80,96.As an example, the coordinate that expression cuts out the upper left corner is (64,48) in Figure 36, and the coordinate in the lower right corner is the situation of the image range of (239,191).
Return the explanation of Figure 35, among the step S313 the orthogonal transform image data storage of QCIF size obtained among the step S312 in frame memory, the orthogonal transform data that will be stored in present frame obtained among orthogonal transform view data before 1 frame of frame memory and the step S312 simultaneously compare by the piece among the MPEG4 (image range that comprises horizontal 16 pixels, vertical 16 pixels) unit, calculate the interframe phase residual quantity of the orthogonal transform data of each piece.
Interframe phase residual quantity and the predetermined threshold value calculated among the comparison step S313 among the step S314 when interframe phase quotient of difference predetermined threshold value is also big, enter step S315, when interframe phase residual quantity is equal to or less than predetermined threshold, enter step S316.
That is, press the processing that block unit is selected step S315 and step S316, carry out the processing of view data according to interframe phase residual quantity.
In step S315, the orthogonal transform data that obtain among the step S312 are carried out MPEG4 entropy coding (huffman coding or the arithmetic coding of MPEG4 regulation) with INTRA pattern (pattern of using the view data in the present frame to encode).On the other hand, among the step S316, be judged as the predicated error that does not have interframe, carry out the MPEG4 entropy coding based on inter-prediction error information with Inter pattern (prediction interframe encoding mode).
MPEG4 coded data by the block unit that generated among sequence arrangement step S315 or the step S316 among the step S317, generate the incomplete MPEG4 coded data of the QCIF size that does not have head, by generating suitable MPEG4 coded data head and be appended to the beginning of data, thereby generate the MPEG4 coded data of QCIF size.
Just handle to the transcoding, coding transform of the mpeg image data of QCIF size from the jpeg image data of QVGA size like this and finish.
In addition, the processing from step S313 to step S316 may not all need.With all piece of INTRA mode treatment the time, omit step S313, S314 and S316 only keep step S315 and get final product.But, and compare with all pieces of INTRA mode treatment, utilized the compression ratio of the coded data of Inter pattern to want high.
＜the ten one embodiment 〉
Use Figure 37 and Figure 38 explanation to utilize the processing of coded data converting mode of image of the image Distributor of the 11 embodiment below.
Figure 37 is illustrated in the 11st embodiment of the present invention from the image of 1/16VGA to show that size (horizontal 160 pixels, vertical 120 pixels) is transformed to the flow chart of processing procedure of coded data converting mode that image than its also big QCIF shows the server of big or small (horizontal 176 pixels, vertical 144 pixels).
In addition, below among the 11 embodiment that will illustrate, as long as the image size of the image size behind the transcoding, coding transform before than transcoding, coding transform is big, then also effective to the combination of other image sizes.
Among the step S321 of Figure 37, carry out JPEG entropy decoding (Hofmann decoding or arithmetic decoding) by jpeg encoded data, generate the orthogonal transform data (more correctly say so generate resulting each piece of each MCU that in the image range of 1/16VGA size, comprises (Minimum Coding Unit) orthogonal transform data) of the image of 1/16VGA size the image of 1/16VGA size.
Such as described in Figure 38 in step S322, by with the image range integral body of 1/16VGA size along the part MCU boundary line of the image range of QCIF size (more generally be part block boundary line) insert, insert pseudo-data (the orthogonal transform data of the value of holding in advance to be determined) to remainder, generate the orthogonal transform data (more correctly say so and generate the orthogonal transform data that each are included in resulting of MCU in the image range of QCIF size respectively) of QCIF size.
Here use Figure 38 that the view data that generates the QCIF size from the view data of QVGA size is described.
Figure 38 is the figure of the corresponding relation of the image range when representing the image range of the entire image scope of 1/16VGA size insertion QCIF size.When the coordinate in the upper left corner of the image range of establishing the QCIF size is that the coordinate in (0,0), the lower right corner is when being (175,143), at the upper left corner coordinate to the image range of the 1/16VGA size of inserting here is (x2, y2) time, the coordinate in the lower right corner be exactly (x2+159, y2+119).But x2 and y2 must be respectively the multiples of minimum treat piece MCU (the Minimum Coding Unit) size of JPEG coding.For example when the image range of corresponding wide 16 pixels of MCU, high 8 lines, x2 must be 16 multiple, and y2 must be 8 multiple, and this moment, the candidate value of x2 was 0 or 16, and the candidate value of y2 is one of 0,8,16,24.As an example, expressing the coordinate in the image range of the 1/16VGA size insertion upper left corner is (0,16) among Figure 38, and the coordinate in the lower right corner is the situation of the position of (159,135).In the residual image scope of representing with oblique line, insert pseudo-data.
Return the explanation of Figure 35, among the step S323 the orthogonal transform image data storage of the QCIF size that is generated among the step S32 in frame memory, the orthogonal transform view data of the present frame that is generated among orthogonal transform view data before 1 frame that will be stored in frame memory and the step S322 compares by the piece among the MPEG4 (image range that comprises wide 16 pixels, high 8 lines) unit simultaneously, calculates the interframe phase residual quantity of the orthogonal transform view data of each piece.
Interframe phase residual quantity and the predetermined threshold value calculated among the comparison step S323 among the step S324 when interframe phase quotient of difference predetermined threshold value is also big, enter step S325, when interframe phase residual quantity is equal to or less than predetermined threshold value, enter step S326.
All handle among the step S325 neutralization procedure S326 by block unit.Among the step S325, the orthogonal transform data that generated among the step S322 are carried out MPEG4 entropy coding (huffman coding or the arithmetic coding of MPEG4 regulation) with INTRA pattern (pattern of using the view data in the present frame to encode).On the other hand, be judged as the predicated error that does not have interframe among the step S326, carry out the MPEG4 entropy coding based on inter-prediction error information with Inter pattern (inter prediction encoding pattern).
The MPEG4 coded data of the block unit that is generated in step S325 or step S326 by sequence arrangement among the step S327, generate the incomplete MPEG4 coded data of QCIF size, by generating suitable MPEG4 coded data head and being appended to the beginning of data, generate the MPEG4 coded data of QCIF size.
Just handle to the transcoding, coding transform of the mpeg image data of QCIF size from the jpeg image data of 1/16VGA size like this and finish.
In addition, above-mentioned the tenth, the 11 embodiment capable of being combined constitutes server.The image of for example prejudging before the transcoding, coding transform shows that size shows that than the image after the conversion size is big or little, decides the processing of carrying out the tenth embodiment or the processing of the 11 embodiment gets final product according to this judgement.
＜the ten two embodiment 〉
Figure 39 is illustrated in to utilize among the 12 embodiment from the image of QVGA to show that size (horizontal 320 pixels, vertical 240 pixels) is transformed to the flow chart that shows another processing procedure of the coded data converting mode that the server of size (horizontal 176 pixels, vertical 144 pixels) carries out than its also little QCIF image.
With respect to obtaining the orthogonal transform data that in the image range that cuts out, comprise among the tenth embodiment, obtain among the 12 embodiment jpeg encoded data this point difference that in the image range that cuts out, comprises.
Although the supposition JPEG coded system or MPEG4 image encoding mode one of in, use the situation of huffman coding mode as the entropy coding mode, but piece is cut apart for comprising, the combination of other coded systems of the variety of processes of orthogonal transform and entropy coding also can realize roughly the same processing procedure.
Among the 12 embodiment that will illustrate, same with the tenth embodiment below, as long as the image of the image size behind the transcoding, coding transform before than transcoding, coding transform is slight greatly, then the combination for other image sizes also is effective.
Among the step S333 of Figure 39, as shown in figure 36, by (more generally be the block boundary line) cuts out the image range of QCIF size along any MCU boundary line from the image range of QVGA size, obtain the JPEG coded image data (more correctly say so obtain in the image range that is included in the QCIF size JPEG coded image data) of QCIF size.
JPEG in the obtained JPEG coded image data carries out conversion to the MPEG4 of INTRA pattern with huffman coding (huffman coding that uses in the INTRA pattern-coding of MPEG4 image encoding mode) with huffman coding (huffman coding that uses in the JPEG coded system) from be included in step S331 in step S332.Here for performing step S332, must discern the JPEG content of huffman coding table and the content that MPEG4 uses huffman coding table in advance.
In addition, establishing MPEG4 is identified in advance with the device (Relay Server) or software self the preparation event of huffman coding table owing to application the 12 embodiment.On the other hand JPEG with huffman coding table owing in the head of jpeg encoded data part, be defined, so partly obtain by the head of before the processing of Figure 39, analyzing jpeg encoded data.Perhaps when knowing that the image Distributor that sends jpeg encoded data uses the JPEG of identical content to use huffman coding table usually, this JPEG of simple storage gets final product with huffman coding table.
When resulting MPEG4 is aggregated into 1 frame with the Hoffman code string in step S332, just obtain incomplete (not having head) MPEG4 coded data of QCIF size.Among the step S333, the MPEG4 that obtains among the step S332 is stored into frame memory successively with the Hoffman code string, MPEG4 before 1 frame that will be stored in frame memory compares by the piece of MPEG4 (laterally vertically the scopes of equal 16 pixels) unit with the Hoffman code string with the MPEG4 of resulting present frame among Hoffman code string and the step S332 simultaneously, and whether the MPEG4 that checks each piece has difference with the interframe of Hoffman code string.
Other has or not according to the frame-to-frame differences of being checked among the step S333 among the step S334, when the interframe difference is arranged, directly enters step S336 (nothing is appended processing), when not having interframe to differ, enters step S335 (appending processing).Among the step S335, it is 0 MPEG4 Hoffman code string that the MPEG4 that comprises in the macro block in the pre-treatment all is replaced into the predicated error that means the Inter pattern with the Hoffman code string.
Among the step S336, resulting MPEG4 among step S332 or the step S335 is aggregated into 1 frame with the Hoffman code string and generates the MPEG4 coded data of QCIF size, by generating suitable MPEG4 coded data head and being appended to the beginning of data, generate the MPEG4 coded data of QCIF size.
Just handle to the transcoding, coding transform of the mpeg image data of QCIF size from the jpeg image data of QVGA size like this and finish.
In addition, the processing from step S333 to step S335 may not need.With the whole macro block of INTRA mode treatment the time, can omit step S333 all processing to S335.But, and compare with the whole macro block of INTRA mode treatment, utilized the compression ratio of the coded data of Inter pattern to want high.
＜the ten three embodiment 〉
Figure 39 represents to utilize in the present embodiment from the image of 1/16VGA to show that size (horizontal 160 pixels, vertical 120 pixels) is transformed to the flow chart of processing procedure of coded data converting mode that image than its big QCIF shows the server of size (horizontal 176 pixels, vertical 144 pixels).
With respect to the image range that among the 11 embodiment orthogonal transform data is inserted into the QCIF size, in the 13 embodiment, in the image range of QCIF size, insert MPEG4 with Hoffman code string this point difference.
Although supposition is in one of JPEG coded system or MPEG4 image encoding mode, use the situation of huffman coding mode as the entropy coding mode, but piece is cut apart for comprising, the combination of other coded systems of the variety of processes of orthogonal transform and entropy coding also can realize roughly the same processing procedure.
Among the 13 embodiment that will illustrate, same below with the 11 embodiment, as long as the image size of the image size behind the transcoding, coding transform before than transcoding, coding transform is big, then show that for other images the combination of size also is effective.
Among the step S341 of Figure 40, carry out conversion to the MPEG4 of INTRA pattern-coding with huffman coding (huffman coding that uses in the INTRA pattern-coding of MPEG4 image encoding mode) with huffman coding (huffman coding that uses in the JPEG coded system) as the JPEG that comprises the data from the JPEG compilation of 1/16VGA size.Here, for performing step S341, must know the JPEG content of huffman coding table and the content that MPEG4 uses huffman coding table in advance, these and the 12 embodiment are prepared equally in advance.
Such as shown in figure 38 among the step S342, by the image range integral body of 1/16VGA size (in comprise MPEG4 Hoffman code string) along any MCU boundary line of the image range of QCIF size (more generally be part block boundary line) insert.Insert pseudo-data (the MPEG4 Hoffman code string of holding the value that is predetermined) to remaining image range then, generate incomplete (not having head) MPEG4 coded data (more correctly say so and be created on the MPEG4 that comprises in the image range of QCIF size Hoffman code string) of QCIF size.
The step S333 of Figure 40～S336 represents the processing identical with Figure 39 then, omits this explanation.
＜the ten four embodiment 〉
Figure 41 is illustrated among the 14 embodiment, utilizes the image of QVGA is shown that size (horizontal 320 pixels, vertical 240 pixels) is transformed to the flow chart of processing procedure of coded data converting mode that shows the server of big or small (horizontal 176 pixels, vertical 144 pixels) than its little QCIF image.
Among the 14 embodiment, before the coded system conversion image size is made as 1/2, inserts pseudo-data in advance, this point is different with the tenth embodiment.
Among the step S361 of Figure 41,, generate the view data of QVGA size by the jpeg encoded data of jpeg image decoding QVGA size.
Carry out among the step S362 image between dredge to handle, on direction in length and breadth, all view data is reduced into 1/2 (so long as than the multiplying power of the also little such 1/n of the image size of QCIF can), the view data of generation 1/16VGA size.
Figure 42 is that to be illustrated on the direction in length and breadth all the image size reduction be the figure of situation of the increase of the piece distortion that produced in 1/2 o'clock.In the figure, there be caused distortion of JPEG coding in solid line position on (wide 8 pixel separation, high 8 lines are at interval).When all the image size reduction being 1/2 on direction in length and breadth, caused distortion of JPEG coding is just mobile to the position (wide 4 pixel separation, high 4 lines are at interval) of solid line and dotted line.And then when this image is carried out the MPEG4 image encoding, the possibility of appending caused distortion of MPEG4 image encoding once more on position shown in the solid line is just arranged.Just, because dwindling of image will produce new piece distortion on dotted line position.
Return Figure 41, not obvious for the piece distortion that makes new generation on the described dotted line position of Figure 42 in step S363, near each pixel the dotted line position (central shaft of the direction separately of the portraitlandscape of each piece) that is positioned at Figure 42 is carried out smoothing handle.
Such as shown in figure 38 in step S364 then, by the data of the image range integral body of 1/16VGA size are inserted along any MCU boundary line (part of more generally saying so block boundary line) of the image range of QCIF size, in remainder, insert pseudo-data (view data of holding the value that is predetermined), generate the view data of QCIF size.Carry out the MPEG4 image encoding by view data among the step S365, generate the MPEG4 coded image data of QCIF size the QCIF size.
These jpeg image data from the QVGA size are just handled to the transcoding, coding transform of the mpeg image of QCIF size and are finished.
＜the ten five embodiment 〉
Figure 43 is illustrated among the 15 embodiment to utilize from the image of 1/16VGA to show that size (horizontal 160 pixels, vertical 120 pixels) is transformed to the flow chart of processing procedure of coded data converting mode that image than its also big QCIF shows the server of big or small (horizontal 176 pixels, vertical 144 pixels).
Among the 15 embodiment, before the coded system conversion image size is made as 2 times, cuts out the image of QCIF size in advance, this point is different with the 11 embodiment.
In the step S351 of Figure 43, generate the view data of QVGA size by the jpeg encoded data of jpeg image decoding QVGA size.
Carry out the interpolation processing of image among the step S352, on direction in length and breadth, all view data is enlarged into 2 times (so long as than the image of QCIF show the also big such multiplying power of size can), generate the view data of QVGA size.
In step S353, such as shown in figure 36, by (part of more generally saying so block boundary line) cuts out the image range of QCIF size along any MCU boundary line from the view data that generated by the QVGA size, just obtain the view data of QCIF size.
Carry out the MPEG4 image encoding by view data among the step S354, generate the MPEG4 coded image data of QCIF size the QCIF size.
Just handle to the transcoding, coding transform of the mpeg image of QCIF size from the jpeg image data of 1/16VGA size like this and finish.
As described above, according to the tenth～the 15 embodiment, when coded data converting is the different pieces of information form, make by trying every possible means that the block boundary line as far as possible moves before and after conversion, just can suppress the deterioration of image of the piece distortion that causes by conversion.
＜other embodiment 〉
Need not superfluous words, purpose of the present invention also can reach like this, the storage medium of the software program code by will having write down the function that realizes above-mentioned form of implementation (for example offers computer system or device, personal computer), this system or device use CPU or MPU to read and carry out the program code that is kept in the storage medium.
In this case, just become the program code of reading from storage medium self will realize the function of above-mentioned form of implementation, the storage medium of storing this program code has just constituted the present invention.
In addition,, for example can use the storage medium of floppy disk, hard disk, CD, magneto optical disk, CD-ROM, CD-R, tape, Nonvolatile memory card and ROM etc., the perhaps computer network of LAN (local area network (LAN)) and WAN (wide area network) etc. for program code is provided.
Need not superfluous words in addition, not only comprise the program code of reading by object computer, the situation that the function of above-mentioned form of implementation is achieved, also comprise indication according to this program code, Yun Hang operating system (OS) etc. is carried out the some or all of of actual treatment on computers, the situation that the function by this said form of implementation in processing front is achieved.
And then need not superfluous words, also comprise the program code of reading when from storage medium, be written to the function expansion card that inserts computer and/or be connected to after the memory that is possessed on the functional expansion unit of computer, indication according to this program code, the CPU that is possessed on this function expansion card and/or the functional expansion unit etc. carries out the some or all of of actual treatment, the situation that the function by this said form of implementation in processing front is achieved.
Be applicable in the present invention under the situation of above-mentioned storage medium, just be kept in this storage medium with the previous flow chart corresponding programs code that illustrated.
The present invention is not limited to above-mentioned form of implementation, can carry out various changes and distortion without departing from the spirit and scope of the present invention.Therefore, in order to disclose scope of the present invention, additional following claim item.
Claims (5)
1. information distribution device, this information distribution device can be connected with a plurality of speech detection devices with a plurality of image-pickup devices, and this information distribution device comprises:
The view data receiving element receives described view data from the picture transmitter device that can send view data;
The speech data receiving element receives speech data from the voice dispensing device;
The information holding unit keeps the corresponding informance between view data and the speech data;
Coding unit, based on the corresponding informance that remains in the above-mentioned information holding unit, above-mentioned view data that will be received by above-mentioned view data receiving element and the above-mentioned speech data that is received by above-mentioned speech data receiving element make up, and encode as band phonetic image data; And
Dispatching Unit will be distributed to receiving system by the above-mentioned band phonetic image data that above-mentioned coding unit generated;
Wherein, under a plurality of speech datas and the corresponding situation of above-mentioned view data, above-mentioned coding unit makes up above-mentioned view data and above-mentioned a plurality of speech data successively, and under the situation that the arbitrary speech data in above-mentioned a plurality of speech datas can not be received by above-mentioned speech data receiving element, above-mentioned coding unit makes up the speech data except that the above-mentioned speech data that can not receive in above-mentioned view data and the above-mentioned a plurality of speech data successively.
2. information distribution device according to claim 1 is characterized in that:
Above-mentioned information holding unit keep the condition relevant, the condition relevant with user's personal information with the state of above-mentioned picture transmitter device and the condition of being correlated with the moment at least one.
3. information distribution device according to claim 1 is characterized in that:
Also comprise the change unit,, change the content of the information that is kept by above-mentioned information holding unit according to request from above-mentioned receiving system.
4. information distribution device according to claim 1 is characterized in that:
Above-mentioned view data receiving element and above-mentioned speech data receiving element are independent and be connected to device on the network from above-mentioned information distribution device.
5. information dispensing method, this information dispensing method are used for the information distribution device that can be connected with a plurality of speech detection devices with a plurality of image-pickup devices, and this information dispensing method may further comprise the steps:
Receive described view data from image-pickup device that can the pickup image data;
Based on remaining on view data in the information holding unit and the corresponding informance between the speech data, receive speech data from the voice dispensing device;
Received view data and received speech data are made up, and encode as band phonetic image data;
Above-mentioned band phonetic image data behind the coding are distributed to receiving system,
Wherein, under a plurality of speech datas and the corresponding situation of above-mentioned view data, above-mentioned coding step makes up above-mentioned view data and above-mentioned a plurality of speech data successively, and under the situation that the arbitrary speech data in above-mentioned a plurality of speech datas can not be received by above-mentioned receiving step, above-mentioned coding step makes up the speech data except that the above-mentioned speech data that can not receive in above-mentioned view data and the above-mentioned a plurality of speech data successively.
Applications Claiming Priority (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP093698/2002 | 2002-03-29 | ||
| JP2002093994A JP2003299082A (en) | 2002-03-29 | 2002-03-29 | Method for image data converting processing |
| JP093697/2002 | 2002-03-29 | ||
| JP093994/2002 | 2002-03-29 |
Related Parent Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB031214878A Division CN1251504C (en) | 2002-03-29 | 2003-03-28 | Image data distribution |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1825949A CN1825949A (en) | 2006-08-30 |
| CN100534174C true CN100534174C (en) | 2009-08-26 |
Family
ID=29386882
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB2006100074797A Expired - Fee Related CN100534174C (en) | 2002-03-29 | 2003-03-28 | Information distribution device, information distribution system, and information distribution system method |
Country Status (2)
| Country | Link |
|---|---|
| JP (1) | JP2003299082A (en) |
| CN (1) | CN100534174C (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4611119B2 (en) | 2005-05-31 | 2011-01-12 | シャープ株式会社 | Relay device and communication system |
| JP2009010586A (en) * | 2007-06-27 | 2009-01-15 | Fujitsu Microelectronics Ltd | Trans-coder, and trans-coding method |
| CN105448296B (en) * | 2014-06-17 | 2019-03-26 | 北京司响无限文化传媒有限公司 | Information dispensing method and device and message receiving method and device |
| CN108090392B (en) * | 2017-12-29 | 2021-06-15 | 北京安云世纪科技有限公司 | Method, system and mobile terminal for processing service based on universal identification function |
-
2002
- 2002-03-29 JP JP2002093994A patent/JP2003299082A/en not_active Withdrawn
-
2003
- 2003-03-28 CN CNB2006100074797A patent/CN100534174C/en not_active Expired - Fee Related
Also Published As
| Publication number | Publication date |
|---|---|
| JP2003299082A (en) | 2003-10-17 |
| CN1825949A (en) | 2006-08-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR20030078794A (en) | Image data delivery | |
| US20080046944A1 (en) | Ubiquitous home media service apparatus and method based on smmd, and home media service system and method using the same | |
| EP1860841B1 (en) | Method and system for replacing media stream in a communication process of a terminal | |
| WO2012039404A1 (en) | Video bit stream transmission system | |
| KR102216956B1 (en) | System and method for preloading multi-view video | |
| KR101138020B1 (en) | Supporting System And Method For Virtual Object Identification Architecture based on a Virtual World | |
| US7277115B2 (en) | Communication terminal device capable of transmitting visage information | |
| CN102065340B (en) | System and method for implementing multimedia synchronous interaction | |
| CN109508362A (en) | A kind of historical geography information system based on three-dimensional panorama immersion | |
| EP1722566A1 (en) | Information distributing system and method, information distributing apparatus therefor, receiver terminal, and information relaying apparatus | |
| KR102181859B1 (en) | System and method for providing holoportation | |
| CN100534174C (en) | Information distribution device, information distribution system, and information distribution system method | |
| CN110868620A (en) | Remote interaction system and method based on television | |
| CN106533912B (en) | User communication system based on intelligent community | |
| KR100750907B1 (en) | Apparatus and method for processing image which is transferred to and displayed on mobile communication devices | |
| CN114978790B (en) | Equipment matching method, device and system | |
| JPH09244981A (en) | Display controller, display control method, transmitter-receiver and transmission and reception method | |
| CN102685027A (en) | Method and apparatus for providing person of interest-based network service | |
| KR102248081B1 (en) | Non-face-to-face universal remote platform providing system using avatar robot | |
| JP2006333332A (en) | Image information supply system | |
| KR100866786B1 (en) | Personal mobile broadcasting service method and system thereof | |
| CN108370456A (en) | Information processing method and display device | |
| WO2007046369A1 (en) | Information processing apparatus | |
| JP2007041718A (en) | Information distribution system, information distribution apparatus, information receiver, and program | |
| US20210289194A1 (en) | System, method, and computer program for generating volumetric video |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20090826 Termination date: 20150328 |
|
| EXPY | Termination of patent right or utility model |