CN1798324B - Information terminal device - Google Patents
Information terminal device Download PDFInfo
- Publication number
- CN1798324B CN1798324B CN2005100592804A CN200510059280A CN1798324B CN 1798324 B CN1798324 B CN 1798324B CN 2005100592804 A CN2005100592804 A CN 2005100592804A CN 200510059280 A CN200510059280 A CN 200510059280A CN 1798324 B CN1798324 B CN 1798324B
- Authority
- CN
- China
- Prior art keywords
- mentioned
- information
- data
- characteristic
- terminal device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F15/00—Digital computers in general; Data processing equipment in general
- G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
Abstract
To provide a usable information terminal device for wider application. A communication terminal device 12 of an image communication system 10 supplies a driver 23 and an output selecting section 56 with image data 42 and voice data 46, generates a synthesized character image by adapting to the characteristic data in an expression synthesizing section within the driver 23, transforms these character image and voice data into the image data 60 and voice data 62 in the respective predetermined communication forms in a form transformation section, sends to the output selecting section 56, and selects to output one pair of the image data 72 and voice data 74 out of the image data 60 and 42 and voice data 62 and 46 of the plurality of types.
Description
Technical field
The present invention relates to information terminal device.Information terminal device of the present invention relates to the terminal installations such as interphone as the personal computer of the terminal of communication system, portable phone or PDA information terminal devices such as (Personal DigitalAssistant), television telephone set and band camera.
Background technology
In the past, the information terminal device and the image distribution system of the band Image Communication function shown in the patent documentation 1, as not being often view data to be communicated but mainly communicate in the Image Communication of video telephone etc., so that confidentiality and the improved information system in recreational two aspects and be able to motion by characteristic point data to face.
System shown in the patent documentation 1, characteristic point from the face image contract face that is taken into, make selected characteristic have expression and be easy to carry out empathic, recreational higher communication system to construct, improve the simple and easy possibility of utilizing of Image Communication based on the data that extract and user-selected characteristic.
In addition, under the situation that the content of using in accepting Image Communication etc. provides, content is accompanied by branch and returns and carry out license management.Patent documentation 2 to 4 is arranged on this management method.
The Content Management System of patent documentation 2 is management systems of available content in certain facility, following method has been proposed: utilize positional information to manage, utilize when permission under the situation of the information terminal device that has the content of to read in the facility of scope and just can download desirable content, forbid that as if appearing at the facility that utilizes then eliminates content outward in content.
In addition, the data distribution systems of patent documentation 3 and data terminal unit have proposed to utilize the method that positional information that data terminal unit represents and licensc e data are decrypted the content-data after encrypting.And then the content reception apparatus of patent documentation 4 has proposed for for the content-data after encrypting is decrypted, according to utilize permissive condition with decruption key as utilizing end message to obtain, with the method that desirable content is decrypted.
No. 3593067 communique of [patent documentation 1] patent
[patent documentation 2] Japan Patent Publication Laid-Open 2004-86441 communique
[patent documentation 3] Japan Patent Publication Laid-Open 2002-140450 communique
[patent documentation 4] Japan Patent Publication Laid-Open 2002-344921 communique
[patent documentation 5] Japan Patent Publication Laid-Open 2003-281539 communique
The flat 8-251561 communique of [patent documentation 6] Japan Patent Publication Laid-Open
[patent documentation 7] Japan Patent Publication Laid-Open 2000-322588 communique
Summary of the invention
Yet, at present, personal computer connects camera and loudspeaker input user's image and sound, carry out data transmit-receive to be suitable for IP (Internet Protocol) by the resulting data of their input with the communication object destination, the execution by the software function the data of being supplied with reproduced in the communication object destination is achieved the Internet Protocol Television phone.In addition, also constantly be equipped with at the platform by means of Image Communication in the video telephone of portable phone etc., Image Communication as described above has been able to extensive utilization.
But at present, Image Communication is restricted to the communication of platform separately. promptly, be defined in communication with similar terminal installation. people wish the Image Communication ease of use well, popularize more widely, launch and cost is increased.
In addition, even if aforesaid any one license management gimmick, because the license information of content and content difference are managed, so when handling huge content, become miscellaneous and so on problem with regard to meaningful and management licence.Particularly, even if in the restriction when patent documentation 2 utilizes content in specific place, also have because of restriction can use two extreme, and utilize the relevant degree of freedom with content.
Purpose of the present invention be exactly eliminate as the shortcoming of prior art, provide a kind of ease of use well also can bring the information terminal device of more extensive utilization.
In order to solve above-mentioned problem, technical scheme of the present invention provides a kind of information terminal device, it is characterized in that: in the image delivery system that utilization is handled image as the communication information information terminal device communicates, this device comprises: acoustic input dephonoprojectoscope is taken into audio signal and becomes voice data and exports; Image-input device is taken into subject and becomes view data and exports as vision signal; Interface arrangement is transformed into this view data and voice data respectively the communication form of regulation; Choice device among a plurality of view data and voice data that communicate with communication object, is selected desirable view data and voice data; R-T unit carries out data transmit-receive with the information terminal device of communication object; And output device, output is from the view data and the voice data of the information terminal device of communication object, and wherein, interface arrangement comprises: the feature extraction device, at least one side extracts the characteristic of expression expression from voice data and view data; The expression synthesizer makes as the performance data of user-selected primary image based on this characteristic and to be out of shape, with the formation characteristic composograph; Image management apparatus manages the above-mentioned performance data of supplying with this expression synthesizer; And the formal argument device, view data and voice data are transformed into the communication form of regulation respectively, and then this device at least also is used to one of above-mentioned information terminal device that communicates.
Information terminal device of the present invention offers interface arrangement and choice device respectively with view data and voice data, the expression synthesizer in conjunction with characteristic formation characteristic composograph, this characteristic image and voice data are distinguished changing image data and voice data in formal argument device communication form in accordance with regulations, and offer choice device, among the view data of a plurality of kinds and voice data, select the one group of view data that will export and voice data to export at choice device, just can easily be provided at thus and recreationally go up excellent images communication and need not significantly change formation.
Description of drawings
Fig. 1 is that expression has been suitable for the block diagram that the summary of the embodiment in the communication terminal of image communication system of information terminal device related to the present invention constitutes.
Fig. 2 is the block diagram that the summary of the driver in the used communication terminal of presentation graphs 1 constitutes.
Fig. 3 is the figure that the picture of looking the monitor picture in the audio output part of key diagram 1 constitutes.
Fig. 4 is the block diagram that the summary of the characteristic administrative center in the image communication system of presentation graphs 1 constitutes.
Fig. 5 is that expression has been suitable for the block diagram that the summary of other embodiment in the communication terminal of image communication system of information terminal device related to the present invention constitutes.
Fig. 6 is that expression has been suitable for the block diagram that the summary of other embodiment in the communication terminal of image communication system of information terminal device related to the present invention constitutes.
Fig. 7 is that expression has been suitable for the block diagram that the summary of other embodiment in the communication terminal of image communication system of information terminal device related to the present invention constitutes.
Fig. 8 is the block diagram of the concrete formation in the characteristic management department of Fig. 1.
Fig. 9 is the flow chart of course of action of the characteristic management department of key diagram 8.
Figure 10 is the flow chart of course of action of the characteristic administrative center of key diagram 4.
Figure 11 is the figure of the relation of the content handled of the content management server in the characteristic administrative center of key diagram 4 and license distribution information.
Figure 12 is the flow chart of other course of action in the characteristic administrative center of key diagram 4.
Figure 13 is the figure of the relation of the content handled of the content management server in the characteristic administrative center of key diagram 4 and license distribution information.
Figure 14 is the figure of the content of the license distribution information handled of the content management server of key diagram 4.
Figure 15 is the block diagram of other configuration examples in the characteristic management department of presentation graphs 8.
Figure 16 is the flow chart of the course of action of the content output request in the characteristic management department of explanation Figure 15.
Figure 17 is the block diagram that expression has been suitable for the formation of the Fare Collection System in the image communication system of characteristic management department of Figure 15.
Figure 18 is the figure of the relation of the content handled and license distribution information in the Fare Collection System of explanation Figure 17.
Figure 19 is the flow chart of course of action of the content output request of the explanation Fare Collection System that utilized Figure 17.
Figure 20 is the block diagram of other configuration examples in the characteristic management department of expression Figure 15.
Figure 21 is the figure of the relation of the content handled of the characteristic management department of explanation Figure 20 and license distribution information.
Figure 22 is the flow chart of course of action of the characteristic management department of explanation Figure 20.
Figure 23 is the block diagram that expression has been suitable for other formations of the Fare Collection System in the image communication system of characteristic management department of Figure 17.
Figure 24 is the figure of the relation of the content handled and license distribution information in the Fare Collection System of explanation Figure 23.
Figure 25 is the flow chart of course of action of the content output request of the explanation Fare Collection System that utilized Figure 23.
Figure 26 is that expression has been suitable for the block diagram that the summary of other embodiment in the communication terminal of image communication system of information terminal device related to the present invention constitutes.
Figure 27 is the figure of formation of the information generating unit in the communication terminal of expression Figure 26.
Figure 28 is the information handled of the communication terminal of expression Figure 26 and the corresponding figure of announcement information.
Figure 29 be illustrated in Figure 26 communication terminal look the figure that notice that audio output part shows shows example.
Figure 30 is the flow chart of course of action in the communication terminal of explanation Figure 26.
Figure 31 is the synthetic figure of object that illustrates in the past.
Figure 32 is the figure of kind of the distortion of explanation expressive object.
Figure 33 is the figure of the composition principle of characterisation image.
Figure 34 is that expression has been suitable for the block diagram that the summary of other embodiment in the communication terminal of image communication system of information terminal device related to the present invention constitutes.
Figure 35 is that expression has been suitable for the block diagram that the summary of other embodiment in the communication terminal of image communication system of information terminal device related to the present invention constitutes.
Figure 36 is that expression has been suitable for the block diagram that the summary of other embodiment in the communication terminal of image communication system of information terminal device related to the present invention constitutes.
Figure 37 is that demonstration in the communication terminal of image communication system of expression Figure 26, Figure 34～Figure 36 is wanted portion's block diagram according to other configuration examples of the notice of prompting.
Figure 38 is the block diagram of other configuration examples in the information generating unit of expression Figure 37.
Figure 39 is the corresponding figure that expression has utilized information and announcement information under the situation of information generating unit of Figure 38.
Figure 40 is the flow chart of course of action of the communication terminal of the explanation information generating unit of having utilized Figure 38.
Figure 41 is the block diagram of wanting portion's formation that expression has been suitable for other embodiment in each communication terminal of image communication system of information terminal device related to the present invention.
Figure 42 is the flow chart of the course of action in the transmitter side communication terminal of image communication system of explanation Figure 41 of being suitable for information terminal device related to the present invention.
Figure 43 is the flow chart of explanation with respect to the course of action in the receiver side communication terminal of the transmitter side communication terminal of Figure 41.
Figure 44 is the block diagram of wanting portion's formation that expression has been suitable for other embodiment in the communication terminal of image communication system of information terminal device related to the present invention.
Figure 45 is the flow chart of course of action in the communication terminal of explanation Figure 44.
Figure 46 is a block diagram of having represented to be suitable for the formation of communication terminal in the image communication system of information terminal device related to the present invention and server respectively.
Figure 47 is the flow chart of course of action in the image communication system of explanation Figure 46.
Figure 48 is the flow chart of course of action of the server in the image communication system of explanation Figure 46.
Figure 49 is the block diagram of wanting portion's formation that expression has been suitable for other embodiment in the communication terminal of image communication system of information terminal device related to the present invention.
Figure 50 is the block diagram of wanting portion's formation that expression has been suitable for other embodiment in the communication terminal of image communication system of information terminal device related to the present invention.
Figure 51 is the block diagram of wanting portion's formation that expression has been suitable for other embodiment in the communication terminal of image communication system of information terminal device related to the present invention.
Figure 52 is the block diagram of wanting portion's formation that expression has been suitable for other embodiment in the communication terminal of image communication system of information terminal device related to the present invention.
Embodiment
Then describe embodiment in detail according to information terminal device of the present invention with reference to accompanying drawing.
Present embodiment is the situation that information terminal device of the present invention is applicable to the communication terminal at least 12 in the image communication system 10.For not having the part of direct relation then to omit diagram and explanation with the present invention.Signal being indicated with the reference numbering of its connecting line that manifests in the following description.
Expression comprises the present embodiment of situation of the image communication system 10 of identical inscape, utilize communication terminal 12 to disclose. communication terminal 12 comprises image input part 20, sound input part 22, driver 23, operating portion 28, characteristic management department 30, encoding section 32, receiving and transmitting part 34, lsb decoder 36, look audio output part 38 and output selection portion 56. with respect to this, under the situation of constructing with different inscapes and simplifying, communication terminal 14 can also not have expressive features extracting part 24, synthetic portion 26 of expression and characteristic management department 30. utilize the synthetic input picture of characteristic just to be sent to communication terminal 12. from communication terminal 14 in the case
In addition, in the later embodiment of present embodiment, establish and do not import selection portion 25 but view data 42 and voice data 46 directly supplied with expression feature extraction portions 24.Under the situation that input selection portion 25 is set, view data 43 and voice data 47 are not only supplied with expression feature extraction portion 24, though diagram can also not make it from driver 23 straight-through outputs.
Expressive features extracting part 24 have based on the view data of being supplied with 43 and voice data 47 try to achieve the expression emotion data function and try to achieve the function of the characteristic of the expression expression that view data 42 held.Expressive features extracting part 24 outputs to the synthetic portion 26 of expression with emotion data and the characteristic of being obtained 48.Wish that expressive features extracting part 24 to major general's characteristic offers the synthetic portion 26 of expression.24 outputs of expressive features extracting part make the position displacement of face or move to reflect the expression data 48 of emotion data and characteristic.In addition, the operation index signal 50 that expressive features extracting part 24 also has according to operating portion 28 is dwindled image, and and sound output to as video/audio signal 52 together and look audio output part 38.
Utilizing under the situation of view data, expressive features extracting part 24 can be utilized the technology of patent documentation 1 and 5.In addition, utilize in expressive features extracting part 24 under the situation of voice data, can carry out threshold process to sound waveform and then generate and export the face or the eyebrow that raises up of characteristic portion that opens as expression more than the value at certain, and in certain threshold value with next generation and export the expression data of closed face or sagging eyebrow.
Operating portion 28 will be supplied with not shown control part or characteristic management department 30 by the selected operation index signal 50 of user's operation.
The synthetic portion 26 of expression has following function: the characteristic 48 of input expression expression and as the characteristic image 58 of selected basic model generates the characteristic point that makes the characteristic image with basic model image and moves and make it be reflected in image in the expression.This function is realized with application software.In this image, implement texture mapping (texture mapping) according to the characteristic image supplied with and wait and generate 2 and tie up image or 3 d images.The synthetic portion 26 of expression will reflect user (caller) expression characteristic image data 54 and for example make it be synchronized with moving of emotion or face so voice data 55 to be outputed to formal argument portion 27.The synthetic portion 26 of expression just can utilize voice data and the view data characteristic with synthetic expression expression simultaneously like this, perhaps utilizes a certain side in image and the sound.
In addition, the synthetic portion 26 of expression will output to characteristic image data 54 and voice data 55 represented video/audios 64 and look audio output part 38.But this is to be used for the sender to confirm sending image, also can be not to the affirmation image of looking audio output part 38.The synthetic portion 26 of expression can also be provided with the voice data transformation component that the tonequality that makes the voice data of being supplied with etc. is out of shape.
Turn back to Fig. 1, the output selection portion 56 in the present embodiment have the view data of being supplied with 42 and 60 and voice data 46 and 62 among, the function of selecting a set of image data 72 and voice data 74 to export.Output is selected to control according to the selection signal 70 that illustrated control part is never supplied with.In addition, though in Fig. 1 diagram not, the information such as kind that each output selects to be supplied to equipment in the view data of output selection portion 56 and signal also are supplied to looks audio output part 38.Look audio output part 38 and show these information, make a group of will export help to select decision by the operation of operating portion 28.Output selection portion 56 outputs to encoding section 32 with a selected set of image data 72 and voice data 74.
In addition,, be not limited thereto, can also import from the signal of a plurality of equipment and select though the output selection portion 56 of present embodiment is only handled 2 kinds or 2 groups of signals from identical device.In addition, wish that output selection portion 56 has the function of the form of the regulation of being transformed into.Particularly, divide into this function effectively for well at view data 72 and voice data 74 for the situation of view data 42 and voice data 46.This carries out being adjusted into of regulations such as form with a plurality of view data and voice data, for example view data 42 and the voice data of being imported 46 as one group except view data 60 and voice data 62.
Receiving and transmitting part 34 has the video/audio 80 that sends own coding portion 32, receives the function of the video/audio of sending here from the communication terminal 14 of communication object, also has the function that the video/audio 82 that will receive outputs to lsb decoder 36.In addition, such as described above, receiving and transmitting part 34 also has the function of carrying out the data communication between characteristic management department 30 and the characteristic administrative center 16.
Do not have display part and loud speaker though there is diagram to look audio output part 38.On the monitor picture 86 of display part, be provided with the guide look viewing area 88 of characteristic image data and the guide look viewing area 90 of background data as shown in Figure 3.Display image during guide look shows moves it to switch by the cursor 92 shown in the manipulation hachure respectively and shows.Selection is that the characteristic image or the background data that to the image field shown in the position of cursor 92, promptly comprise the intersection point when move up and down are selected respectively.
Here, on showing, selected data show to understand it selected by means of thick frames images.Selected image is the synthetic demonstration for affirmation on preview screen 94.The user presses decision button 96 and selects decision satisfying under this situation of confirming to show.In addition, on monitor picture 86, also show the text data of sending here from the other side's communication terminal 14.The selection picture of characteristic image is not limited to Fig. 3, can also be preview screen 94 not to be set or decision button 96 makes it show various demonstration system of selection like this.
Turn back to Fig. 1 once more, characteristic administrative center 16 for example is connected to internet 18 via communication line 98.Characteristic administrative center 16 comprises content management server 100 and memory 102 as shown in Figure 4.Preserve after in memory 102, making the content-data data base system.Content-data is 2 dimensions or 3 performance datas of tieing up that never illustrated seller supplied with.Content management server 100 is connected with memory 102, carries out writing/reading of content-data via holding wire 104.Content management server 100 also has the user profile (log in history etc.) of operating characteristic data and the management function of vendor information.Here, the seller is meant and creates the practitioner that performance data is supplied with.The use of 16 pairs of authentications at characteristic management department 30 of characteristic administrative center, performance data and background data is maked a promise could and use the time management of binding hours etc. and pay imformation etc. to manage
In the present embodiment, utilize the function of driver 23 and output selection portion 56 by the communication terminal in the image communication system 10 12, just can make while doing one's utmost to check cost and be provided at recreational upward excellent images communication, and the form of utilizing of platform be changed without the user.Provide by this and just can help popularizing of this Image Communication.In addition, the ease of use for the user is improved.
Then, other embodiment that just have been suitable in the communication terminal of image communication system 10 of information terminal device of the present invention disclose.The later identical mark of used reference marker among additional and the embodiment formerly on common inscape is so and avoid the repetition omission explanation of miscellaneous explanation.
The image communication system 10 of present embodiment is characterized in that abrogating among the embodiment formerly as shown in Figure 5 as the characteristic administrative center 16 that inscape comprised, and makes communication terminal 12 bear the function this point of being abrogated.That is, communication terminal 12 is followed this to abrogate and memory 106 is set in characteristic management department 30 in advance, and then data input part 108 also is set.Data input part 108 has reads the performance data that for example write down and the function of background data in CD that is bought (Compact Disc) or DVD external memory medias such as (Digital Versatile Disc), and the data 110 of being read are outputed to characteristic management department 30.Memory 106 can also be preserved the data of being supplied with 110 and carry out overwrite.Thus, characteristic management department 30 just can increase the variation of data.Like this owing to utilize the recording medium bought, so characteristic management department 30 does not authenticate, uses the management of promise, time and charge etc. to solve.Move identical with previous embodiment.
Like this owing to do not need communication with characteristic administrative center 15, so the function of the characteristic management department 30 among the previous embodiment is just simplified, can not realize the function of image communication system 10 yet and utilize even if do not carry out processing that user with characteristic administrative center 16 trouble such as logins as the user.
Then, other embodiment that just have been suitable in the communication terminal of image communication system 10 of information terminal device of the present invention disclose.As shown in Figure 6, the communication terminal 12 in the image communication system 10 of present embodiment is characterised in that and increases text input part 112 this point newly on the formation of Fig. 1.Text input part 112 has from input unit input text datas 114 such as keyboards, and output is as the function of the text data 114 of text and TTS (the Text To Speech) mapping function that is transformed into voice data.Text input part 112 outputs to expressive features extracting part 24 with text data and/or voice data 116.Which combination utilization in output text data and/or voice data 116 and view data 42 and the voice data 46 is selected to the operation of operating portion 28.
Expressive features extracting part 24 utilizes text data 116 to extract expression data 48, and expression data 48 and the voice data that extracts outputed to the synthetic handling part 54 of expression together.In the case, for example as 1,2,3 ... wait numeral like that, be changed to setting, and export special expression data according to the special expression data of predetermined special text data output.
In addition, expressive features extracting part 24 utilizes voice data 116 to extract expression data 48, and expression data 48 and the voice data that extracts outputed to the synthetic handling part 54 of expression together.In the case, expressive features extracting part 24 makes expression data 48 be varied to the shape of the mouth of voice data 116 expressions.For example if import " あ ", then output is made as the expression data 48 of the degree of lip-rounding shape of " あ ".In addition, with the voice data of output not the voice data 46 imported but voice data 116 that output is synthesized.
Two kinds of ways of above-mentioned this both can independent like this utilization also can make up utilization.In the case, for example combination special text data are imported on common text data, usually text data output only makes the expression data of degree of lip-rounding shape variation, if the special text data are transfused to, the special expression data that set in advance are output as, degree of lip-rounding shape expression data sync is exported in special expression data.
And then expressive features extracting part 24 can also input image datas 42, voice data 46 and text data 116, and these data are made 48 outputs of expression data after synthetic.The synthetic method of data can be utilized various methods, but for example imported at the same time under the situation of view data 42, voice data 46 and text data 116, on the expression data of degree of lip-rounding shape, utilize text data 116 with regard to espressiove feature extraction portion 24, beyond degree of lip-rounding shape, then be used to expression data, on the expression data pick-up, do not utilize methods such as voice data 46 from view data 42.Can also make the TTS function setting that is arranged at text input part 112 in expressive features extracting part 24.
By increasing text input part 112 newly, import the processing of changing expression feature extraction portion 24 like this, just can increase the variation of communication, and can realize recreational higher Image Communication corresponding to this.
Other embodiment that have been suitable for that in the communication terminal of image communication system 10 of information terminal device of the present invention disclose.As shown in Figure 7, the communication terminal 12 in the image communication system 10 of present embodiment is characterised in that and has increased basic facial expression generating unit 118 this point newly on the formation of Fig. 1.
Basic facial expression generating unit 118 is a benchmark with amimia state usually, has the function of creating basic facial expression data 122 with respect to this benchmark based on the basic facial expression signal of being supplied with from operating portion 28 120.Basic facial expression generating unit 118 outputs to the synthetic handling part 54 of expression with the basic facial expression data of being created 122.Basic facial expression data 122 are to represent the expression data of basic facial expression such as happiness, anger, grief and joy with the data mode of expression data 48 same form of expressive features extracting part 24 output.
The synthetic handling part 54 of expression is based on to from the expression data 48 of expressive features extracting part 24 outputs with carried out synthetic back result from the basic facial expression data 122 of basic facial expression generating unit 118 outputs and make 58 distortion of characteristic image data. and the synthetic of expression data can be adopted various ways, but being not limited to the method. the synthetic handling part 54 of expression is that the synthetic expression data of sending here from expressive features extracting part 24 48. of benchmark are by handling like this with the basic facial expression data 122 of smile face for example, even if actual input picture is not the smile face, Yi Bian make it be deformed into the actual degree of lip-rounding shape that changes on one side characteristic image also can be kept the smile face all the time.
Thus, the expressive ability of characteristic is improved, further increases recreational.In addition, as constituting the side also can on other initial embodiment, make up at least among other embodiment of the 2nd and the 3rd.
Other embodiment that have been suitable for that in the communication terminal of image communication system 10 of information terminal device of the present invention disclose., in terms of content as the information additional license information of expression place or time etc. the monobasic management method of content and license information is represented at its later embodiment.In addition, also in back segment, represent with corresponding charging method such as the restriction that causes because of place or time etc.Communication terminal 12 in the image communication system 10 of present embodiment is different on the inscape this point that comprises Fig. 8 in the characteristic management department 30 among the formation of Fig. 1.The characteristic management department 30 of present embodiment obtains according to the content from the user and asks to communicate by letter with characteristic administrative center 16 on one side and manage, according to from user's content export the license information that in request and content added export in store content on one side.
Location information management portion 128 has the function of the current location that obtains the communication terminal 12 that utilizes GPS (Global PositioningSystem) etc., find the solution current location with latitude, longitude etc., resulting positional information 134 is outputed to content control portion 126.
The action of communication between characteristic management department 30 and the characteristic administrative center 16 by separately discloses simply with that.Accept content by the input control part 124 of characteristic management department 30 shown in Figure 9 and obtain request, and be notified to content control portion 126.(step S10).After accepting this notice, 126 location information management departments 128 of content control portion send positional information and obtain order 134 (step S12).Location information management portion 128 utilizes GPS etc., obtains the positional information 134 of communication terminal 12, outputs to content control portion 126 (step S12).
Follow content control portion 126, the cross-check information 76 that content is obtained request 132 and positional information 134 sends to characteristic administrative center 16 (step S14).
Reception is from the information (step S16) of characteristic administrative center 16. and whether the information of receiving is permitted distribution verify (step S18). if the distribution of content is allowed (being), then content control portion 126 offers memory 106 with content and enters the processing (to step S20) of preserving. in addition, if the distribution of content be rejected (deny) then content control portion 126 the distribution refusal information of content is outputed to output control part 130. distribute refusal information 78 and offer via output control part 130 and look audio output part 38, and be presented on the display part of looking audio output part 38. afterwards, end process.
In step S20, accept above-mentioned normal content distribution, memory 106 is preserved the content of being supplied with (step S20).Secondly content control portion 126 reads the content of being preserved from memory 106 according to content output request as performance data and background data 58, and via output control part 130 content 58 of being regenerated is outputed to the synthetic handling part 54 (step S22) of expression.By means of this output, the synthetic handling part 54 of expression just generates the characteristic image of band expression and background.Afterwards, end process.
The action of characteristic administrative center 16 shown in Figure 4 then is described.Characteristic administrative center 16 receives the content from communication terminal 12 that is provided for content management server 100 and obtains request 76 (step S24) as shown in figure 10.
The content that then will meet the request of obtaining is taken out (step S26) from memory 102.The content 136 of being read as shown in figure 11, comprises header region 138 and content body 140.In header region 138, include license distribution information 142.License distribution information 142 is with the information 144 of expression license distribution region, particularly indicating that the latitude longitude shown in 146 is the center, as the circle regional 148 that surrounds with radius r or represent according to the polygon cut zone 150 of latitude longitude.Particularly, the latter's polygon cut zone 150 is to represent with the coincidence of a plurality of latitude longitude institute area surrounded.Coincidence can give with logical operation representing it is to be in each field or outer inequality condition, and makes data hold it.In addition, just become more easy if prepare then to manage according to the table that the latitude longitude is transformed into state or region number, the no problem etc. of having of the copyright in the different states region also can be resolved.For example can also attaching terminal make number etc. as the terminal intrinsic information, the perhaps additional content-data etc. of using has carried out the terminal intrinsic information of encrypting, and perhaps prevents to carry out in end side the bootlegging or the such structure of content tampering of content.
Then judge whether to distribute (step S28).Content management server 100 will be represented that the positional information 76 in the place that communication terminal 12 exists is checked with license distribution information 142 and confirm to distribute (step S28).If positional information 76 expression is in certain place of license distribution (being), then enter the transmission processing (to step S30) that content management server 100 sends to content communication terminal 12.In addition, for being in the place that disapproves distribution or being (denying) under the situation of objectionable content, enter the transmission processing (to step S32) that the refusal information of distribution is sent to communication terminal 12 in positional information 76.
Then content is sent to communication terminal 12 (step S30).Self-evident in this transmission stage, allowed at authentication, rights of using and the charge etc. of communication terminal 12.In step S32, send the refusal information of distribution.After these information send, tenth skill.
By making its action like this, just can control and the distribution of positional information 76 content corresponding of communication terminal 12, and can make the management simplification and license information and content-data need not be cut apart and manage.In addition,, same content must be managed with a plurality of so far, but also can carry out the monobasic management by license distribution information 142 is set a plurality of permissions region.And then if prepare to be transformed into according to the latitude longitude table of state or region number, then on management becomes more easy basis, what can also solve copyright in the different states region has a no problem etc.
Other embodiment that have been suitable for that in the communication terminal of image communication system 10 of information terminal device of the present invention disclose.In the present embodiment, the characteristic administrative center 16 in the image communication system 10, inscape shown in Figure 4 with new memory 152 serve as a mark the collection usefulness database append.Memory 152 has the label sets database (DB) of the identifying information, event information or the positional information history that keep each communication terminal, according to the positional information history that is kept in this database the content 154 of preserving is distributed to content management server 100.Memory 152 is managed by content management server 100.
Obtain request according to the content from communication terminal 12 obtains content by content DB to content management server 100 like that as described above, the license distribution information that could distribute that is comprised in the positional information history of the positional information 76 of communication terminal 12 and the label sets DB of memory 152 and the content is checked, if permit then distributing contents.The license distribution condition can also obtain time etc. with the order of position history or positional information and make up.
The action of just different with previous embodiment present embodiments describes.Difference is the action of characteristic administrative center 16.As shown in figure 12, content management server 100 is obtained request (step S24) from communication terminal 12 received contents, and the positional information with the communication terminal 12 supplied with is kept at (step S34) among the label sets DB simultaneously.Then will there be of the content DB taking-up (step S26) of the content of request from memory 102.
In the present embodiment, the position history that is comprised in the license distribution information 142 with communication terminal 12 is as shown in figure 13 taken out (step S36) from label sets DB.In license distribution information 142 as the position history in the field 156 comprise corresponding to A place, B place, C place ... positional information 146A, 146B, 146C ....And then for example license distribution information 142 is preserved identifying information, event information and positional information history as shown in Figure 14 successively.
Turn back to Figure 12, then content management server 100 location information history and current position information and distribution could information be checked to confirm distributing (step S28).If the time license distribution certain place (being) enter step S30, content is sent to communication terminal 12.In addition, in content being (denys) then to enter step S32 under the situation of forbidding distributing, and the distribution refusal is sent to communication terminal 12.
Thus, characteristic administrative center 16 is with regard to the historical content corresponding distribution of positional information, the positional information of may command and communication terminal 12, up to now license information, event information and content-data are cut apart respectively and handled, must intricately manage, but now just can be only manage, thereby license management is become than easy with contact with the label sets information of content and communication terminal 12.
Other embodiment that have been suitable for that in the communication terminal of image communication system 10 of information terminal device of the present invention disclose.As shown in figure 15, in the present embodiment, increase license management portion 158 newly on the inscape of the characteristic management department 30 in communication terminal shown in Figure 8 12, and be connected respectively to input control part 124, location information management portion 128, memory 106 and output control part 130.Communication terminal 12 has based on the output request with content and is attached to the characteristics that the license distribution information 142 in the content is come the license information of organize content.
In communication terminal 12, the content of input control part 124 accepting to comprise respectively obtains request or content output requested operation index signal 50, and content is obtained request 132 or content output asks 160 to output to content control portion 126 and license management portion 158 respectively.License management portion 158 accepts the content output request of being supplied with 160 and content is exported request 162 to output to location information management portion 128.
Location information management portion 128 obtains the current location information of communication terminal 12 self according to the supply of content output request 162.Location information management portion 128 outputs to license management portion 158 with obtained current location information 162.
Then disclose exporting in the present embodiment simply and ask the ground action corresponding to content.Content output request is meant that the memory 106 of selecting communication terminal 12 keeps the ground content, is confirming that license information makes it be shown in the process of looking audio output part 38.At first, as shown in figure 16, will operate index signal 50 according to the operation of operating portion 28 and input to input control part 124 (step S38).Here, the operation index signal of being supplied with 50 is the content output requests from the user.
Then, will pass to license management portion 158 (step S40) in the content output request 160 that input control part 124 is accepted.In license management portion 158, ask 160 to serve as that (step S42) retrieved to memory 106 in the basis with the content output of getting from input control part 124.Memory 106 will output to license management portion 158 via holding wire 164 according to the license distribution information 142 of being added in resulting content 140 of this result for retrieval and the content.License management portion 158 obtains content 140 and license distribution information 142 by this input.In addition,, in content, adding under the situation of terminal intrinsic information, can judge bootlegging by just checking with the terminal intrinsic information of user's communications terminal installation 12 though do not illustrate.And then, though do not illustrate, but utilize content-data etc. to carry out under the situation of the encryption terminal intrinsic information encrypted adding, in communication terminal 12, for example utilize content 140 and terminal intrinsic information to generate terminal intrinsic information, just can detect bootlegging or distort by encryption terminal intrinsic information that is generated at communication terminal 12 and the encryption terminal intrinsic information that is added in content are checked through encrypting.
Then obtain the current location information 162 (step S12) of communication terminal 12 by means of location information management portion 128 in license management portion 158.Then the license distribution information 142 of 158 pairs of obtained contents of license management portion and obtained current location information 162 compare, to judge whether to permit the use (step S44) on the current location.Current location is included under the situation in the permission to use scope that is recorded in the license distribution information 142 (being), enters to the output of output control part 130 and handles (to step S22).In addition, be under the extraneous situation of permission to use that is recorded in the license distribution information 142 in current location, then enter the notifier processes (to step S46) that can not use intention to output control part 250 contents of announcement.
At step S22, license management portion 158 is from the desirable content of memory 106 regeneration, and content 58 and/or 78 is offered output control part 130 so that it shows.Output control part 130 output contents 58 and/or 78.
License information management department 158 can not use the message of intention to pass to output control part 130 content according to the extraneous judgement of permission to use, and offers and look audio output part 38 with notification message (step S46).
In addition, under the situation that communication terminal 12 uses from characteristic administrative center 16 download contents, communication terminal 12 is obtained request to content and is sent to characteristic administrative center 16 (step S48).Communication terminal 12 comes received content (step S16) by the processing in the characteristic administrative center 16.The content that receives is stored in (step S20) in the memory 106.Afterwards, 106 pairs of desirable contents of memory are read and are regenerated and export like this.
By additional permission to use scope in content like this, and can limit using region etc., just can be not via network services such as license server, the licence of content is managed.In addition, owing in the management of content, also can not keep license server etc., so just can cut down equipment cost or communication cost.
As the utilization of present embodiment,, then can make the increase of a number or promote that effect such as facility utilization is brought into play for example if only create utilizable content in the facility of the regulation in amusement park or library and so on.
Other embodiment that have been suitable for that in the image communication system 10 of information terminal device of the present invention disclose.Present embodiment is characterised in that by utilizing content with the corresponding charge in position that utilizes the license management in the communication terminal 12 or region.The formation of present embodiment is conceived to the difference with as shown in figure 15 image communication system 10, the Fare Collection System that the communication terminal in the image communication system 10 12 is charged shown in Figure 17.
In store content-data is as database in memory 102. and comprise license distribution information 142. as shown in figure 18 in content, license distribution information 142 writes the expression region and also comprises simultaneously with information 144. information 144 of corresponding cost system in certain distance or the scope and particularly represent as the information 144a of expense system exactly. the mark utilization of positional information and the previous identical form of embodiment.
Turn back to Figure 17, charging server 166 has acceptance and obtains request from the usage charges of the content of license management portion 158, conduct interviews and retrieve via the content DB of 168 pairs of memories 102 of holding wire, the usage charges that extract in the content to be added, the function of notifying via holding wire 170, IP network 18 and 172 pairs of license management portions 158 of holding wire.
The license management portion 158 of characteristic management department 30 will import from importing control part 124 as content output request 160 from the operation index signal 50 of input control part 124, and content output request 162 is outputed to location information management portion 128.License management portion 158 obtains the current location information 162 of communication terminal 12 from location information management portion 128 according to the supply of content output request 160.The positional information 164 of the license distribution information 142 that is contained in content that license management portion 158 obtains content to be preserved from memory 106.158 pairs of current location informations of license management portion 162 compare with the positional information 164 of license distribution information 142, to judge the permission to use of content.License management portion 158 inquires via holding wire 172, IP network 18 and 170 pairs of charging servers 166 of holding wire under the situation that is not the permission to use position.License management portion 158 is from the use location corresponding usage charges information of charging server 166 acceptance with content, and the use of licensed content by the payment usage charges outputs to output control part 130 so that output control is managed control with content.License management portion 158 manages control to output control part 130 under the situation of permission to use, and carries out the output of the content supplied with according to this control.
Characteristic administrative center 16 obtains request according to the content from communication terminal 12 content is distributed, and points out pay imformation by charging server 166 according to the positional information of communication terminal 12.Communication terminal 12 is to utilize content after the corresponding equivalence in basic Zhi Fuyu place with the pay imformation that is provided.
The then action of open present embodiment in Figure 19.License management portion 158 carries out the process of the judgement (step S38～44) of the accepting of content output request～permission to use, (is) to enter content output to handle (to step S22) being judged as under the situation of permission to use.In addition, (deny) to enter inquiry (to step S50) under the situation of the extraneous judgement of permission to use to charging server 166.
The user sees shown expense, and operating portion 28 is operated, and selects to utilize (step S54).Promising to undertake under the situation about utilizing (being), entering the notifier processes (to step S56) of payment.In addition, (denying) enters the notifier processes (to step S58) that refusal utilizes under the situation that refusal utilizes.
The notifier processes of payment is to utilize the information of intention and information (step S56) that payment of fees is finished from the expression of 12 pairs of charging servers of communication terminal 166 notice.In addition, accept this and finish after communication terminal 12 read desirable content (the step S22: output) that regenerates.After this, finishing content output request handles.
On the other hand, the notifier processes of refusal utilization is the information of using from 12 pairs of charging servers of communication terminal, 166 notice expression refusals (step S58).License management portion 158 outputs to the intention of not utilizing content and looks audio output part 38 and show (step S60) after accepting this notice.After this, finishing content output request handles.
Like this, while by making effective utilization of the content of creating at the content provider obtain promoting according to the utilization restriction that utilizes the place to charge just can to relax content, and also can regain the funds of following content to provide rightly from usage charges.
Secondly, other embodiment that just have been suitable in the communication terminal of image communication system 10 of information terminal device of the present invention disclose.The communication terminal 12 of present embodiment is characterised in that and is not according to positional information but comes the administration of licences this point according to useful life or the time information of time on date.
Originally execute being formed in of example and make on the basis that above-mentioned feature is achieved, possess time information management department 174 shown in Figure 20, be connected to license management portion 158 via holding wire 162a.Formation and connection beyond characteristic management department 30 its inscapes of Figure 20, the time information management department 174 are identical with the formation of Figure 15.Be that position Information Management Department 128 usefulness moment Information Management Department 174 is replaced.
In addition, only handle time information, be not limited thereto information, though there is not diagram self-evidently can also comprise the inscape that location information is handled though be made as in the present embodiment.
Time information management department 174 has and obtains current date clock function constantly, the Gregorian calendar, date, Hour Minute Second etc. is carried out instrumentation, and output to license management portion 158.
Then the action to present embodiment discloses.The course of action of present embodiment is such as shown in figure 22, and obtain (step S42) up to license distribution information 142 in above-mentioned formation handles with identical process before.After this, in license management portion 158, obtain the current time information 162a (step S62) of communication terminal 12 from time information management department 174.
Then in license management portion 158, the license distribution information 142 of obtained content compared to judge with obtained current time information 162a and could use (step S64).Under the situation in current time is the useful life of license distribution information 142 (being), make content output to output control part 130, enter the processing (to step S22) of regenerating and showing with the content of looking 130 outputs of 38 pairs of output control parts of audio output part.In addition, represent to use the message of intention to pass to output control part 130 content under the situation that useful life arrived (denying), enter the notifier processes (to step S46) that the user is notified at current time.(step S22, S46) process with Figure 16 makes its action in this each later step.
Like this by additional useful life in content, can just can not managing to limiting between the operating period via the licence of network services such as license server to content.In addition, even if owing in the management of content, do not keep license server etc. also can solve, so just can cut down equipment cost or communication cost.As concrete example, just can easily carry out for the on probation of user's content or only during certain during the propaganda activity, make content utilized.
Other embodiment that have been suitable for the communication terminal in the image communication system 10 of information terminal device of the present invention with that disclose.Present embodiment is characterised in that: be not according to positional information equally with previous embodiment but come administration of licences according to useful life or the time information of time on date, utilize the time limit also can continue to utilize the content this point by charge even for example crossed.
The characteristic management department 30 of the communication terminal 12 in the image communication system 10 of Figure 23, formation and connection beyond inscape, the time information management department 174 are identical with the formation of Figure 17. and be that position Information Management Department 128 usefulness moment Information Management Department 174 is replaced.
In addition, only handle time information, be not limited thereto information, though there is not diagram self-evidently can also comprise the inscape that location information is handled though be made as in the present embodiment.
In store content-data is as database in memory 102.In content, comprise license distribution information 142.As shown in figure 24, license distribution information 142 writes the information 144c of expression at the expense system of useful life.Information 144 is particularly represented as the information 144c of expense system exactly.The form that the mark utilization of positional information is identical with previous embodiment.
Turn back to Figure 23, charging server 166 has acceptance and obtains request from the usage charges of the content of license management portion 158, conduct interviews and retrieve via the content DB of 168 pairs of memories 102 of holding wire, the usage charges that extract in the content to be added, the function of notifying via holding wire 170, IP network 18 and 172 pairs of license management portions 158 of holding wire.
Then the action to present embodiment discloses.The course of action of present embodiment is such as shown in figure 25, and obtain (step S42) up to license distribution information in above-mentioned formation handles with identical process before.After this, in license management portion 158, obtain the current time information (step S62) of communication terminal 12 from time information management department 174.
Then in license management portion 158, the license distribution information 142 of obtained content compared to judge with obtained current time information 162a and could use (step S64).(being) enters and makes content output to output control part 130 under the situation in current time is the useful life of license distribution information, the processing (to step S22) of regenerating and showing in the content of looking 130 outputs of 38 pairs of output control parts of audio output part.In addition, represent to enter to charging server 166 and inquire (to step S50) under the situation that useful life arrived (denying) at current time.
User prompt in these usage charges carries out above-mentioned processing later on, promptly carries out step S56, S58, S60 and S22 respectively.
By upgrade expanded service life like this by means of charging, after having carried out content grade on probation, formally offer the user as commercial version just to become easily, just can when want to use content, provide by the part of wanting to use and charge accordingly for the user.In addition, the establishment funds of regaining content effectively by usage charges just become possibility.
Secondly other embodiment that just have been suitable in the communication terminal of image communication system 10 of information terminal device of the present invention disclose. as shown in figure 26, comprise in the communication terminal 12 of present embodiment: information generating unit 176, notice control part 178, object preservation portion 180 and tracking handling part 182. self-evident communication terminals 12 consist essentially of the inscape of Fig. 1, and characteristic management department 30 and characteristic administrative center 16 can also comprise above-mentioned inscape.
Known in such Image Communication, user's camera is taken pictures each other communication quality is brought influence.In the past, in this field, taked Several Methods.In the 1st method, there is demonstration user self to be carried out the method for the image of shooting.The method adopts to be shown as on the picture-in-picture mode of wicket or the zone after picture is separated into a plurality of zones in the picture that is showing from the image of the other side's side carries out the picture displayed partitioning scheme.
As the 2nd method following method is arranged, promptly, user interface with Graphics Communications Terminal device of patent documentation 6, under the situation outside the camera coverage of user at camera, by changing show state later on and show the other side's side image that receives being carried out various operations, just can not show local dot image ground judges whether the user is positioned at the camera coverage of camera, and notify the user to break away from from the camera coverage of camera.
But, in the demonstration that cited dual mode carries out in by the 1st above-mentioned method, have following problem: thus since only the image of explicit user self be difficult to confirm the photography situation, the adjustment of photography situation also has only the user to watch actual shown image to judge.
In addition, in the 2nd kind of method, there is following problem: under the situation in the user is in camera coverage, can not confirm it is in what kind of position etc.Though the method can be adjusted the position to the user in camera coverage, but because explicit user image and the notice of only accepting the other side's image shows not, so just can not confirm for the lightness around for example, other photography situations such as distance between camera user.
Turn back to Figure 27, notice control part 178 has the function that generates the information 186 that should notify to the user from the information supplied with 186.Notice control part 178 outputs to announcement information 188 looks audio output part 38.Announcement information 188 is number information corresponding of the lamp that makes it to light with stage corresponding to the information of being supplied with 186.Notice control part 178 for example make as shown in figure 28 information 186 and announcement information 188 corresponding to extraction be processed into power and lamp light number.
Looking audio output part 38 shows the state that extracts processing according to the announcement information of being supplied with 188.Preferably for example such as shown in figure 29, the character string display portion 190 of the character string that shows result of determination " success " or " failure " and so on is set or shows except the mark display part 192 of the shape of mark " zero " or " * " display graphics etc. and according to the stage display part 194 that their size is reported situation.The change color of character string or figure is reported.In addition, in addition, can also it be reported to the user by voice output.For example be not only the output beeping sound, can also read aloud " success " or " failure ", " zero " or characters such as " * " or character string and export, character or the character string that also can read aloud in advance to be logined be exported.Voice output wishes at random to export/setting of non-output.
The action of the Image Communication in the communication terminal shown in Figure 26 12 is described simply.The user utilizes communication terminal 12 opposite hole patterns to look like to photograph.As shown in figure 30, communication terminal 12 is obtained face image (step S66) via image input part 20.The view data 42 of face image is outputed to expressive features extracting part 24.
Secondly by expressive features extracting part 24 judge whether to carry out image at communication object sends (step S68) though. not have this judgement of diagram to carry out corresponding to control signal. control signal for example generates based on the operation index signal of being supplied with from operating portion 28 according to the operation of being undertaken by the user 50. (be) to output to the processing (to step S70) that expressive features extracting part 24 enters extraction under the situation that does not send characteristic 48. in addition, (deny) that (to step S72) handled in the transmission that enters view data 42 under situation about sending.
In step S70, extract the feature of expression from view data 42.Next judges whether formation characteristic image (step S74).This judgement is also carried out according to operation.(be) to enter image and generate processing (to step S76) under the situation of formation characteristic image.In addition, (to step S78) handled in the generation that (denying) enters information under the situation of formation characteristic image not.
Generate in the processing at image, by the synthetic handling part 54 formation characteristic images (step S76) of expression.The processing here serves as that the basis generates information 186 with the characteristic of being supplied with 48 also.Become at the characteristic image that makes the band background after expression obtains reflecting on the content 58 of characteristic image and background.
In the generation of information is handled, generate extraction by information generating unit 176 and be processed into power etc., and output to notice control part 178.Though the generation treatment region of information is divided into process and represents, also can under the situation of the characteristic image that generates the band background, carry out.Thus, the user just can confirm the photography situation.
Then send the characteristic image or the face view data (step S72) of the band expression that is generated.Looking the image (step S80) that demonstration is sent on the audio output part 38.After this shows, end process.
On the other hand, generate announcement information 188, and output to and look audio output part 38 (step S82) corresponding to the information of being supplied with 186 at notice control part 178.How the user for example obtains next image and carries out correspondence according to being presented at the notice of looking on the audio output part 38.This notice not only can be according to the light on and off number of lamp, can also be processed into power according to extraction and the output of sound is increased by stages or reduce.After this shows, end process.
By the announcement information 188 corresponding to information 186 is shown, to user notification photography situation, user self just can easily confirm the photography situation when utilizing, and can reduce user's burden.In addition,, also can confirm, so just can make convenience very high the photography situation even if the user takes rightly with camera.
Turn back to Figure 26, disclose with regard to object preservation portion 180 and tracking handling part 182.The image processing apparatus of patent documentation 7 and method thereof have proposed according to the variation of the size of the face in the face, position the big or small position as the object of imaginary object to be changed, and be synthetic with the image that moves of following the tracks of face.Utilize this image synthetic, then as shown in figure 31,, just obtain Figure 31 (b) if on the face image of being imported of Figure 31 (a), make it put on imaginary cap.If with the face image along move left and right then as Figure 31 (c) shown in, this cap just just in time is worn on the head.So image is synthetic just can use as the means of communication at mobile image.
But, because above-mentioned its object of technology is the figures of 2 dimensions, so even if turn round later wait to face towards changing, the object that synthesize towards also not changing the composograph that does not just become nature very and dullness.The patent documentation 7 of having in this is meanss of communication that function lacks.
In the present embodiment, utilize object preservation portion 180 and follow the tracks of the composograph that 182 couples of users of handling part provide the object of nature at this.Provide by this, just can help the user is enlarged.
This action is disclosed.In expressive features extracting part 24, at least face analyzed establishment characteristic 48 towards the position at each position of, face such as eye mouth etc. etc.Characteristic 48 comprises the deflection of object.This deflection uses the ranks at 3 dimension figures general employed 4 * 4 to be expressed as.If use it then shown in figure 32, just can with the eye of the face image of the benchmark shown in (a), nose, mouthful the position compare, all parallel mobile, (expansions /) (c) of performance (b) are dwindled and rotation (d).Characteristic 48 comprises the information which kind of is deformed into of object.
The face image that will become benchmark in expressive features extracting part 24 and each position of the face image of being imported, i.e. offset in eye, nose, the mouth are analyzed as the deflection of object.This analysis can also be carried out in following the tracks of handling part 182.Analytically use patent documentation 5 grades of the position at each position in face are for well.In Figure 32, be judged as the amount that the parallel mobile eye of face image, nose, mouth have as one man moved, narrow the dwindling that distribute, so according to based on the nose of two center or mouthful skew turning round of face turned one's head to measure analyze as rotation amount.
More particularly, if the face image shown in Figure 33 (a) is supplied to, then expressive features extracting part 24 is analyzed the object deflection shown in Figure 33 (b) like that.Then as Object Selection silk ribbon (with reference to Figure 33 (c)).Generate and make parallel amount of movement, enlarge reduction volume and rotation amount and be reflected to tracking data 198 in the object, shown in Figure 33 (d) respectively following the tracks of handling part 182.Synthesize portion 26 by creating the composograph shown in Figure 33 (e) in expression to its object that is deformed into appropriate shape and face image or selected characteristic image are synthesized.
Here, in the composograph number of employed object be what all it doesn't matter.Be the plate polygon as 2D model if establish object, then processing speed that can also image is synthetic improves.In the distortion of object can also by dwindle make object very the little object that makes does not show.
Establishment considered the user face towards the object of 3 dimension shapes, by synthesizing the composograph that just can create nature with characteristic image etc.In addition, if use the such object of face that covers the face image of being imported, while then the face that can also make background intactly hide oneself communicates.
Formation after the image communication system 10 of Figure 34～Figure 36 is illustrated on the formation of Fig. 5～Fig. 7 configuration prompt information generating unit 176, notice control part 178, object preservation portion 180 respectively and follows the tracks of handling part 182.Constitute according to this, to obtain widely the user good and follow the additional recreational higher image communication system 10 of object to ease of use with regard to the user simultaneously with regard to providing.
Then only being conceived to notification list according to prompting as other embodiment in the communication terminal of the image communication system 10 that has been suitable for information terminal device of the present invention shows and describes.The communication terminal 12 of present embodiment sets memory 200 as shown in figure 37 respectively on the formation of Figure 34～Figure 36.The memory 200 in store reminder-data storehouses that the correspondence of announcement information is managed.Memory 200 couples together via holding wire 202 and notice control part 178.
In addition, to information generating unit 176 not only supply characteristic data 48 also supply with view data 42.Information generating unit 176 as shown in figure 38, comprises extracted data detection unit 204, image light and shade detection unit 206, face range estimation portion 208, opens and closes detection unit 210, face angle detection unit 212 and synthetic determination portion 214.Extracted data detection unit 204, face range estimation portion 208, switching detection unit 210 and face angle detection unit 212 are imported the characteristic of being supplied with 48 respectively, as result of determination data 216 are outputed to synthetic determination portion 212.In addition, image light and shade detection unit 206, the view data 42 that input is supplied with outputs to synthetic determination portion 212 to data 216 as result of determination.
Extracted data detection unit 204, corresponding to information detection unit 184. image light and shade detection units 206 by being suitable for by means of to extract the judgement that results are benchmark from captured view data 42 at the histogrammic calculating of arbitrary region, perhaps by means of the concentration value to arbitrary region carry out after mutually in addition pixel count is divided by various gimmicks such as judgement realize that the light and shade of image judges. judge that by this user just can be in the light and shade of holding appropriate photography environment in advance.
Face range estimation portion 206 can be suitable for the judgement by means of the distance of a plurality of combinations freely of the left eye that is comprised and right eye, a left side mouthful end and right mouthful of end etc. in extracting result, perhaps by means of being that the various gimmicks such as judgement of size in zone of the face of benchmark are judged user's appropriate face distance to extract result.Judge that by this user just can hold and the appropriate distance of image input part 20 in advance.
Open and close detection unit 208 by be suitable for from as the characteristic 48 that extracts result and captured view data 42 based on the judgement of the change of the concentration value in the eye zone, perhaps just can carry out the judgement that left eye, right eye have or not switching separately by means of the various gimmicks such as judgement of template matches.Judge by this, for think that the situation of the extraction that can't stably carry out the face characteristic amount just can inform to the user so that the extraction precision of face raising and carry out appropriate communication by the information of eyes being closed and so in advance.Can also on switching detection unit 210, connect in the case with input image data 42.
Face angle detection unit 212 by by means of the left eye of characteristic 48 and right eye respectively with the calculating of the distance of the inner eye corner and the tail of the eye and left end with the distance of mouth right-hand member, left eye and right eye respectively with the calculating of the distance at the distance of upper eyelid and lower eyelid and glabella and mouthful center, and link left eye and right eye pupil straight line and level gradient and link mouthful left end and the judgement of the straight line of mouthful right-hand member and the calculating at the inclination angle of level, perhaps just can the opposite orifice angle judge by means of the suitable of various gimmicks such as judgement of gradient that with the extraction result is the face zone of benchmark.Judge by this, just can be by carrying out appropriate communication informing to the user in advance.
Synthetic determination 214 has the function of comprehensive above-mentioned result of determination 216, generates information 186.Synthetic determination portion 214 will notify control part 178 to output to information 186.
Action with regard to present embodiment describes simply.As shown in figure 40, course of action is provided with between step S76 shown in Figure 30 and step S82 based on information 186 and uses the prompting DB of memory 200 to generate the step (step S84) of announcement information 188.
By using prompting DB to generate announcement information 188 and it being shown, the user is carried out appropriate guidance, user self situation of just can detail knowledge photographing in the utilization of the image communication system of band expression, and user's burden is significantly reduced.In addition, even if so owing under the situation of inappropriate photography situation be not only notify its reason under the situation of photographing rightly, also can hold photography situation convenience very high.
Then only being conceived to notice according to prompting as other embodiment in the communication terminal of the image communication system 10 that has been suitable for information terminal device of the present invention shows and describes.The communication terminal 12 of present embodiment utilizes inscape shown below respectively on the formation of Figure 34～Figure 36.The 1st, on communication terminal 14, dispose receiving and transmitting part 218, notice control part 220, memory 222 as shown in figure 41 at least and look audio output part 224 via the communication object destination of IP network 18.Receiving and transmitting part 218 in the communication terminal 14, notice control part 220 and look audio output part 224 have with communication terminal 12 in used receiving and transmitting part 34, notice control part 178 and look audio output part 38 identical functions.
Receiving and transmitting part 218 will offer notice control part 220 from the information 226 that communication terminal 12 receives.The prompting DB visit of 220 pairs of memories 222 of notice control part obtains qualified announcement information 228 corresponding to the announcement information of information 226, the announcement information 230 that is generated is outputed to look audio output part 224.
The action of communication terminal 12 as shown in figure 42.At first, obtain captured image (step S66).Next judges whether to send (step S68).All extract expressive features as characteristic 48 (step S70) no matter whether send.(deny) to generate information 186 (step S78) under situation about sending based on the characteristic 48 that extracts.Together information 186 is sent to communication terminal 14 (step S86) with the view data 42 that will send.After this sends, looking the demonstration image (step S80) identical on the audio output part 38 with the image that is sent.Afterwards, (to step S84) handled in the generation that enters announcement information.
On the other hand, judgement could send, under situation about not sending immediately (being), and generating feature data 48 (step S70).Then judge whether formation characteristic image (step S74).(be) to enter characteristic image to generate processing (to step S76) under the situation of formation characteristic image.In addition, (to step S78) handled in the generation that (denying) enters information under the situation of formation characteristic image not.
The generation of characteristic image is handled and is made the selected characteristic image of user hold expression, and selected background and the composograph that is synthesized are generated (step S76) as characteristic image.Processing herein serves as that the basis generates information 186 with the characteristic of being supplied with 48 also.Characteristic 48 not only is supplied to the synthetic handling part 54 of expression and also is supplied to information generating unit 176.Serves as that the basis generates information 186 in information generating unit 176 with characteristic 48 and view data 42.
Also not in the judgement of formation characteristic image (denying), generate information 186 as described above.The information 186 that is generated sends to communication terminal 14 (step S92) from notice control part 178 via receiving and transmitting part 34.
Behind step S80, S90 or S92, generate announcement information 188 based on the information of being supplied with 186 at notice control part 178.The announcement information 188 that is generated is output to and looks audio output part 38 (step S84).Make the announcement information of being supplied with 188 be presented at not shown display part (step S82) looking audio output part 38, its processing of moving back communication terminal 12 is finished.
In communication terminal 14, as shown in figure 43, receive information, image, characteristic image and information (step S94) from communication terminal 12.Among the information that receives, with information 226, output to and notify control part 220.Secondly generate announcement information (step S84) corresponding to information 226.The generation of announcement information 230 is looked audio output part 224 by reading and output to from the prompting DB of memory 222 corresponding to the announcement information of information 226 from the visit of notice control part 220.
Look audio output part 224, show the announcement information of being supplied with 230 (step S82).
In the Image Communication of having used characteristic image, even if by the communication object destination is not for example sent from the photographs of camera only the transfer characteristic image also can notify information 226, the user of communication object destination just can confirm the existence of communication object thus, thereby can eliminate the whether actual uneasiness sense that communication object arranged and can concentrate on communication.
Then only being conceived to notice according to prompting as other embodiment in the communication terminal of the image communication system 10 that has been suitable for information terminal device of the present invention shows and describes.Communication terminal 12 makes information 186 input to the synthetic portion 26 of expression as shown in figure 44 like that in the present embodiment, and newly is provided with memory 232.Memory 232 is preserved model generated data storehouse (DB).The synthetic DB of model is meant the database that information 186 and model composite signal are mapped and preserve.The model composite signal means the information that the generation at characteristic image is reflected.Memory 232 will output to the synthetic portion 26 of expression corresponding to the model composite signal 234 that information 186 is read.
In communication terminal 12, by its connect the synthetic portion 26 of expression use a model model composite signal that synthetic DB similarly receives in the information that makes Yu received reflected after the formation characteristic image.
Here, by having LCD etc. at least looking not shown on the display part of audio output part 38, the voice output of loud speaker etc. also is set, just can gives characteristic image and export the sound that is added corresponding to the audio signal of information 186 is additional.
Enumerate several informations that make and be reflected in method in the model composite signal.For example, under the situation of the dark of information 186 expression photography situations all blackouts of image are showed like this, selected characteristic is in darker place and shows like this.In addition, photography situation that information 186 expression and camera are separated, promptly away from situation under characteristic is shown lessly, characteristic is in place far away and shows like this.And then, object as the model composite signal both can add that background corresponding to the characteristic of some information changes, sheds tears, special expression effect is held in sweating or the symbol string demonstration in dialogue etc. also can append to sound such as effect sound in the characteristic image.
Explanation action simply.As shown in figure 45, communication terminal 12 offers expression with information 186 and synthesizes portion 26 after obtaining information 186.The synthetic portion 26 of expression visits information 186 and obtains corresponding model composite signal 234 (step S96) on memory 232.
In the synthetic portion 26 of expression, make model composite signal 234 be reflected such generation image to the basic characteristic image of being supplied with according to selection 58.The synthetic portion of expression 26 outputs to receiving and transmitting part 34 with the characteristic image that is generated.Face image or the characteristic image that generated are sent to communication terminal 14 (step S72) in receiving and transmitting part 34.Communication terminal 12 does not show as holding wire in Figure 44 with being appeared, at the characteristic image (step S80) of looking display surface hole pattern picture on the audio output part 38 or being generated.After showing, this makes the release of an Image Communication.
By based on the model composite signal formation characteristic image after information is reflected, the user just can confirm the photography situation in the formation characteristic image, and can significantly reduce user's burden.In addition, show that the display part of the notice of following information just no longer needs, visual concern place of user just reduces, thereby can more concentrate on communication.
Then only being conceived to notice according to prompting as other embodiment in the communication terminal of the image communication system 10 that has been suitable for information terminal device of the present invention shows and describes.The inscape of communication terminal 12 is simplified as far as possible, made server 236 have the characteristic image systematic function.Server 236 also can be arranged at characteristic administrative center 16.
On server 236, as shown in figure 46, inscape used in the communication terminal 12 is added identical reference marks.Server 236 comprises receiving and transmitting part 34, expressive features extracting part 24, expression synthetic portion 26, information generating unit 176 and memory 232.Particularly, memory 232 is not only preserved synthetic also in store characteristic image of DB of model and background.Thereby memory 232 will comprise with the corresponding characteristic image of user's selection and background and corresponding to the data 234 of the model composite signal of information 186 exports to the synthetic portion 26 of expression.
The synthetic portion 26 of expression, according to the data 234 formation characteristic images of being supplied with. the characteristic image that is generated receives the characteristic image of being supplied with 82 from the receiving and transmitting part 34 of server 236 via the communication terminal 12 that IP network 18 offers communication terminal 12 and 14. Figure 46 respectively, and outputs to and look audio output part 38. for example display communication object and the selected characteristic image of user in looking audio output part 38.
The action of the communication terminal 12 of image communication system shown in Figure 47 10.Communication terminal 12 is obtained captured view data 42 (step S66).Judge whether to send view data 42 (step S68a).(deny) view data 42 to be sent to look audio output part 38 under situation about not sending, enter demonstration and handle (to step S80).(being) sends to server 232 (step S98) with view data 42 from receiving and transmitting part 34 under situation about sending.Send image communication system 10 entrance managements by this and handle (to subprogram SUB).
Management processing is the processing in the server 236.Thus, the formation characteristic image, and send to the communication terminal 14 of communication object not shown among Figure 46.Server 236 also receives the view data formation characteristic image from communication terminal 14.Server 236 is sent characteristic image and the information supplied with from communication object at least among the characteristic image that is generated.Server 236 can also return the selected characteristic image of user self.
Management processing is shown in Figure 48 as subprogram.Server 236 is received in the captured view data of each communication terminal 42 (step S94).Secondly serve as the expression expressive features is extracted on the basis in expressive features extracting part 24 characteristic 48 with the view data 42 that receives.Characteristic 48 is supplied with synthetic portion 26 of expression and information generating unit 176 respectively.
In the synthetic portion 26 of expression, serve as that the basis generates the characteristic image (step S76) as composograph with the information of being supplied with.Server 236 sends to composograph and information 186 communication terminal 12 and 14 (step S102) of communication object.Transmission to communication terminal 12 is to send in order to be undertaken image to confirm by the user, also can self not sent.After this sends, server 236, the management processing that will carry out the image generation finishes and enters to return.Communication terminal receives such as described above composograph of supplying with respectively and information 186 shows.
In addition, though tentatively send information 186 in the present embodiment, do not send good as can be known from aforesaid record.
If like this it is moved, then make and generate the characteristic image that has reflected information, load high processing such as face characteristic extraction or characteristic image generation are carried out in server 236, even and if do not notify display part, the also available general TV telephone device of communication terminal 12 and 14 etc. substitutes, and just can more easily realize the image communication system with prompt facility.
Then only be conceived to utilize synthetic demonstration to describe to image tracing as other embodiment in the communication terminal of the image communication system 10 that has been suitable for information terminal device of the present invention. in the present embodiment, in the formation of Figure 34～Figure 36, utilize formation shown below respectively. the 1st, communication terminal 12 as shown in figure 49, comprising 238. Object Creation portions 238 of Object Creation portion and have the function of creating object. the establishment of object can also be the combination of pasting simple shapes such as ball square, perhaps round, the creation method of basic configuration texture images such as four jiaos. Object Creation portion 238 offers object preservation portion 180 with the object data of being created 240 and preserves. move with keep the object data 240 created and select to read the communication terminal 12 of Figure 34 that object uses by user's operation identical.
Just not only can create the object user who is predetermined and freely to create various objects by Object Creation portion 238 is set like this.Thus, just can increase the kind of the composograph that can create simply.
The 2nd, communication terminal 12 as shown in figure 50, comprises Expression Recognition portion 242 and the synthetic portion 244 of characteristic among the previous embodiment.The view data 42 that Expression Recognition portion 242 has with the face image of being supplied with is that the basis is analyzed, and creates the function of the characteristic 246 of the action that makes expression meet recognition result.As the recognition methods of expression, be what kind of expression with judgement for example with state parameterizations separately such as eye, mouths.Under the situation of eyes according to the sagging smile state that is identified as of the eye tip.In addition, can judge beyond expression that also downward state is to create the expression parameter.Particularly expression is sad discerns then that Expression Recognition portion 242 will output to the synthetic portion 244 of characteristic corresponding to the characteristic 246 of this identification if be judged as.
The synthetic portion 244 of characteristic has synthetic characteristic of being supplied with from expressive features portion 24 48 and 246 function.Characteristic is synthesized portion 244, the characteristic 248 of being synthesized is outputed to follow the tracks of handling part 182.
Follow the tracks of the object of handling part 182 access object preservation portions 180, and serve as that processing is implemented to follow the tracks of to the object of being read in the basis with the characteristic 248 that is supplied to the expression of reading expression and being discerned.Follow the tracks of handling part 182 and read tears as object, make tears be presented at the eyeground as following the tracks of to handle according to the identification of above-mentioned sadness.By making its demonstration just can perform facial expression image effectively like this, and can make the performance variation.
Thus, just can be not only also object be out of shape, thereby can shows effectively according to the expression of face according to the position of face.
The 3rd, the communication terminal 12 shown in Figure 51 comprises voice recognition portion 250 and the synthetic portion 244 of characteristic in the formation of Figure 49.It is that the basis is analyzed that voice recognition portion 250 has with the voice data of being supplied with from sound input part 22 46, creates the function of the characteristic 252 of the action that makes expression meet recognition result.Characteristic 252 is created to make and not only is mixed with the sound characteristic that also suitable object moves after the admixture volume for well.In addition, characteristic 252 can also only be discerned with volume, creates characteristic.Voice recognition portion 250 outputs to the synthetic portion 244 of characteristic with the characteristic of being discerned 252.
The object of the expression that expression discerns is read by tracking handling part 182 access object preservation portions 180 in the communication terminal 12, and serves as that the basis is implemented to follow the tracks of to the object of being read and handled with the characteristic 248 that is supplied to.Follow the tracks of handling part 182 for example send "? " sound the time head around with character "? " mark is read as object, and the language of perhaps analyzing sound import outputs to the synthetic portion 26 of expression with object images 198 after the language that is pronounced is generated the dialogue image as object.Thus, the synthetic portion 26 of expression synthesizes the characteristic image that generated and object images 198 and shows.Can also make the performance variation in view of the above, just can give image auxiliary effect by object is moved.
The 4th, the communication terminal 12 shown in Figure 52 comprises the synthetic portion 244 of Identification of events portion 254 and characteristic in the formation of Figure 49.Identification of events portion 254 has establishment corresponding to the function of exporting from the characteristic 258 of user's input information 256.Identification of events portion 254 outputs to the synthetic portion 244 of characteristic with the characteristic of being discerned 258.
Operating portion 28 has the function as the incident input part, for example provides the information that has reflected user view by the specific keys by lower keyboard.
For example, it is first-class to make the object of heart-shaped mark be presented at eyes by the specific keys by lower keyboard, just can export the image of the exaggerative performance that can't show on common image, and can make image have the performance that the user plans.
By constituting as described above, communication terminal 12 in the image communication system 10 offers driver 23 and output selection portion 56 respectively with view data 42 and voice data 46, the performance data 54 that the synthetic portion 26 of expression in driver 23 generates as the characteristic composograph in conjunction with characteristic 48, this performance data 54 and voice data 55 are transformed to respectively in formal argument portion 27 regulation communication form view data 60 and voice data 62 and offer output selection portion 56, output selection portion 56 from the view data 60 of a plurality of kinds and 42 and voice data 62 and 46 among select the one group of view data that will export 72 and voice data 74 to export, while just can do one's utmost to check cost rise easily be provided at recreational on excellent images communication and need not the formation of utilizing form as platform significantly being changed.Provide by this and just can help popularizing of this Image Communication.In addition, can also make for user's use more convenient.
The performance data that characteristic management department 30 not only manages as primary image also manages background data, have to these with the performance data of management and the characteristic administrative center 16 that manages concentratedly of background data obtain function and among the obtaining of data input part 108 that the performance data that write down the recording medium and background data are read at least one side obtain function, particularly, do not need characteristic administrative center 16 just can simplify aspect system is constructed by just obtaining of the latter as shown in Figure 5.
The text input part 112 of communication terminal 12 by as shown in Figure 6 is taken into text data just can be as numeral, being changed to the setting according to the special expression data of the special text data output that is predetermined, serves as that the characteristic image that basis generation band is expressed one's feelings is exported with special expression data.
Also can make text input part 112 hold the function that text data is transformed into voice data.
12 pairs of communication terminals are held in basic facial expression generating unit 118 shown in Figure 7 the essential characteristic data 122 that are made as basic expression of predetermined fixed and are set, selection data 120 output essential characteristic data 122 according to the user who is supplied with from operating portion 28, synthetic portion's 26 composite character data 48 of expression and essential characteristic data 122 are to generate the characteristic composograph of band expression, thereby the expressive ability of characteristic is improved, and can make and recreationally become higher.
The information 186 that the synthetic portion 26 of expression supplies with by making is reflected in the characteristic composograph that will generate shown in Figure 44 generate composograph, the user just can confirm the photography situation in the formation characteristic image, and can significantly reduce user's burden.In addition, show that the display part of the notice of following information just no longer needs, visual concern place of user just reduces, thereby can more concentrate on communication.
The notice control part outputs to receiving and transmitting part 34 with information shown in Figure 41 186, information is offered the communication terminal 14 of communication object.Communication terminal 14 comprises receiving and transmitting part 34, notice control part 178 at least and looks audio output part 38, even if thus by the communication object destination is not for example sent from the photographs of camera only the transfer characteristic image also can notify information 226, whether thereby the user of communication object destination just can confirm the existence of communication object, and can eliminate and actually the uneasiness sense of communication object is arranged and can concentrate on communication.
The image communication system 10 that is connecting communication terminal 12 has server shown in Figure 46 236, and server 236 comprises receiving and transmitting part 34, expressive features extracting part 24, the synthetic portion 26 of expression and information generating unit 176.Server 236 serves as that the basis is at the synthetic portion of expression 26 formation characteristic composographs with performance data, characteristic and information, and output to the communication terminal 14 of communication object at least, make thus and generate the characteristic image that has reflected information, load high processing such as face characteristic extraction or characteristic image generation are carried out in server 236, even and if do not notify display part, the also available general TV telephone device of communication terminal 12 and 14 etc. substitutes, and just can more easily realize the image communication system with prompt facility.
In communication terminal 12, expressive features extracting part 24 is a benchmark with the face image in front, the position at each position from the face image of being imported to face towards analyzing with amount of movement, object is followed the tracks of the variation of characteristic image.
The expression that communication terminal 12 is held from view data identification face image in Expression Recognition portion 242 shown in Figure 50, generate expressive features data 246, synthetic expressive features data 246 and characteristic 48 in the synthetic portion 244 of characteristic, composite character data 248 are outputed to tracking handling part 182, is basic formation object data 198 following the tracks of handling part 182 with composite character data 248, just facial expression image can be performed effectively thus, and the performance variation can be made.
Communication terminal will provide as end message at the judgement information of request from the location information management portion 128 of characteristic management department 30 shown in Figure 8, in content control portion 126 with the performance data at least supplied with and background data as content information, according to the output request of the use that obtains request and content information of the distribution of content information to content information obtain and output manages, characteristic administrative center 16 is connected to the communication line in the image communication system 10, in memory shown in Figure 4 102, preserve at least the license distribution information of the distribution of licensed content information and the content-data that content information is handled integratedly, in content management server 100 to this content-data and at least, a side request among the output request that obtains request and content-data of the content-data of being supplied with from communication terminal 12, the control of management distribution or output, communication terminal 12 outputs to content management server 100 according to the request of being supplied with end message, content management server 100 judges whether the end message of being supplied with satisfies the condition that comprises in the license distribution information, according to the judgement that satisfies condition to the request of communication terminal 12 permission at content-data, just can easily manage thus, what can also solve copyright in for example different states region has a no problem etc.
Here, preferably license distribution information be at least will permission positional information and useful life information in some, end message is some in the positional information of communication terminal 12 and the time information that communication terminal 12 sends request at least.
In communication terminal 12, some in the time information management department 174 that constantly measures of location information management portion 128 that the position to communication terminal 12 self of Fig. 8 or Figure 15 is detected and Figure 20, the management that just can help information to provide thus are provided information provider unit at least.
In image communication system 10, memory 152 also is mapped with the historical information of the position of the effluxion of attendant position information shown in Figure 4 and preserves, content management server 100 also utilizes the historical information of position in the judgement at request, request is controlled, thus up to now to license information, event information and content-data are cut apart respectively and are handled, must intricately manage, but now just can be only manage, thereby the license management can make incident such as label sets the time becomes than easy with contact with the label sets information of content and communication terminal 12.
In image communication system 10, communication terminal 12 comprises intrinsic information that corresponds respectively to communication terminal 12 and 14 and the user profile of determining the user uniquely as end message in content-data, and send to characteristic administrative center 16, the License Info of being preserved in 16 pairs of end messages of being supplied with of characteristic administrative center and memory 106 and 152 compares, the communication terminal that consistency according to these information is judged as permission provides and satisfies information requested, in accepting the communication terminal that this information provides, utilize content-data, individual's license management is become than easy with contact according to this information.
In image communication system 10, the license management portion 158 that communication terminal 12 possesses memory 152 and has the management function of content server device, manage the simplification of constructing that request just can make system by in content-data, embedding end message, and can be on constructing cutting down cost.
In image communication system 10, characteristic administrative center 16 collects usage charges according to the charge method (charge form) of the utilization of following content-data from above-mentioned user at charging server shown in Figure 23 166, in communication terminal 12 and 14, come the use of limiting content data respectively according to the judgement of license distribution information and end message, the condition of using restriction is removed in charging server 166 promptings, confirming that the user pays on the basis of suitable usage charges, the utilization of licensed content data, just can easily upgrade prolongation thus, for example after having carried out content grade on probation, can formally offer the user as commercial version, just can be when wanting to use content for the user, provide by the part of wanting to use and charge accordingly. in addition, the establishment funds of regaining content by usage charges effectively just become possibility.
Wish charging server 166, the range information that obtains from the positional information set stage by stage as the charge method utilization and between the operating period information charge.
Claims (26)
1. information terminal device is characterized in that:
In the image delivery system that utilization is handled image as the communication information information terminal device communicates, this information terminal device comprises:
Acoustic input dephonoprojectoscope is taken into audio signal and becomes voice data and exports;
Image-input device is taken into subject and becomes view data and exports as vision signal;
Interface arrangement is transformed into this view data and tut data respectively the communication format of regulation;
Choice device among a plurality of view data and voice data that communicate with communication object, is selected desirable view data and voice data;
R-T unit carries out data transmit-receive with the information terminal device of above-mentioned communication object; And
Output device is exported view data and voice data from the information terminal device of above-mentioned communication object,
Wherein, above-mentioned interface arrangement comprises:
The feature extraction device, at least one side extracts the characteristic of expression expression from tut data and above-mentioned view data;
The expression synthesizer makes as the performance data of user-selected primary image based on this characteristic and to be out of shape, with the formation characteristic composograph;
Image management apparatus manages the above-mentioned performance data of supplying with this expression synthesizer; And
The formal argument device is transformed into the communication format of regulation respectively with above-mentioned view data and voice data,
This information terminal device at least also is used to one of above-mentioned information terminal device that communicates.
2. according to the described information terminal device of claim 1, it is characterized in that:
This information terminal device comprises:
Code device is encoded to the above-mentioned characteristic composograph that generates;
Decoding device is decoded to view data and the voice data supplied with from the information terminal device of above-mentioned communication object.
3. according to claim 1 or 2 described information terminal devices, it is characterized in that:
Above-mentioned image management apparatus not only also manages background data above-mentioned performance data,
Above-mentioned information terminal device, have from centralized management device that the above-mentioned performance data of these management and above-mentioned background data are managed concentratedly, the above-mentioned image delivery system obtain function and from device that the above-mentioned performance data that write down the recording medium and above-mentioned background data are read obtain among the function, at least one side obtain function.
4. according to the described information terminal device of claim 2, it is characterized in that:
This device comprises: the text input device that is taken into text data.
5. according to the described information terminal device of claim 4, it is characterized in that:
Above-mentioned text input device has the function that above-mentioned text data is transformed into voice data.
6. according to the described information terminal device of claim 4, it is characterized in that:
This device comprises: the feature output device, and the essential characteristic data that are made as basic expression of fixing holding are in advance set, and export this essential characteristic data according to above-mentioned user's selection,
Above-mentioned expression synthesizer, synthetic above-mentioned characteristic and above-mentioned essential characteristic data are to generate the characteristic composograph of band expression.
7. according to the described information terminal device of claim 6, it is characterized in that:
This device comprises: the information generating apparatus is that the basis generates the advisory information which kind of situation is the photography situation of representing above-mentioned user be in above-mentioned characteristic.
8. according to the described information terminal device of claim 7, it is characterized in that:
Above-mentioned expression synthesizer above-mentioned characteristic composograph that generates of output and the above-mentioned advisory information of being supplied with.
9. according to the described information terminal device of claim 7, it is characterized in that:
Above-mentioned expression synthesizer makes the advisory information of being supplied be reflected in the characteristic composograph of above-mentioned generation.
10. according to the described information terminal device of claim 7, it is characterized in that:
This device comprises: above-mentioned advisory information is transformed into the notice converting means of announcement information,
Wherein, above-mentioned announcement information is accepted and exported to above-mentioned output device.
11., it is characterized in that according to the described information terminal device of claim 10:
Above-mentioned information generating apparatus, providing on the basis of above-mentioned suggestion, extraction success rate to above-mentioned characteristic is judged, also comprise the 1st functional block that the light and shade of image is detected from above-mentioned view data simultaneously, the 2nd functional block that the distance of face and camera is detected from above-mentioned characteristic, to nictation and mouthful the 3rd functional block that detects of switching and the 4th functional block that sight line is detected, and
Among above-mentioned judgement and each functional block at least one carried out integration, generates the synthetic determination device of above-mentioned advisory information,
And then this device comprises: the storage device that above-mentioned advisory information and above-mentioned announcement information are mapped and manage,
Above-mentioned notice converting means conducts interviews to above-mentioned storage device, obtaining the laggard line translation of above-mentioned announcement information corresponding to above-mentioned advisory information, and outputs to above-mentioned output device.
12., it is characterized in that according to the described information terminal device of claim 11:
Above-mentioned notice converting means outputs to above-mentioned R-T unit with above-mentioned advisory information, and above-mentioned advisory information is offered the information terminal device of communication object,
The information terminal device of above-mentioned communication object comprises above-mentioned R-T unit, above-mentioned notice converting means and above-mentioned output device at least.
13., it is characterized in that according to the described information terminal device of claim 1:
This device comprises: will remove object above-mentioned performance data, the subsidiary object of conduct and save as the object save set of data; And
Tracking processing apparatus is implemented to follow the tracks of processing and is exported resulting object data above-mentioned object according to above-mentioned characteristic,
Wherein, above-mentioned expression synthesizer is that the basis generates above-mentioned characteristic composograph with object data and the above-mentioned performance data of being supplied with.
14., it is characterized in that according to the described information terminal device of claim 13:
Above-mentioned feature extraction device is made as benchmark with the face image in front, the position at each position from the face image of being imported to face towards analyzing with amount of movement.
15., it is characterized in that according to the described information terminal device of claim 13:
This device comprises: the Expression Recognition device, and the expression of the represented face image of above-mentioned view data is held in identification, and generates the expressive features data; And
The feature synthesizer, synthetic these expressive features data and above-mentioned characteristic, and output composite character data,
Wherein, above-mentioned tracking processing apparatus is that the basis generates above-mentioned object data with above-mentioned composite character data.
16., it is characterized in that according to the described information terminal device of claim 13:
This device comprises: voice recognition device is that sound recognition is come on the basis with above-mentioned voice data, and generates the sound characteristic data; And
The feature synthesizer, synthetic these sound characteristic data and above-mentioned characteristic, and output composite character data,
Wherein, above-mentioned tracking processing apparatus is that the basis generates above-mentioned object data with above-mentioned composite character data.
17., it is characterized in that according to the described information terminal device of claim 13:
This device comprises: operating means, the incident of the operation input regulation by above-mentioned user;
The Identification of events device is discerned the incident of this regulation, generates the affair character data; And
The feature synthesizer, synthetic these affair character data and above-mentioned characteristic, and output composite character data,
Wherein, above-mentioned tracking processing apparatus is that the basis generates above-mentioned object data with above-mentioned composite character data.
18. the server unit that has in the image delivery system is characterized in that:
This system is connected with any described information terminal device in the claim 1 to 6,
This server unit comprises: above-mentioned R-T unit, above-mentioned feature extraction device, above-mentioned expression synthesizer and be the information generating apparatus that photography situation that the basis generates the above-mentioned user of expression is in the advisory information of which kind of situation with above-mentioned characteristic,
This server unit serves as that the basis generates above-mentioned characteristic composograph at above-mentioned expression synthesizer with above-mentioned performance data, above-mentioned characteristic and above-mentioned advisory information, and outputs to the information terminal device of above-mentioned communication object at least.
19. an image delivery system is characterized in that:
This system is connected with any described information terminal device in the claim 1 to 18, above-mentioned information terminal device also comprises: the request management device, above-mentioned at least performance data and the above-mentioned background data supplied with are made as content information, according to the request this content information distribution call request and ask this content information use the output request to foregoing information obtain and output manages; And
Information provider unit will offer this request management device as end message at the judgement information of request,
Centralized management device in the above-mentioned image delivery system is connected to the communication line in the above-mentioned image delivery system, above-mentioned centralized management device comprises: content storage devices, preserve the License Info of the distribution that will permit foregoing information and the content-data that foregoing information is handled integratedly at least; And content server device, among the output request that obtains request and content-data to this content-data and the content-data supplied with from above-mentioned information terminal device at least, a side request management distribution or the control of output,
Above-mentioned information terminal device outputs to the foregoing server unit according to the request of being supplied with above-mentioned end message,
Whether this content server device satisfies the condition that is contained in the above-mentioned License Info to the end message supplied with is judged, and according to the judgement of satisfying above-mentioned condition to the request of above-mentioned information terminal device permission at the foregoing data.
20., it is characterized in that according to the described image delivery system of claim 19:
Above-mentioned License Info is at least some in the positional information of permission and the useful life information,
Above-mentioned end message is at least some in the positional information of above-mentioned information terminal device and the time information that this information terminal device sends request.
21., it is characterized in that according to the described image delivery system of claim 19:
Above-mentioned information provider unit comprises device that the position of this device self is detected and some in the device of constantly measuring at least.
22., it is characterized in that according to the described image delivery system of claim 19:
The foregoing storage device also is mapped above-mentioned positional information and preserves with the historical information of following the position of effluxion,
The foregoing server unit also utilizes the historical information of above-mentioned position at the judgement of above-mentioned request, and above-mentioned request is controlled.
23., it is characterized in that according to the described image delivery system of claim 19:
Above-mentioned information terminal device will corresponding to above-mentioned information terminal device separately intrinsic information and determine that uniquely above-mentioned user's user profile is included in the foregoing data as above-mentioned end message, send to above-mentioned centralized management device,
This centralized management device compares the License Info of being preserved in the end message supplied with and the foregoing storage device, and the information terminal device that the consistency according to these information is judged as permission provided satisfies information requested,
In being provided, the information terminal device that provides of this information utilizes the foregoing data according to this information.
24. an information terminal device is characterized in that:
In claim 19,20 or 22 described image delivery systems, above-mentioned information terminal device possesses the management function of foregoing storage device and foregoing server unit, embeds above-mentioned end message and manage above-mentioned request in the foregoing data.
25. a Fare Collection System is characterized in that:
In claim 19,20 or 22 described systems, above-mentioned centralized management device comprises: charging device, collect usage charges according to the charge method of the utilization of following the foregoing data from above-mentioned user,
In above-mentioned information terminal device, limit the use of foregoing data by the judgement of above-mentioned License Info and above-mentioned end message,
The condition of above-mentioned use restriction is removed in above-mentioned charging device prompting, is confirming that above-mentioned user pays on the basis of suitable usage charges, the utilization of permission foregoing data.
26., it is characterized in that according to the described Fare Collection System of claim 25:
Above-mentioned charging device, the range information that obtains from the above-mentioned positional information set stage by stage as the charge method utilization and between the above-mentioned operating period information charge.
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2004381300A JP4218637B2 (en) | 2004-12-28 | 2004-12-28 | Information terminal equipment |
| JP2004381300 | 2004-12-28 | ||
| JP2004-381300 | 2004-12-28 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1798324A CN1798324A (en) | 2006-07-05 |
| CN1798324B true CN1798324B (en) | 2010-05-12 |
Family
ID=36738435
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2005100592804A Expired - Fee Related CN1798324B (en) | 2004-12-28 | 2005-03-25 | Information terminal device |
Country Status (4)
| Country | Link |
|---|---|
| JP (1) | JP4218637B2 (en) |
| KR (1) | KR20060076158A (en) |
| CN (1) | CN1798324B (en) |
| TW (1) | TW200622783A (en) |
Families Citing this family (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2008085421A (en) * | 2006-09-26 | 2008-04-10 | Asahi Kasei Corp | Video telephone, calling method, program, voice quality conversion-image editing service providing system, and server |
| KR101364844B1 (en) * | 2007-04-02 | 2014-02-19 | 엘지전자 주식회사 | Mobile communication terminal with video call function and method of controlling the same |
| JP2008288626A (en) * | 2007-05-15 | 2008-11-27 | Sony Ericsson Mobilecommunications Japan Inc | Event holding system |
| JP5515205B2 (en) * | 2007-09-19 | 2014-06-11 | 日本電気株式会社 | Video distribution system, server system, client system, video distribution method, and program |
| JP2011151773A (en) * | 2009-12-21 | 2011-08-04 | Canon Inc | Video processing apparatus and control method |
| CN101917585A (en) * | 2010-08-13 | 2010-12-15 | 宇龙计算机通信科技(深圳)有限公司 | Method, device and terminal for regulating video information sent from visual telephone to opposite terminal |
| JP2012085009A (en) | 2010-10-07 | 2012-04-26 | Sony Corp | Information processor and information processing method |
| JP2015172883A (en) * | 2014-03-12 | 2015-10-01 | 株式会社コナミデジタルエンタテインメント | Terminal equipment, information communication method, and information communication program |
| CN105787976A (en) * | 2016-02-24 | 2016-07-20 | 深圳市金立通信设备有限公司 | Method and apparatus for processing pictures |
| KR102165271B1 (en) * | 2016-08-22 | 2020-10-13 | 스노우 주식회사 | A message sharing method for sharing image data reflecting the state of each user through a chat room A message sharing method and a computer program for executing the method |
| US10681310B2 (en) | 2018-05-07 | 2020-06-09 | Apple Inc. | Modifying video streams with supplemental content for video conferencing |
| WO2020129959A1 (en) * | 2018-12-18 | 2020-06-25 | グリー株式会社 | Computer program, server device, terminal device, and display method |
| JP6687717B1 (en) * | 2018-12-28 | 2020-04-28 | 新井 敏夫 | Terminal control program, user terminal, and terminal control method |
| JP7231301B2 (en) * | 2020-10-28 | 2023-03-01 | 圭司 田谷 | Online meeting support system and online meeting support program |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1274240A (en) * | 2000-05-30 | 2000-11-22 | 卢世柱 | Visual mobile telephone |
-
2004
- 2004-12-28 JP JP2004381300A patent/JP4218637B2/en not_active Expired - Fee Related
-
2005
- 2005-03-01 TW TW094106006A patent/TW200622783A/en unknown
- 2005-03-25 CN CN2005100592804A patent/CN1798324B/en not_active Expired - Fee Related
- 2005-06-21 KR KR1020050053699A patent/KR20060076158A/en not_active Application Discontinuation
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1274240A (en) * | 2000-05-30 | 2000-11-22 | 卢世柱 | Visual mobile telephone |
Non-Patent Citations (2)
| Title |
|---|
| JP特开2003-16475A 2003.01.17 |
| JP特开200486441A 2004.03.18 |
Also Published As
| Publication number | Publication date |
|---|---|
| TW200622783A (en) | 2006-07-01 |
| KR20060076158A (en) | 2006-07-04 |
| CN1798324A (en) | 2006-07-05 |
| JP4218637B2 (en) | 2009-02-04 |
| JP2006185393A (en) | 2006-07-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN1798324B (en) | Information terminal device | |
| US10008242B2 (en) | Automatically syncing recordings between two or more content recording devices | |
| JP3593067B2 (en) | Information terminal device with image communication function and image distribution system | |
| US11955125B2 (en) | Smart speaker and operation method thereof | |
| US7967200B2 (en) | Authorization verification method and devices suited therefor | |
| US20080109867A1 (en) | Service and policies for coordinating behaviors and connectivity of a mesh of heterogeneous devices | |
| CN105165079B (en) | Portable terminal, information management system and approaches to IM | |
| JPH11177699A (en) | Method for calling user designation information through mobile telephone | |
| US8750518B2 (en) | Method and system for archiving communication data by means of data tracing | |
| EP1347646A2 (en) | Communication terminal for transmission of facial information | |
| JPH08315106A (en) | Digital camera and image data distribution system | |
| KR100686076B1 (en) | Wireless Communication Terminal with Message Transmission According to Feeling of Terminal-User and Method of Message Transmission Using Same | |
| JP2002117075A (en) | Signal acquiring device, database, database system, signal presentation system, and system for signal acquisition, storage, and presentation | |
| US7680863B2 (en) | Contents providing method, contents providing system, and contents server | |
| US9092885B2 (en) | Method of processing a text, gesture, facial expression, and/or behavior description comprising a test of the authorization for using corresponding profiles for synthesis | |
| CA2468890A1 (en) | Method for making secure access to a digital resource | |
| KR101664407B1 (en) | Personal On-line Recording Management System by Using Network and Method thereof | |
| JP2006216006A (en) | Information output service providing system, and portable information terminal, output device, relay device constituting this system | |
| JP5082699B2 (en) | Minutes creation device, minutes creation system, minutes creation method, and minutes creation program | |
| KR102086609B1 (en) | Device and Method for charging | |
| CN102037496A (en) | Photo realistic talking head creation, content creation, and distribution system and method | |
| KR100535640B1 (en) | Image processing system using the communication network | |
| JP5182286B2 (en) | Information transfer control device and information transfer control method for transferring content | |
| KR20230118430A (en) | Interactive idol goods device, system and method for providing voice service using the same | |
| Evans | Consumer gadgets: 50 ways to have fun and simplify your life with today's technology... and tomorrow's |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C17 | Cessation of patent right | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20100512 Termination date: 20130325 |