JP4175232B2 - Videophone system and videophone device - Google Patents

Description

The present invention relates to a videophone system and videophone apparatus, by displaying on the same screen, particularly to synthesize a communication partner image and the advertisement image, videophone system and a television that enables the advertisement distribution service of video phone usage It relates to a telephone device .

Conventionally, a telephone system described in Patent Document 1 (Japanese Patent No. 3056140) is known as a videophone device that provides advertisement information together with a caller image. According to this, a user can be provided with advertisement information during a call by intervening advertisement information such as still image information, moving image information, or audio information between call paths connected to the other party. It can be done. Moreover, according to this means, it is possible to provide a telephone system in which the sender of the advertisement information bears the billing cost associated with the user's call.
Patent No. 3056140

  However, in the telephone system according to the invention described in Patent Document 1, since it is necessary to switch between a call state and an advertisement providing state at a predetermined time, a caller is kept waiting for a call while receiving advertisement information. There is a problem that it takes time to complete the requirements, and if the intervention time of the advertisement information is shortened to improve the requirement, the exposure time of the advertisement information is reduced and the advertisement effect is lowered.

On the other hand, the present invention has been made to solve such a problem. Regarding the screen display of the videophone device, an advertisement image recorded in the videophone device in advance is used as a background image, and this and the other party image are displayed. By combining and displaying on the screen, switching control between the call state and the advertisement provision state is eliminated, thereby solving the problem of periodically waiting for the caller, and providing advertisement images at all times It is an object of the present invention to provide a videophone system and a videophone device that can be dramatically improved.

In addition, regarding the composition of the partner image and the advertisement image, the size and arrangement of the partner image can be specified for each advertisement image to prevent the advertisement image from being always hidden in the partner image, and to move the display of the partner image. An object of the present invention is to provide a videophone system and a videophone device that can improve the advertising effect.

The present invention comprises the following means (1) and (2 ) in order to solve the above problems.
(1) In a videophone system in which a plurality of videophone devices and an advertisement management server are connected via a network,
The advertisement management server
A plurality of advertisement images to be displayed on the monitor, each of the plurality of video telephone apparatus has, been implicated for each advertisement image, layout data of the partner image is an image of the other party, the scaling value of the counterpart image, and the have a advertisement database in which motion data storing advertisement data contained for changing the display of the other party image,
Each of the plurality of videophone devices is
Storage means for storing advertisement data downloaded from the advertisement database of the advertisement management server;
From the captured image by the camera, an image extraction means for extracting the principal image image by separating the person and background images parties,
And voice processing means for detecting a speech variable level values and the frequency values of the principal sound entering the personal audio microphone or found before Symbol party,
A person image extracted by the image extracting means, encoding means for a personal voice to sign-reduction, respectively it input from the microphone,
Sends a coded data respectively to the television telephone apparatus of the communication partner are multiplexed by the encoding means, receiving the multiplexed encoded data transmitted from the video telephone of the calling party And data transmission / reception means for demultiplexing,
Decoding means for decoding the encoded data demultiplexed by the data transmitting / receiving means;
And the mating image obtained by decoding by the decoding means, and an advertisement image contained in the stored advertisement data in the storage means, corresponding to the detected speech variables by the speech processing unit, the advertisement data based on the arrangement data, the scaling value, and the motion data included in the arrangement position of the other party image, Ru is displayed size, and the monitor are combined on the same screen by changing at least one of the angles Image composition means,
A videophone system characterized by comprising.
(2) A plurality of advertisement images, arrangement data of an opponent image that is an image of the other party, a scaling value of the opponent image, and motion data for changing the display of the opponent image, associated with each advertisement image. Storage means for storing advertisement data downloaded via the network from the advertisement management server in which the included advertisement data is stored;
An image extracting means for separating a caller's identity image and a background image from the image captured by the camera and extracting the identity image;
Voice processing means for inputting the voice of the caller from a microphone and detecting a level value and a frequency value of the voice of the caller as voice variables;
Encoding means for encoding the principal image extracted by the image extraction means and the principal voice input from the microphone;
The encoded data encoded by the encoding means is multiplexed and transmitted to the videophone device of the other party, and the multiplexed encoded data transmitted from the videophone device of the other party is received. Data transmission / reception means for demultiplexing,
Decoding means for decoding the encoded data demultiplexed by the data transmitting / receiving means;
The other party image obtained by decoding by the decoding unit and the advertisement image included in the advertisement data stored in the storage unit are associated with the audio variable detected by the audio processing unit, and the advertisement data Based on the arrangement data, the scaling value, and the movement data included in the image, an image to be combined and displayed on the same screen by changing at least one of the arrangement position, size, and angle of the counterpart image Combining means,
A videophone device characterized by comprising.

As described above, according to the present invention, by combining and displaying the partner image and the advertisement image on the screen of the videophone device, the advertisement image is always displayed without interrupting a normal videophone call. Thus, there is an effect that a long time is not given to a telephone user, and a videophone service that allows an advertisement provider to always provide an advertisement image can be provided. In addition, when synthesizing the partner image with the advertisement image, the image is synthesized based on information on the arrangement position or size of the partner image determined in advance for each advertisement image. This has the effect of preventing defects.

Furthermore, according to the present invention, the arrangement position of the other party image, or size, or the inclination, it is possible to vary according to the level value or frequency value of the voice of the person voice, advertising images hidden counterpart image And the effect of improving the advertising effect by attracting the user's attention to the advertising image.

  The best mode for carrying out the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing the configuration of the entire videophone system according to Embodiment 1 of the present invention. In this figure, reference numeral 1 is a transmitting side videophone device, 2 is a receiving side videophone device, and 1 and 2 are videophone devices having the same configuration. 3 is an advertisement management server for managing advertisement information, 3a is an advertisement database stored in the advertisement management server 3, 4 is a billing management server for managing expenses generated by the user of the videophone device, and 5 is a PSTN. It is a network constructed by a communication network such as a network (public telephone switched circuit network), an ISDN network, an Internet network controlled by an Internet protocol. The devices 1 to 4 are connected to a network 5 and can transmit and receive data between the devices. As described above, the entire videophone system according to the present embodiment is configured. In the above configuration, there may be a plurality of advertisement management servers 3 or accounting management servers 4 as well as videophone devices.

  Next, each part will be described in detail based on the configuration of FIG. FIG. 2 is a block diagram showing the configuration of the videophone apparatus. In FIG. 2, 11 is an image extraction means, 12 is an image codec means, 13 is a data transmission / reception means, 14 is an image composition means, 15 is a storage means, 15a is advertisement data stored in the storage means 15 for a predetermined time, and 16 is a sound. The processing means, 17 is an audio codec means, 18 is an advertisement control means, and the videophone apparatus is configured as described above.

  Next, the operation of each part of the videophone device will be described. First, the image extraction means 11 separates the caller's identity image and the background image from the captured image input from the camera, and outputs only the identity image data to the image codec means 12 and the image composition means 14. However, in this embodiment, the output to the image composition means 14 is not essential.

  In this case, as a method for separating and extracting a foreground image such as a person image from the entire image, for example, a method called a chroma key technique is generally known. Briefly describing this, the chroma key technique is a method of taking a foreground image in front of a background of a specific color such as blue and extracting the foreground image based on the difference in color information between the foreground image and the background image. It is. An image extraction method that does not depend on the above-described chroma key technique is also known. This is a method of specifying and extracting a foreground image portion that is moving with respect to a background image, and is described in, for example, Japanese Patent Application Laid-Open No. 2003-179811. In this embodiment, any method may be used.

  On the other hand, the voice processing means 16 outputs the voice of the caller himself / herself from the microphone to the voice codec means 17 and sequentially measures the level value and the frequency value of the voice of the person himself / herself, and uses these as voice variables for the image synthesis means 14. Output in real time. Also, the other party's voice decoded by the voice codec means 17 is output to the speaker. Furthermore, the sound effect data included in the advertisement data 15a is reproduced and output to the speaker. The speaker includes an audio output device such as an earphone or a headphone.

  The image codec unit 12 performs high-efficiency encoding on the person image extracted by the image extraction unit 11 and outputs the encoded image to the data transmission / reception unit 13, and also decodes the encoded data input from the data transmission / reception unit 13 to decode the partner image. And output to the image composition means 14.

  The voice codec means 17 performs high-efficiency coding on the principal voice inputted from the voice processing means 16 and outputs the same to the data transmission / reception means 13, and also decodes the encoded data inputted from the data transmission / reception means 13. It is converted into the other party's voice and output to the voice processing means 16.

  Here, as a method for highly efficient encoding / decoding of image data, for example, H.264 standardized by ITU-T (International Telecommunication Union Telecommunication Standardization Sector) is used. 261 and H.264. It is desirable to use a standardized system capable of efficiently compressing a large amount of data, such as those using various systems such as H.263. Also, the method for high-efficiency encoding / decoding of audio data is the same as in the case of the image data. 711 and G.G. It is desirable to use a standardized system such as those using various systems such as 722, or those using the MPEG system.

  The data transmission / reception means 13 is input from the high-efficiency encoded data of the principal image input from the image codec means 12, the high-efficiency encoded data of the principal voice input from the audio codec means 17, or the advertisement control means 18. One or more types of advertisement viewing information data are multiplexed and transmitted to the network 5, and data received from the network 5 is demultiplexed to the image codec means 12, audio codec means 17, or advertisement control means 18. Output each. The advertisement viewing information data will be described later.

  The advertisement control means 18 outputs a request for downloading advertisement data from the advertisement database 3 a to the advertisement management server 3 connected to the network 5, and supplies the downloaded advertisement data to the storage means 15. Control and control for creating advertisement viewing information data based on the processing status of the advertisement data 15 a stored in the storage unit 15 and outputting the advertisement viewing information data to the accounting management server 4 connected to the network 5 are performed.

  The storage means 15 stores the downloaded advertisement data as advertisement data 15a for a predetermined effective period. This valid period is specified in the advertisement data 15a as described below.

  Here, the advertisement data 15a will be described. FIG. 3 is a diagram illustrating a data structure of the advertisement data 15a. In FIG. 3, reference numeral 31 denotes a data path for referring to various data stored in the advertisement data as indicated by arrows in the figure, data indicating the size of the advertisement image, a scaling value of the call partner image, One or more advertisement parameters comprising data indicating the validity period of the advertisement image and reproduction time data of the advertisement image; 32, advertisement image data comprising one or more advertisement images combined with the counterpart image; Is arrangement data composed of one or more caller arrangement data for designating a position where the partner image is combined with the advertisement image, 34 is for moving the arrangement of the partner image, changing the size of the partner image, etc. Motion data consisting of one or more action data for generating an action, 35 is sound effect data consisting of one or more sound effects for generating a sound effect , And the above-mentioned 32 to 35 are configured to be referred to by the data path of the ad parameters 31.

  In the advertisement database 3a of the advertisement management server 3, a plurality of advertisement data having the structure shown in FIG.

  The image synthesizing unit 14 synthesizes the partner image decoded by the image codec unit 12 and a predetermined advertisement image in the advertisement image data 32 referred to from the advertisement parameter 31 of the advertisement data 15a and outputs the synthesized image to the monitor. . In this, based on predetermined action data in the motion data 34 referred to from the advertisement parameter 31, according to the level value or frequency value of the principal voice included in the voice variable input from the voice processing unit 16, The display position of the counterpart image can be moved in a predetermined direction, the size can be changed within a predetermined scale range, or the direction can be changed with a predetermined inclination.

  Next, the operation of the videophone system according to the present embodiment will be described using a sequence diagram. FIG. 4 is a sequence diagram of the videophone system according to the configuration of FIG. Here, the case where valid advertisement data does not exist in the storage means 15 of the videophone device will be described as an initial value.

  First, the user himself / herself makes a call by calling the other party's incoming videophone device 2 from the outgoing videophone device 1 (101). Next, the caller-side videophone device 1 makes a download request for advertisement data to the advertisement management server 3 (102), whereby at least one advertisement data is downloaded to the caller-side videophone device 1 (103). ). On the other hand, when the other party's incoming videophone device 2 responds (104), the incoming videophone device 2 also requests advertisement data from the advertisement management server 3 (105), thereby at least one or more advertisement data. Is downloaded to the incoming videophone device 2 (106). Next, when display of the videophone device on the calling side / calling side is started, advertisement start notifications are issued from the respective videophone devices to the accounting management server 4 (107), (108). After the call / advertisement display time elapses (109), when the call / advertisement display time is longer than the time set in the advertisement parameter 31 of the advertisement data 15a, the sequences (102), (103), (105 ), (106), (107), and (108) are repeatedly executed. When the call is finished (110) and (111), the respective videophone devices send advertisement viewing information data to the billing management server 4 to notify advertisement termination (112) and (113).

  In this case, the advertisement viewing data has a data structure shown in FIG. According to FIG. 5, the videophone device used by the telephone device ID can be identified, the advertisement viewed by the advertisement ID list can be identified, and the advertisement usage time can be calculated from the advertisement start time, the advertisement end time, and the number of repetitions. it can. In this way, after the processing according to the sequence described above, the billing management server 4 calculates an advertisement usage fee based on the advertisement viewing information data.

  Next, processing of the videophone device will be described with reference to the flowchart shown in FIG. FIG. 6 shows the processing contents of the user's own videophone apparatus. First, when a videophone device call is started, the advertisement control means 18 determines whether or not there is valid advertisement data in the storage means 15 (step S1). The control means 18 downloads at least one or more advertisement data from the advertisement database 3a of the advertisement management server 3 to the storage means 15 (step S2). Next, the advertisement control means 18 reads the advertisement parameter 31 from the valid advertisement data 15a stored in the storage means 15 (step S3). Subsequently, the arrangement position and scaling value of the partner image are acquired based on the advertisement parameter 31 (step S4). In this case, since the size of the caller image captured by the camera is usually various, the scaling value is set in advance to an appropriate ratio between the size of the advertisement image and the size of the partner image between 0.1 and 1.0. Has been. The partner image whose display position is determined based on the arrangement position of the partner image and the scaling value is combined with the advertisement image by the image combining unit 14 and output to the monitor (step S5). A time is set in the advertisement parameter 31, and the advertisement control means 18 determines whether or not the time has passed. When the set time has passed, the process returns to step S1 (step S6). Subsequently, the end of the call is confirmed. If the call has not ended, the process returns to step S5 (step S7).

  As described above, according to the videophone system according to the first embodiment, based on the advertisement data 15a stored in the storage unit 15, the other party image of the call and the advertisement image are combined on the same screen with an appropriate relationship. Thus, the advertisement image can be displayed at all times without interrupting the display of the partner image and the call, and the telephone use of the telephone user and the advertisement provision of the advertising company can coexist. . An example of this display is shown in FIG. FIG. 7 is a display example in which the partner image and the advertisement image displayed on the monitor of the videophone device are combined. As this shows, the partner image 71 is combined with the advertisement image 72 on the monitor, and both the advertisement image and the partner image can be displayed.

  In addition, the size and arrangement of the partner image can be changed based on the motion data 34 of the advertisement data 15a in accordance with the volume of the voice of the person's voice and the voice variable depending on the height depending on the state of the conversation during the call. Attract the attention of the person to the advertisement. This is illustrated in FIG. FIG. 8 is an example in which the size of the partner image is reduced and moved to the lower right of the screen corresponding to the loudness or height of the voice of the principal voice from the display example of FIG.

  The videophone device according to the first embodiment includes an integrated videophone device in which a camera and a display screen are integrated, a set-top type videophone device in which the display device and the camera are connected by a cable, and a mobile phone with a camera. It can be applied to any form of telephone. In particular, when the display screen of the camera-equipped mobile phone or the like is small, the advertisement image may be difficult to see. Therefore, by using the present invention, the advertisement image can be displayed more easily by cutting out only the face of the partner image, adjusting the size, and adjusting the arrangement. This state is illustrated using FIG. 9 and FIG. That is, FIG. 9 shows a state where the partner image is displayed in a large size and the advertisement image is difficult to see as in a normal mobile phone, and in FIG. 10, the partner image is reduced and moved to the lower right of the screen, so that the advertisement image is easy to see. is doing.

  As described above, if the videophone system according to the present embodiment is used, the user can view the advertisement image at the same time while performing the normal viewing communication in which the user talks while watching the other party image, thereby preventing the videophone call. It is possible to provide an advertisement service that provides an advertisement image without any problem.

  In addition, the billing management server can calculate the advertisement usage fee for each videophone user by transmitting data related to the advertisement usage status from the videophone device to the billing management server on the network. Using the calculated advertisement usage fee information, it is possible to provide a videophone service in which the advertising provider bears the videophone call fee in association with the billing cost associated with the user's call.

  As described in detail above, in the videophone system according to the present invention, the arrangement data included in the advertisement data includes the partner image transmitted from the partner videophone device and the advertisement image of the advertisement data downloaded from the advertisement management server. By providing the image composition means for compositing and displaying on the same screen of the person's videophone device based on the above, the user can simultaneously view the advertisement image while performing normal viewing communication in which he / she talks while viewing the other party's image. be able to.

  Therefore, the advertisement image is not limited to the advertisement information, and image data related to the other party can be used to obtain more detailed information specific to the other party while talking to the other party. For example, if a conference material or the like is converted into image data and stored in advance in a server corresponding to an advertisement management server, a special effect can be obtained when the videophone system according to the present invention is used as a videophone conference system. Can do.

It is a block diagram which shows the structure of the whole videophone system which concerns on Example 1 of this invention. It is a block diagram which shows the structure of the video telephone apparatus which concerns on Example 1 of this invention. It is a figure which shows the data structure of the advertisement data which concern on Example 1 of this invention. It is a sequence diagram which shows operation | movement of the videophone system which concerns on Example 1 of this invention. It is a figure which shows the data structure of advertisement viewing-and-listening data based on Example 1 of this invention. It is a flowchart which shows the processing content of the video telephone apparatus which concerns on Example 1 of this invention. It is a figure which shows the example of a screen display of the video telephone apparatus which concerns on Example 1 of this invention. It is a figure which shows the example of a display which gave the change to the display of the other party image by the audio variable and the motion data in the screen display of the video telephone apparatus which concerns on Example 1 of this invention. In the videophone device according to Embodiment 1 of the present invention, it is a diagram showing a display example in which a partner image on a mobile phone is displayed large and an advertisement image is difficult to see. It is a figure which shows the example of a display which made the other party image with a mobile phone small displayed in order to make an advertisement image legible in the video telephone apparatus concerning Example 1 of the present invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Originating side video telephone apparatus 2 Incoming side video telephone apparatus 3 Advertisement management server 3a Advertisement database 4 Billing management server 5 Network

Claims (2)

In a videophone system in which a plurality of videophone devices and an advertisement management server are respectively connected via a network,
The advertisement management server
A plurality of advertisement images to be displayed on the monitor, each of the plurality of video telephone apparatus has, been implicated for each advertisement image, layout data of the partner image is an image of the other party, the scaling value of the counterpart image, and the have a advertisement database in which motion data storing advertisement data contained for changing the display of the other party image,
Each of the plurality of videophone devices is
Storage means for storing advertisement data downloaded from the advertisement database of the advertisement management server;
From the captured image by the camera, an image extraction means for extracting the principal image image by separating the person and background images parties,
And voice processing means for detecting a speech variable level values and the frequency values of the principal sound entering the personal audio microphone or found before Symbol party,
A person image extracted by the image extracting means, encoding means for a personal voice to sign-reduction, respectively it input from the microphone,
Sends a coded data respectively to the television telephone apparatus of the communication partner are multiplexed by the encoding means, receiving the multiplexed encoded data transmitted from the video telephone of the calling party And data transmission / reception means for demultiplexing,
Decoding means for decoding the encoded data demultiplexed by the data transmitting / receiving means;
And the mating image obtained by decoding by the decoding means, and an advertisement image contained in the stored advertisement data in the storage means, corresponding to the detected speech variables by the speech processing unit, the advertisement data based on the arrangement data, the scaling value, and the motion data included in the arrangement position of the other party image, Ru is displayed size, and the monitor are combined on the same screen by changing at least one of the angles Image composition means,
A videophone system characterized by comprising. A plurality of advertisement images, the arrangement data of the other party image that is the image of the other party of the call, the scaling value of the other party image, and the movement data for changing the display of the other party image are included. Storage means for storing the advertisement data downloaded via the network from the advertisement management server storing the advertisement data;
An image extracting means for separating a caller's identity image and a background image from the image captured by the camera and extracting the identity image;
Voice processing means for inputting the voice of the caller from a microphone and detecting a level value and a frequency value of the voice of the caller as voice variables;
Encoding means for encoding the principal image extracted by the image extraction means and the principal voice input from the microphone;
The encoded data encoded by the encoding means is multiplexed and transmitted to the videophone device of the other party, and the multiplexed encoded data transmitted from the videophone device of the other party is received. Data transmission / reception means for demultiplexing,
Decoding means for decoding the encoded data demultiplexed by the data transmitting / receiving means;
The other party image obtained by decoding by the decoding unit and the advertisement image included in the advertisement data stored in the storage unit are associated with the audio variable detected by the audio processing unit, and the advertisement data Based on the arrangement data, the scaling value, and the movement data included in the image, an image to be combined and displayed on the same screen by changing at least one of the arrangement position, size, and angle of the counterpart image Combining means,
A videophone device characterized by comprising.

Images