JP2013201715A - Portable terminal, control method for portable terminal, program, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a portable terminal, executing a video telephone function, capable of suppressing wasteful current consumption (power consumption).SOLUTION: The portable terminal 10 includes a control section 50 for stopping at least one block of an image decoder 27, an image output section 28, an image encoder 29, an image input section 30, an LCD 31 and a camera 32 when a face recognition determination section 40 determines that a face is not included in at least either one of an opposite image and a self-portrait.

Description

  The present invention relates to a portable terminal that transmits and receives an image stream between a terminal and a partner terminal, a method for controlling the portable terminal, a program, and a recording medium.

  In recent years, with the widespread use of mobile terminals equipped with a camera, a function of using an image display means and a camera provided in the terminal has become common. Since these terminals are generally driven by the power of the secondary battery, techniques for suppressing current consumption or power consumption have been developed.

  In Patent Literature 1, a predetermined distance range is imaged by a camera, whether or not there is a face image within the predetermined distance range, and if the face image cannot be detected for a predetermined time or more, the user is not looking at the display unit. A technique for performing power saving control by determining and turning off the power of the display unit by the display control unit is described.

  Japanese Patent Application Laid-Open No. 2004-151867 discloses that even if power consumption is suppressed based on information on whether or not a video is recorded in a video camera, information on whether or not a face detection function is enabled, and information on whether or not a face is detected. A technique for controlling so as to reduce power consumption when it is determined that there is no problem is described.

  In Patent Document 3, when making a call using a portable terminal, the voice call mode and the videophone mode are automatically mistaken by reliably determining whether the portable terminal is near the user's ear. The technology to switch without any is described.

Japanese Patent Laid-Open No. 11-242733 (published September 7, 1999) JP 2010-272998 A (released on December 2, 2010) JP 2009-4959 A (published January 8, 2009)

  The techniques described in Patent Documents 1 and 2 are intended to save power in a single system that uses an image display unit and a camera that is an imaging unit. In other words, it is not considered to save power in a situation where there is a communication partner such as a telephone and a videophone.

  The technique described in Patent Document 3 has a problem of automatically switching between the voice call mode and the videophone mode without fail, and is not intended to save power of the mobile terminal.

  In order to execute a bidirectional communication function using an image display unit typified by a videophone and a camera using a mobile terminal, it is necessary to operate at least the image display unit, the imaging unit, the image encoder, and the image decoder. is there. Therefore, a large amount of current (power) is required to execute the bidirectional communication function using the image display unit and the camera, and the current consumption (power consumption) is several tens to several hundreds mA at the battery end. It becomes an order. A technology that can reduce such current consumption as much as possible is now required.

  The present invention has been made in view of the above problems. An object of the present invention is to provide a portable terminal capable of suppressing wasteful current consumption (power consumption) and a method for controlling the portable terminal when executing a bidirectional communication function using an image display unit and a camera. That is.

  In order to solve the above problems, a mobile terminal according to an aspect of the present invention includes an image display unit, an imaging unit that captures a direction in which the image display unit is facing, and a partner image stream transmitted by the partner terminal. Receiving means for receiving, image decoding means for decoding the counterpart image stream into counterpart image data, image output means for outputting the counterpart image data to the image display means, and self-image data via the imaging means. An image input means for inputting, an image encoding means for encoding the self-image data into a self-image stream, a transmission means for transmitting the self-image stream to the counterpart terminal, and a counterpart image represented by the counterpart image data includes a face. Face recognition determination means for determining whether or not a self-portrait represented by the self-image data includes a face, and the face recognition determination When the stage determines that at least one of the partner image and the self-portrait does not include a face, at least one of the image display unit, the image decoding unit, the imaging unit, and the image encoding unit And stop means for stopping the one means.

  According to the portable terminal configured as described above, bidirectional communication can be performed while looking at each other's face by transmitting and receiving an image stream to and from the partner terminal. At this time, the face recognition determining means determines whether or not a face is included in the partner image and the self-portrait. When the face recognition determination unit determines that at least one of the partner image and the self-portrait does not include a face, the stop unit is configured to display the image display unit or the image output unit according to the determination result. At least one of the image decoding unit, the imaging unit, the image input unit, and the image encoding unit is stopped.

  In the portable terminal, the imaging unit images a direction in which the image display unit is facing. If a face is included in the self-portrait, it means that the user of the portable terminal is facing the imaging means, that is, the user is looking at the image display means.

  On the other hand, if the self-portrait does not include a face, the user is not within the range imaged by the imaging means, or turned away from the imaging means. That is, the user does not look at the image display means. Thus, when the self-portrait does not include a face, in other words, when the user is not looking at the image display means, for example, the image output means and the image display means are not operated as usual. Meaning.

  Similarly, if the partner image does not include a face, the user of the partner terminal does not see the image represented by the self-portrait stream transmitted from the mobile terminal to the partner terminal. Therefore, when the image does not include a face, for example, it is meaningless to operate the image encoding means as usual.

  In the portable terminal, when at least one of the partner image and the self-portrait does not include a face, the stop unit stops means that is meaningless to operate. Therefore, useless current consumption (power consumption) can be suppressed.

  In addition, when the face recognition determination unit determines that the self-portrait does not include a face and the counterpart image includes a face, the stop unit includes the mobile terminal according to an aspect of the present invention. The structure which stops the said image display means may be sufficient.

  According to the above configuration, when the face recognition determination unit determines that the self image does not include a face and the partner image includes a face, the stop unit displays the image display unit. To stop. The fact that no face is included in the self-portrait data means that the user of the portable terminal is not within the range imaged by the imaging means or is facing away from the imaging means. That is, the user does not look at the image display means. For this reason, it is meaningless to operate the image display means.

  Therefore, when the stop means stops the image display means, the self-image data does not include its own face, and the other-person image data includes a face of the other party. Current (power consumption) can be suppressed.

  Note that the face recognition determining means continues to determine whether or not a face is included in the counterpart image due to the operation of the decoding means. Therefore, when the face is no longer detected in the partner image, the stopping means can detect that the state of the mobile terminal has transitioned.

  The mobile terminal according to an aspect of the present invention may be configured such that the stop unit stops the image decoding unit in addition to the image display unit.

  According to the above configuration, when the face recognition determination unit determines that no face is included in the self-portrait and the other party's face is included in the counterpart image, the stop unit includes the image In addition to the display means, the image decoding means is stopped. The fact that the self-portrait does not include a face means that the user of the portable terminal is not looking at the image display means. Therefore, even if the stop means stops the image decoding means, There is no hindrance to communication.

  By stopping both the image display means and the image decoding means, a wasteful current consumption (when a face is not included in the self-image data and a face is included in the counterpart image data ( (Power consumption) can be further suppressed.

  In the mobile terminal according to one aspect of the present invention, when the face recognition determination unit determines that the self-portrait includes a face and the partner image does not include a face, the stop unit includes The image encoding unit may be stopped.

  According to the above configuration, when the face recognition determination unit determines that a face is included in the self-portrait and a face is not included in the counterpart image, the stop unit includes the image encoding unit. To stop. When the partner image does not include a face, the user of the partner terminal does not see the self-portrait represented by the self-portrait stream transmitted by the mobile terminal. Therefore, it is meaningless to operate the image encoding means as usual when no face is included in the counterpart image. In other words, even if the stop means stops the image encoding means, no trouble occurs in the bidirectional communication. Therefore, by stopping the image encoding means, it is possible to suppress unnecessary current consumption (power consumption) when a face is included in the self-image data and a face is not included in the counterpart image data. It is.

  Note that the imaging means and the image input means operate when the self-portrait includes a face and the counterpart image does not include a face. That is, the face recognition determination means continues to determine whether or not a face is included in the self-portrait. Therefore, when a face is no longer detected in the self-portrait, the stop means can detect that the state of the mobile terminal has transitioned.

  The mobile terminal according to an aspect of the present invention may be configured such that the stop unit stops the imaging unit in addition to the image encoding unit.

  According to the above configuration, when the face recognition determination unit determines that the self-portrait includes a face and the partner image does not include a face, the stop unit includes the image encoding unit. In addition, the imaging means is stopped. The fact that no face is included in the counterpart image is a situation in which the user of the counterpart terminal does not see the image represented by the self-portrait stream, so even if the stop means stops the imaging means, There is no problem with bidirectional communication.

  By stopping both the image encoding means and the imaging means, a wasteful current consumption (power consumption) when the face is included in the self-image and the face is not included in the counterpart image is obtained. Further suppression is possible.

  In addition, when the face recognition determination unit determines that the face does not include a face in the self-portrait and the face in the partner image does not include the face, However, the structure which stops the said image coding means may be sufficient.

  According to the above configuration, when the face recognition determination unit determines that the self-portrait does not include a face and the partner image does not include a face, the stop unit performs the image encoding. Stop means. The fact that both the self-portrait data and the counterpart image data do not include a face means that the user of the portable terminal is not within the imaging range of the imaging means, or is facing away from the imaging means. Furthermore, the user of the counterpart terminal is in a situation where the imaging means provided in the counterpart terminal is not within the imaging range or the face is turned away from the imaging means provided in the counterpart terminal. For this reason, even if the stop means stops the image encoding means, the bidirectional communication is not hindered.

  By stopping the image encoding means, it is possible to suppress useless current consumption (power consumption) when no face is included in the self-portrait and no face is included in the counterpart image.

  In addition, when the face recognition determination unit determines that the face does not include a face in the self-portrait and the face in the partner image does not include the face, However, the structure which stops the said image display means may be sufficient.

  According to the above configuration, when the face recognition determination unit determines that the self-portrait does not include the user's face and the partner image does not include the partner's face, the stop unit includes The image display means is stopped. The fact that both the self-portrait data and the counterpart image data do not include a face means that the user of the mobile terminal is not within the range imaged by the imaging means, or is facing away from the imaging means, Furthermore, the user of the partner terminal is in a situation where the image capturing unit included in the partner terminal is not within the range of imaging, or the user is facing away from the image capturing unit included in the partner terminal. For this reason, even if the stop means stops the image display means, the two-way communication is not hindered.

  By stopping the image display means, it is possible to suppress unnecessary current consumption (power consumption) when the self-portrait does not include a face and the counterpart image does not include a face.

  The mobile terminal according to an aspect of the present invention may be configured such that the stop unit stops the imaging unit when the receiving unit does not receive the partner image stream.

  According to said structure, if the said receiving means with which the said portable terminal is provided does not receive the said other party image stream, the said stop means will stop the said imaging means. Therefore, wasteful current consumption (power consumption) when the partner image stream is not received can be suppressed.

  The mobile terminal according to an aspect of the present invention may be configured such that the stop unit stops the image decoding unit when the receiving unit does not receive the partner image stream.

  According to said structure, if the said receiving means with which the said portable terminal is provided does not receive the said other party image stream, the said stop means will stop the said image decoding means. Therefore, wasteful current consumption (power consumption) when the partner image stream is not received can be suppressed.

  The mobile terminal according to an aspect of the present invention may be configured such that the stop unit stops the image display unit in addition to the image decoding unit.

  According to the above configuration, when the receiving unit does not receive the partner image stream, the stopping unit stops the image display unit in addition to the image decoding unit. Therefore, wasteful current consumption (power consumption) when not receiving the partner image stream can be further suppressed.

  In the mobile terminal according to one aspect of the present invention, the receiving unit may receive the partner image stream via a wireless communication line, and the transmitting unit may transmit the own image stream via a wireless communication line. Good.

  According to said structure, even if it does not connect the wiring for connecting the said portable terminal and a communication line, a portable terminal can receive an other party image stream.

  In addition, the portable terminal according to one aspect of the present invention provides the face recognition determination unit when at least one of the image display unit, the image decoding unit, the imaging unit, and the image encoding unit is stopped. However, when it is determined that a new face is included in the self-portrait, or when it is determined that a face is newly included in the counterpart image, a restoration unit includes the image display unit, the image decoding unit, The configuration may be such that all the imaging means and the image encoding means are operated.

  According to the above configuration, when at least one of the image display unit, the image decoding unit, the imaging unit, and the image encoding unit included in the mobile terminal is stopped, the face recognition determination unit is When it is determined that the face of the subject is newly included in the self-portrait, or when it is determined that the face of the other party is newly included in the counterpart image, the stop means is the image display means, the image decoding All of the converting means, the imaging means, and the image encoding means are operated.

  Therefore, the user of the mobile terminal does not perform any special operation, and the mobile terminal is moved from a state where at least one of the image display unit, the image decoding unit, the imaging unit, and the image encoding unit is stopped. , All the means can be returned to the operating state.

  The mobile terminal control method according to an aspect of the present invention includes an image display step, an imaging step of capturing a direction in which an image is displayed in the image display step, and a reception step of receiving a partner image stream transmitted by the partner terminal. An image decoding step of decoding the counterpart image stream into the counterpart image data, an image output step of outputting the counterpart image data to the image display step, and an image input step of inputting the self-image data through the imaging step An image encoding step of encoding the self-image data into a self-image stream, a transmission step of transmitting the self-image stream to the counterpart terminal, and whether or not a face is included in the counterpart image represented by the counterpart image data. And a face recognition determination step for determining whether or not a face is included in the self-portrait represented by the self-image data, and the face recognition determination step. When it is determined that at least one of the counterpart image and the self-image does not include a face, at least one of the image display step, the image decoding step, the imaging step, and the image encoding step is performed. And a stop step for stopping.

  According to the control method of the mobile terminal configured as described above, the same effect as that of the mobile terminal can be obtained.

  In addition, a program for operating a computer as each unit included in the mobile terminal according to one embodiment of the present invention and a computer-readable recording medium on which the program is recorded are also included in the scope of the present invention.

  According to the mobile terminal of one aspect of the present invention, it is possible to reduce wasteful current consumption (power consumption) when bidirectionally communicating images.

It is a block diagram which shows the outline of a structure of the portable terminal which concerns on one Embodiment of this invention. It is a figure which shows the correspondence table of the process which the portable terminal which concerns on one Embodiment of this invention performs according to the presence or absence of other party image data, and the determination result of the face recognition determination part. It is a figure which shows the flowchart when the portable terminal which concerns on one Embodiment of this invention is a normal operation state. FIG. 6 is a block diagram illustrating an example of processing performed by a control unit when there is no face in the self-image data and there is a face in the partner image data in the embodiment of the present invention. FIG. 6 is a block diagram illustrating an example of processing performed by a control unit when there is no face in the self-image data and there is a face in the partner image data in the embodiment of the present invention. FIG. 5 is a block diagram illustrating an example of processing performed by a control unit when a self-image data has a face and a counterpart image data has no face in an embodiment of the present invention. FIG. 5 is a block diagram illustrating an example of processing performed by a control unit when a self-image data has a face and a counterpart image data has no face in an embodiment of the present invention. FIG. 6 is a block diagram illustrating processing performed by a control unit when there is no face in the self-image data and there is no face in the partner image data in the embodiment of the present invention. In one Embodiment of this invention, when there is no other party image data, it is a block diagram which shows an example of the process which a control part performs.

  A mobile terminal 10 according to an embodiment of the present invention will be described below with reference to FIGS. The type of the mobile terminal 10 according to the present embodiment is not limited as long as it is a device capable of transmitting and receiving an image / audio stream via a wireless communication line. Examples of typical portable terminals include smartphones, feature phones, tablet terminals, and the like.

  In the present embodiment, the mobile terminal 10 is also expressed as its own terminal. The portable terminal 10 transmits / receives an audio / video stream to / from a terminal (hereinafter also referred to as a partner terminal) separate from the terminal itself via a wireless communication line. In other words, the mobile terminal 10 and the partner terminal perform two-way communication, thereby realizing that the user of the own terminal and the user of the partner terminal communicate while looking at each other's face as an image.

  As an application of communication performed by the user of the own terminal and the user of the partner terminal while looking at each other's face as an image, for example, a video phone can be cited. In the present embodiment, a videophone will be described below as an example. However, the mobile terminal 10 only needs to perform at least bidirectional communication by transmitting and receiving an image stream. In other words, bidirectional communication by transmitting and receiving an audio stream in the mobile terminal 10 is not an essential component.

  For example, an MPEG2 stream or an MPEG4 stream may be used as an audio / video stream transmitted / received between the own terminal and the partner terminal.

(Configuration of mobile terminal 10)
The configuration of the mobile terminal 10 will be described with reference to FIG. FIG. 1 is a block diagram showing an outline of the configuration of the mobile terminal 10. As shown in this figure, the mobile terminal 10 includes a wireless processing unit 15, an image / sound processing unit 20, a liquid crystal display (LCD) 31, a camera 32, a speaker 33, a microphone 34, a face recognition determination unit 40, a control unit 50, and a storage. Part 60 is provided.

(Wireless processor 15)
The wireless processing unit 15 serving as a receiving unit and a transmitting unit receives an image / audio stream (hereinafter also referred to as a partner terminal stream) output from the partner terminal via a wireless communication line. Then, the partner terminal stream is output to the audio / video processing unit 20 described later. On the other hand, the wireless processing unit 15 receives an image / audio stream of its own terminal (hereinafter also referred to as its own terminal stream) from the audio / video processing unit 20 described later, and transmits the own terminal stream to the wireless communication line.

  As the wireless communication line, for example, an existing wireless communication line such as a cellular phone communication network, a wireless local area network (LAN), and a wireless Internet connection such as a public wireless line can be used.

(Image / Audio Processing Unit 20)
As shown in FIG. 1, the audio / video processing unit 20 includes an input frame processing unit 21, an audio decoder 22, a PCM output unit 23, an audio encoder 24, a PCM input unit 25, an output frame processing unit 26, an image decoder 27, and an image output unit. 28, an image encoder 29 and an image input unit 30 are provided.

  When the user of the mobile terminal 10 (hereinafter also referred to as user A) selects a specific partner terminal and activates the videophone function provided in the mobile terminal 10, bidirectional communication described below is executed. The In addition, activation of the videophone function of the mobile terminal 10 may be activated by a person other than the user A. For example, a user of a terminal separate from the mobile terminal 10 (hereinafter also referred to as a user B) requests communication using the videophone function from the mobile terminal 10 owned by the user A. Also good. In that case, the mobile terminal 10 notifies the user A that there is the request. When the user A performs an operation for approving the request to the mobile terminal 10, the videophone function provided in the mobile terminal 10 is activated.

(Receiver processing)
The input frame processing unit 21 separates the partner terminal stream received from the wireless processing unit 15 into a partner voice stream and a partner image stream. After that, the partner audio stream is output to the audio decoder 22, and the partner image stream is output to the image decoder 27.

  The audio decoder 22 decodes the other party audio stream into the other party audio data, and outputs it to the PCM output unit 23. The PCM output unit 23 converts the partner voice data into an electrical signal and outputs the electrical signal to the speaker 33. As a result, the user A can hear the voice of the user B through the speaker 33.

  The image decoder 27 is image decoding means for decoding the counterpart image stream. The image decoder 27 decodes the partner image stream into partner image data, and outputs the decoded image data to the image output unit 28 and a face recognition determination unit 40 described later.

  The image output unit 28 appropriately performs video processing on the supplied video data as necessary to convert it into RGB signals. After that, the image output unit 28 outputs the RGB signal to the LCD 31. A signal output from the image output unit 28 is not limited to an RGB signal, and a signal suitable for the LCD 31 is output. For example, when the LCD 31 includes four subpixels composed of RGBY, the image output unit 28 may be configured to output RGBY signals to the LCD 31.

  The LCD 31 as image output means displays the RGB signals as an image. As a result, the user A can see an image captured by the camera provided in the counterpart terminal.

  In this way, the user A of the mobile terminal 10 can view the voice and image of the counterpart terminal without significant delay, that is, in real time.

  Note that the image output means included in the mobile terminal 10 is not limited to the LCD. As image output means other than the LCD, for example, image output means using organic EL (electroluminescence) or electronic ink may be used. In the present embodiment, description will be made assuming that the image output means is an LCD.

  The process described so far, that is, the process of receiving the partner terminal stream and outputting the image and the sound from the LCD 31 and the speaker 33 of the mobile terminal 10 will be referred to as the reception side process of the own terminal or simply the reception side process in the following Express.

(Sender processing)
In addition to the above-described reception-side processing, the mobile terminal 10 performs transmission-side processing (hereinafter also simply referred to as “transmission-side processing”) of its own terminal. Hereinafter, the transmission side process will be described with reference to FIG.

  The portable terminal 10 includes a camera 32 and a microphone 34 that are imaging means. The camera 32 outputs the captured image to the image input unit 30 as an RGB signal.

  The image input unit 30 appropriately performs video processing on the supplied RGB signals as necessary to convert the RGB signals into self-image data. Further, the image input unit 30 outputs the self-portrait data to the image encoder 29 and a face recognition determination unit 40 described later.

  The image encoder 29 that is an image encoding means encodes the self-image data into a self-image stream and outputs it to the output frame processing unit 26.

  The PCM input unit 25 captures a voice uttered by the user A through the microphone 34 as an electric signal, converts it into own voice data, and outputs it to the voice encoder 24. The audio encoder 24 encodes the own audio data into the own audio stream. Further, the audio encoder 24 outputs the own audio stream to the output frame processing unit 26.

  The output frame processing unit 26 generates one self-terminal stream by combining the self-image stream and the self-audio stream, and outputs the stream to the wireless processing unit 15.

  The wireless processing unit 15 transmits the terminal stream to the wireless communication line. When the partner terminal receives the own terminal stream via the wireless communication line, the user B can view the voice and video of the user A in real time.

  When the mobile terminal 10 executes the above-described reception-side processing and transmission-side processing, the mobile phone function included in the mobile terminal 10 is realized.

(Face included in image)
Compared to voice-only communication, the major advantage of videophone is that communication is possible while acquiring image information such as the other party's facial expression and gestures. Therefore, it is preferable that the partner image represented by the partner image data includes at least the face of any user of the partner terminal. Similarly, the self-portrait represented by the self-image data preferably includes at least the face of the user of the mobile terminal 10. In the following, the partner image represented by the partner image data is also simply expressed as the partner image, and the host image represented by the host image data is also simply expressed as the host image.

  Here, the form of communication in the videophone is not necessarily limited to one person. That is, there is a possibility that the communication form is one-to-multiple or multiple-to-multiple. Therefore, the face of the user included in the partner image is not necessarily the face of the user B, and may not necessarily be the face of one user. For example, there is a possibility that a user other than the user B (herein referred to as a user C) is present. At this time, only the face of the user B may be included in the partner image, or only the face of the user C may be included, or the faces of the users B and C may be included. Sometimes. In other words, the partner image data preferably includes at least one of the faces of the users near the partner terminal, and may include a plurality of user faces.

  The face included in the self-portrait is the same as the face included in the partner image. It is assumed that there are a plurality of users in the vicinity of the mobile terminal 10. Therefore, the self-portrait preferably includes at least one of the faces of the users in the vicinity of the mobile terminal 10, and may include a plurality of user faces.

  In the present embodiment, for simplicity of explanation, it is assumed that the user of the mobile terminal 10 is the user A and the user of the other terminal is the user B unless otherwise specified. .

(When face is not included in image data)
As described above, in the form of communication using a videophone, it is preferable that the face of the user is included in both the partner image and the self-portrait. However, during the communication using the videophone, there is a possibility that the user's face is not included in one or both of the self-portrait and the other-party image.

  For example, when the user B moves from the front surface of the counterpart terminal to a different location, the counterpart image does not include the face of the user B. Even when the user B is in front of the partner terminal, the face of the user B is not included in the partner image by turning away from the camera of the partner terminal.

  Similarly, when the user A moves from the front surface of the mobile terminal 10 to a different place, the self-portrait does not include the face of the user A. Even when the user A is in front of the mobile terminal 10, the face of the user A is not included in the self-portrait by turning away from the camera 32.

  Here, the current consumption (power consumption) when the videophone function is executed will be described. In order to execute the videophone function, the mobile terminal 10 is required to operate more means than a voice-only telephone. Specifically, the mobile terminal 10 transmits the self-image stream to the counterpart terminal and receives the counterpart image stream transmitted by the counterpart terminal. For this purpose, the portable terminal 10 operates each block of the image decoder 27, the image output unit 28, the image encoder 29, the image input unit 30, the LCD 31, and the camera 32.

  Even if the user's face is not included in one or both of the self-image data and the partner image data, operating all the blocks in the mobile terminal 10 increases useless current consumption (power consumption). It means that

  In order to suppress wasteful current consumption (power consumption) as much as possible, the mobile terminal 10 includes one of the above blocks when the user's face is not included in one or both of the self-portrait and the partner image. Stop blocks that do not make sense. That is, the stop factor is that the user's face is not included in one or both of the self image and the partner image.

(Block to stop)
In the present embodiment, among the image decoder 27, the image output unit 28, the image encoder 29, the image input unit 30, the LCD 31, and the camera 32, the image output unit 28 and the LCD 31 are regarded as one block unit. Similarly, the image input unit 30 and the camera 32 are regarded as one block unit.

  The image output unit 28 is means for supplying RGB signals to the LCD 31. Therefore, it is preferable that the image output unit 28 is operating when the LCD 31 is operating, and it is preferable that the image output unit 28 is not operated when the LCD 31 is stopped.

  The image input unit 30 is means for converting RGB signals supplied from the camera 32 into self-image data. Therefore, it is preferable that the image input unit 30 is also operating when the camera 32 is operating, and it is preferable that the image input unit 30 is not operated when the camera 32 is stopped.

Therefore, the blocks that can be stopped in this embodiment are as follows.
(A) Image output unit 28 and LCD 31
(B) Image decoder 27
(C) Image encoder 29
(D) Image input unit 30 and camera 32
When the user's face is not included in one or both of the self-portrait and the partner image, that is, when any stop factor exists, any one of these (a) to (d), or By stopping the plurality, the mobile terminal 10 suppresses unnecessary current consumption (power consumption).

  When a plurality of (a) to (d) are selected and stopped, if the control unit 50 receives a return factor and the portable terminal 10 can return to the standard state, (a) to (d) The designer can arbitrarily design how to combine them at the design stage. The return factor will be described later.

(Face recognition determination unit 40)
The face recognition determination unit 40 that functions as a face recognition determination unit determines whether or not the face of the user A is included in the self-image represented by the self-image data received from the image input unit 30 by performing face recognition processing. As a face recognition processing method performed in the face recognition determination unit 40, a known face recognition processing method may be used.

  The face recognition determination unit 40 supplies the control unit 50 with information of the result of the face recognition process, that is, “the face is recognized” or “the face is not recognized” in the self-portrait.

  Similarly, the face recognition determination unit 40 determines whether or not the face of the user B is included in the partner image represented by the partner image data received from the image decoder 27 by performing face recognition processing. The face recognition determination unit 40 supplies the control unit 50 with information on the result of the face recognition process, that is, “the face is recognized” or “the face is not recognized” in the partner image.

  In the following, “recognizing a face in the image” is simply expressed as “a face is in the image”. Similarly, “the face is not recognized in the image” is simply expressed as “the face is not in the image”.

(Control unit 50)
The control unit 50 serving as a stop unit and a return unit receives the result of face recognition processing from the face recognition determination unit 40 for the self-image data and the partner image data. The control unit 50 controls each block included in the audio / video processing unit 20 based on the result of the face recognition process received from the knowledge determination unit 40.

  Specifically, when either one or both of the self image and the partner image does not have a face, the control unit 50 controls the image decoder 27, the image output unit 28, the image encoder 29, the image input unit 30, the LCD 31 and the camera. Any one or more of the 32 blocks are stopped. That is, the control unit 50 functions as a stopping unit by detecting a stop factor. At this time, which block the control unit 50 stops is described later.

  Note that a state in which any one or a plurality of blocks of each block is stopped is hereinafter referred to as a stopped state.

  For example, when the face of the user A is not included in the self-portrait, the user A may not be in front of the mobile terminal 10 or may be turning away from the camera 32 face. The control unit 50 detects that there is no face in the self-image as a stop factor, and any one of the blocks of the image decoder 27, the image output unit 28, the image encoder 29, the image input unit 30, the LCD 31 and the camera 32, Alternatively, a plurality of blocks are stopped.

  In this stop state, since the camera 32 and the image input unit 30 are performing normal processing, the face recognition determination unit 40 continues the face recognition processing based on the self-image data. In a period in which the face recognition determination unit 40 does not recognize a face in the self-portrait, the face recognition determination unit 40 continues to output to the control unit 50 a determination that “the face is not in the self-portrait”. That is, the stopped state is maintained.

  When the user A returns to the front surface of the mobile terminal 10 or faces the camera 32, the face recognition determination unit 40 that has not recognized the face in the self-image until then recognizes the face in the self-image. . As a result, the face recognition determination unit 40 determines that “the face is in the self-portrait”. In this way, the face recognition determination unit 40 determines that a face is recognized in an image that has not previously recognized a face as “a new face is recognized” or “a new face is included. It also expresses. In addition, when a face is newly recognized in the self-portrait or the partner image whose face has not been recognized by the face recognition determination unit 40 until then, this is called a return factor.

  When the face recognition determination unit 40 newly recognizes a face, the control unit 50 receives from the face recognition determination unit 40 a determination that “the face is recognized” as a return factor. Upon receiving the return factor, the control unit 50 operates all the blocks of the image decoder 27, the image output unit 28, the image encoder 29, the image input unit 30, the LCD 31, and the camera 32. The state in which all the blocks are operating as described above is also referred to as a normal state below.

  A predetermined delay time may be provided between the time when the control unit 50 detects the stop factor and the time when each block is stopped (shifted to the stop state). The manufacturer may set the delay time to an arbitrary length in the design stage. Moreover, the structure which the user can set the said delay time to arbitrary time may be sufficient so that it may be convenient according to a user's liking.

  The control unit 50 may include a CPU (Central Processing Unit) for realizing each block included in the mobile terminal 10 as software. The CPU executes a program for realizing the function of each block of the mobile terminal 10.

(Storage unit 60)
The portable terminal 10 includes a ROM (Read Only Memory) for storing the program, a RAM (Random Access Memory) for expanding the program, and a storage device (recording medium) such as a memory for storing the program and various data. May be provided. In the present embodiment, the ROM, RAM, and storage device (recording medium) described above are collectively referred to as a storage unit 60.

(Processing correspondence table)
The blocks that the control unit 50 stops will be described with reference to FIG. FIG. 2 is a diagram illustrating a correspondence table of stop processing performed by the control unit 50 in the mobile terminal 10.

  When the videophone function with the other party terminal is used by using the mobile terminal 10 that is the own terminal, there are cases where the own image has a face and no face and when the other image has a face and no face. In FIG. 2, “when there is a face” is described as “face is present”, and “when there is no face” is described as “no face”.

  As will be described in detail later, when the counterpart terminal is a mobile terminal according to an aspect of the present invention, the counterpart terminal may not output the counterpart image stream. This case will be referred to as “no partner image data” below.

  Therefore, the correspondence table shown in FIG. 2 describes the processing for six cases. The six cases are a combination of the two cases of “having a face” and “no face” for the own terminal and the three cases of “having a face”, “no face” and “no other image data” for the other terminal. There are six cases.

  In each cell in which processing in the above six cases is described, the upper subcell describes the transmission side processing of the own terminal, and the lower subcell describes the reception side processing of the own terminal.

  Regarding the processing in the portable terminal 10, the processing flow will be described first with reference to the flowchart shown in FIG. Next, blocks to be stopped in each of the six processes will be described with reference to the block diagrams shown in FIGS.

(Process flow)
The flow of processing in the portable terminal 10 is shown in the flowchart of FIG. The flowchart shown in FIG. 3 assumes that the counterpart terminal always outputs the counterpart image stream, that is, there is always counterpart image data. Also, the flowchart shown in FIG. 3 is a flowchart when performing the following stop process among the processes described in the correspondence table shown in FIG.
“Image encoder & camera stop” is performed as a transmission side process in the case where “the face is present” in the own image and “the face is absent” in the partner image.
“LCD & image decoder stop” is performed as a receiving side process when “no face” in the own image and “face present” in the partner image.

  The processing in the image / sound processor 20 does not necessarily have to be performed in the order shown in the flowchart of FIG. 3, and the order of some processes may be changed. The flowchart shown in FIG. 3 shows an example of the flow of processing in the audio / video processing unit 20.

  The face recognition determination unit 40 acquires an image captured by the camera 32 as self-portrait data via the image input unit 30 (S100).

  The face recognition determination unit 40 acquires partner image data from the image decoder 27 (S102). The face recognition determination unit 40 performs face recognition processing on the self-image represented by the self-image data, and determines whether or not the self-image has a face. Then, the determination result of “face present” or “face absent” is output to the control unit 50 (S104). If the determination result in S104 is “face present”, ie, YES, the process proceeds to S106, and if “no face”, ie, NO, the process proceeds to S108.

  In S <b> 106, the face recognition determination unit 40 outputs the determination result of “face present” or “no face” to the control unit 50 for the partner image represented by the partner image data. If the determination result in S106 is “face present”, ie, YES, the process proceeds to S110, and if “no face”, ie, NO, the process proceeds to S120.

  In S <b> 108, the face recognition determination unit 40 outputs to the control unit 50 the determination result of “face present” or “no face” for the partner image represented by the partner video data. If the determination result in S108 is “face present”, ie, YES, the process proceeds to S130, and if “no face”, ie, NO, the process proceeds to S140.

(Self-portrait: with face, partner image: with face)
As described above, the process proceeds to S <b> 110 when the subject image is “faced” and the partner image is “faced”. Therefore, the processing of S110 to S116 corresponds to the processing of “Own terminal has face” and “Opposite terminal has face” in FIG. In other words, the processing of S110 to S116 is processing when the mobile terminal 10 is in a normal state.

  The control unit 50 controls the audio / video processing unit 20 to perform camera encoding. More specifically, the control unit 50 outputs a control signal for operating the image encoder 29 to the image encoder 29 (S110). Note that performing camera encoding means encoding self-portrait data, which is image data captured by the camera 32, into a self-image stream.

  Next, the control unit 50 controls the audio / video processing unit 20 to perform camera processing. More specifically, the control unit 50 outputs a control signal for operating the camera 32 and the image input unit 30 to the camera 32 and the image input unit 30 (S112). Note that performing the camera process means a process of inputting an RGB signal of an image captured by the camera 32 to the image input unit 30, and the image input unit 30 converts the RGB signal into self-image data.

  S110 and S112 correspond to the transmission-side processing of “Own terminal has face” and “Opposite terminal has face” in FIG. Further, the image encoder 29, the image input unit 30, and the camera 32 operate as processing results of S110 and S112. This state is described as normal processing in FIG.

  Next, the control unit 50 controls the audio / video processing unit 20 to perform screen display. More specifically, the control unit 50 outputs a control signal for operating the image output unit 28 and the LCD 31 to the image output unit 28 and the LCD 31 (S114). Note that displaying an image means that the image output unit 28 converts the partner image data into an RGB signal and then outputs the RGB signal to the LCD 31, and the LCD 31 displays the RGB signal.

  Further, the control unit 50 controls the audio / video processing unit 20 to perform the partner image decoding. More specifically, the control unit 50 outputs a control signal for operating the image decoder 27 to the image decoder 27 (S116). Note that performing partner image decoding means that the image decoder 27 decodes the partner image stream into partner image data.

  S114 and S116 correspond to the processing on the receiving side in FIG. 2 that “Own terminal has face” and “Opposite terminal has face”. Further, the image decoder 27, the image output unit 28, and the LCD 31 operate as the processing results of S114 and S116. This state is described as normal display in FIG.

  As described above, the image encoder 29, the image input unit 30, the camera 32, which is responsible for the transmission side processing, and the image decoder, which is responsible for the reception side processing, when “face is present” in the own image and “face is present” in the counterpart image 27, the image output unit 28 and the LCD 31 are all in operation. Therefore, this state is a normal state in the mobile terminal 10.

(Self-portrait: with face, partner image: no face)
If the self-portrait is “face present” and the partner image is “no face”, the processing step proceeds to S120. Therefore, the processing of S120 to S126 corresponds to the processing of “Own terminal has face” and “Other party terminal has no face” in FIG.

  “No face” in the partner image may indicate that the user B is out of the front of the partner terminal or the user B is facing away from the camera face of the partner terminal. high. In such a situation, it is highly likely that the user B is not looking at the image display unit (for example, LCD) provided in the counterpart terminal.

  Therefore, there is no need to output the self-image stream to the counterpart terminal, and the terminal device 10 stops the image encoder 29 or stops the image encoder 29, the image input unit 30, and the camera 32.

  In this way, by stopping some or all of the blocks responsible for the transmission side processing, it is possible to suppress useless current consumption (power consumption) in a situation where the possibility that the user B does not see the self-portrait data is high. is there. Here, a process of stopping the image encoder 29, the image input unit 30, and the camera 32 will be described as S120 to S126.

  The control unit 50 controls the audio / video processing unit 20 to stop camera encoding. More specifically, the control unit 50 outputs a control signal for stopping the image encoder 29 to the image encoder 29 (S120). Next, the control unit 50 controls the audio / video processing unit 20 to stop the camera input. More specifically, the control unit 50 outputs a control signal for stopping the image input unit 30 and the camera 32 to the image input unit 30 and the camera 32 (S122). The control unit 50 controls the audio / video processing unit 20 to perform screen display in the same manner as S114 (S124). Further, the control unit 50 controls the audio / video processing unit 20 so as to perform the partner image decoding similarly to S116 (S126).

(Self-portrait: no face, partner image: face present)
If “no face” in the self-portrait and “face is present” in the counterpart image, the processing step proceeds to S130. Therefore, the processing of S130 to S136 corresponds to the processing of “the terminal is faceless” and “the partner terminal is faced” in FIG.

  In the self-portrait, “no face” means that the user A is out of the seat from the front of the mobile terminal 10 or the user A is facing away from the camera 32 included in the mobile terminal 10. Probability is high. In such a situation, the user A is highly likely not to look at the LCD 31. Therefore, it is not necessary to display the partner image on the LCD 31, and the terminal device 10 stops the image output unit 28 and the LCD 31, or stops the image output unit 28, the LCD 31, and the image decoder 27.

  In this way, by suspending some or all of the blocks responsible for receiving-side processing, wasteful current consumption (power consumption) is suppressed in situations where it is highly probable that user A is not viewing the partner image data. Is possible. Here, a process of stopping the image decoder 27, the image output unit 28, and the LCD 31 will be described as S130 to S136.

  The control unit 50 controls the audio / video processing unit 20 to perform camera encoding as in S110 (S130). In addition, the control unit 50 controls the audio / video processing unit 20 so as to perform camera processing in the same manner as in S112 (S132). Further, the control unit 50 controls the audio / video processing unit 20 to stop the screen display. More specifically, the control unit 50 outputs a control signal for stopping the image output unit 28 and the LCD 31 to the image output unit 28 and the LCD 31 (S134). Next, the control unit 50 controls the image / sound processing unit 20 to stop the counterpart image decoding. More specifically, the control unit 50 outputs a control signal for stopping the image decoder 27 to the image decoder 27 (S136).

(Self-portrait: no face, partner image: no face)
If “no face” in the self-portrait and “no face” in the partner image, the processing step proceeds to S140. Therefore, the processing of S140 to S146 corresponds to the processing of “the terminal is faceless” and “the partner terminal is faceless” in FIG.

  In the self-portrait, “faceless” is highly likely that the user A is not looking at the LCD 31 as described above, that is, not looking at the partner image. Also, “faceless” in the partner image is highly likely that the user B has not seen the image display unit provided in the partner terminal, that is, has not seen the self-portrait.

  Therefore, the portable terminal 10 does not need to transmit the self-portrait stream to the counterpart terminal, and stops the image encoder 29. In addition, since it is not necessary to display the partner image data on the LCD 31, the mobile terminal 10 stops the image output unit 28 and the LCD 31.

  Thus, by stopping a part of the transmission side process and a part of the reception side process, there is a possibility that the user B does not see the self-portrait and the user A does not see the partner image. In a high situation, useless current consumption (power consumption) can be suppressed.

  Note that it is preferable not to stop the image input unit 30 and the camera 32 in the transmission-side processing. By stopping the image input unit 30 and the camera 32, the face recognition determination unit 40 cannot obtain self-portrait data. This means that the face recognition determination unit 40 cannot perform face recognition processing on the self-portrait. In other words, even if the user A returns to the front of the portable terminal 10 or the user A faces the camera 32, the face recognition determination unit 40 cannot recognize the face of the user A as a return factor. Means that.

  Similarly, it is preferable not to stop the image decoder 27 in the reception side processing. By stopping the image decoder 27, the face recognition determination unit 40 cannot obtain partner image data. This means that the face recognition determination unit 40 cannot perform face recognition processing on the partner image. In other words, even if the user B returns to the front of the partner terminal or the user B faces the camera of the partner terminal, the face recognition determination unit 40 uses the face of the user B as a return factor. It means that it cannot be recognized.

  If the self-image data is “no face” and the partner image data is “no face”, the control unit 50 adds the image decoder 27, the image input in addition to the image output unit 28, the LCD 31 and the image encoder 29. Assume that the unit 30 and the camera 32 are stopped. In this case, the control unit 50 cannot detect the return factor at all. Therefore, the mobile terminal 10 cannot return from the stopped state to the normal state.

  For the above reasons, the control unit 50 stops the image output unit 28, the image encoder 29, and the LCD 31 as the processing of S140 to S146. This process will be described below.

  The control unit 50 controls the audio / video processing unit 20 to stop the camera encoding (image encoder 29) (S140). Next, the control unit 50 controls the audio / video processing unit 20 so as to perform camera processing in the same manner as in S112 (S142). Further, the control unit 50 controls the audio / video processing unit 20 to stop the screen display (the image output unit 28 and the LCD 31), similarly to S134 (S144). Next, the control unit 50 controls the audio / video processing unit 20 to perform the partner image decoding in the same manner as in S116 as in S116 (S146).

(Block where the control unit 50 stops)
Next, blocks to be stopped in each of the six processes shown in FIG. 2 will be described with reference to the block diagrams shown in FIGS. The block diagrams shown in FIGS. 4 to 9 are the same as the block diagram shown in FIG. 1, but the blocks where the control unit 50 stops are hatched.

  In the correspondence table of FIG. 2, blocks that are stopped when “the terminal is faceless” and “the partner terminal is faced” are shown in FIGS. 4 and 5. In the correspondence table of FIG. 2, the blocks to be stopped when “Own terminal has face” and “Other party terminal has no face” are shown in FIGS. In the correspondence table of FIG. 2, FIG. 8 shows blocks to be stopped when “the terminal is faceless” and “the partner terminal is faceless”. The above is the stop process that can be performed by the control unit 50 when the partner terminal stream includes the partner image stream as described above, in other words, when there is partner image data.

  In contrast to these stop processes, FIG. 9 shows a block that the control unit 50 stops when the partner terminal stream does not include the partner image stream, in other words, when there is no partner image data.

(Self-portrait: no face, partner image: face present)
A case where the determination result by the face recognition determination unit 40 is “no face” in the self-portrait and “face present” in the counterpart image will be described with reference to FIGS. 4 and 5.

  In this case, the control unit 50 performs normal processing as the transmission side processing. On the other hand, the control part 50 performs LCD stop as a receiving side process (refer FIG. 4). LCD stop here means stopping the image output unit 28 and the LCD 31.

  The merit of not stopping the image decoder 27 in FIG. 4 is that when no face is detected in the partner image, it is possible to transition to a stop state of “no face” in the subject image and “no face” in the partner image data. That is.

  Next, FIG. 5 shows a case where the control unit 50 executes LCD and image decoder stop as the receiving side processing. In addition to the image output unit 28 and the LCD 31 described above, the image decoder 27 is also stopped. By increasing the number of blocks to be stopped, the current consumption (power consumption) in the stopped state can be further suppressed as compared to the case shown in FIG.

  When the image decoder 27 is stopped, the face recognition determination unit 40 cannot obtain the partner image data when the image decoder 27 is stopped. Therefore, regardless of whether or not the face recognition determination unit 40 recognizes a face in the partner image, the control unit 50 determines “no face” in the self-portrait data and “face presence” in the partner image data until detecting the return factor. Keep the stop state corresponding to.

  In both the case where the LCD stop is executed as the stop processing and the case where the LCD and the image decoder are stopped, the return factor is to detect a face in the self-image data. In order to detect this return factor, the transmission side process performs a normal process.

(Self-portrait: with face, partner image: no face)
A case where the determination result by the face recognition determination unit 40 is “face present” in the self-portrait and “face-free” in the partner image will be described with reference to FIGS. 6 and 7.

  In this case, the control unit 50 performs normal display as a reception-side process. On the other hand, the control part 50 stops the image encoder 29 as a transmission side process (refer FIG. 6). By stopping the image encoder 29, current consumption (power consumption) when a face is not recognized in the partner image can be suppressed.

  Note that the image input unit 30 and the camera 32 are operating in the transmission-side processing. Therefore, the face recognition determination unit 40 can acquire the self-image data and can determine whether or not the self-image has a face. If a face cannot be detected from the self-image data, it is possible to transition to a stop state of “no face” in the self-image data and “no face” in the partner image data.

  On the other hand, the block diagram shown in FIG. 7 shows a state where the image input unit 30 and the camera 32 are stopped in addition to the image encoder 29. By stopping all the blocks related to the transmission side processing, it is possible to further suppress the current consumption (power consumption) in the stop state as compared with the stop state shown in FIG.

  In the stop state illustrated in FIG. 7, the face recognition determination unit 40 cannot acquire self-portrait data. Therefore, the control unit 50 maintains a stop state corresponding to “face present” in the self-image data and “face-free” in the partner image data until the return factor is detected.

  In both the case where the image encoder 29 is stopped as the stop state and the case where the image encoder 29, the image input unit 30 and the camera 32 are stopped, the return factor is to detect a face in the counterpart image. In order to detect this return factor, the receiving side process normally displays.

(Self-portrait: no face, partner image: no face)
A case where the determination result by the face recognition determination unit 40 is “no face” in the self-image data and “no face” in the partner image data will be described with reference to FIG.

  In this case, the control unit 50 stops the image output unit 28 and the LCD 31 among the blocks responsible for the reception side processing. Furthermore, the control part 50 stops the image encoder 29 among each block which bears a transmission side process.

  Thus, by stopping the image output unit 28, the image encoder 29, and the LCD 31, it is possible to suppress current consumption (power consumption) in the stopped state.

  On the other hand, the face recognition determination unit 40 can execute the face recognition process on the partner image by operating the image decoder 27 among the blocks responsible for the reception side process. Similarly, the face recognition determination unit 40 can execute the face recognition process in the self-image by operating the image input unit 30 and the camera 32 among the blocks responsible for the transmission side processing.

  Therefore, when the face recognition determination unit 40 newly recognizes a face in either the partner image or the self-portrait, the face recognition determination unit 40 outputs the recognition of the face to the control unit 50 as a return factor. That is, when the face recognition determination unit 40 newly detects a face in either the partner image or the self image, the mobile terminal 10 returns to the normal state.

(When there is no partner image data)
The stop states according to FIGS. 4 to 8 have been described on the assumption that the partner terminal always transmits the partner image stream. However, there may be a case where the partner image stream is not transmitted. For example, when the other terminal is the mobile terminal 10 as well as the own terminal, that is, when the other terminal stops the image encoder in the case of a videophone call between the mobile terminals 10. In other words, this is a case where the partner terminal does not transmit the partner image stream. The stop state when the partner image stream is not included in the partner terminal stream will be described below.

  The input frame processing unit 21 to which the partner terminal stream is supplied via the wireless processing unit 15 separates the partner terminal stream into a partner voice stream and a partner image stream. At this time, the input frame processing unit 21 detects whether there is a partner image stream. When the input frame processing unit 21 detects that there is no partner image stream, the input frame processing unit 21 outputs the detection result to the control unit 50.

  The control unit 50 receives the detection result and outputs a control signal for stopping the image decoder 27, the image output unit 28, and the LCD 31, which are all blocks responsible for the reception-side processing, to each of the above-described blocks. That is, as shown in FIG. 9, the control unit 50 stops the image decoder 27, the image output unit 28, and the LCD 31.

  Further, the control unit 50 may output a control signal for stopping an image encoder and a camera, which are blocks responsible for transmission side processing, to each of the above-described blocks. Stopping the camera means stopping the image input unit 30 and the camera 32 as before. That is, as shown in FIG. 9, the control unit 50 stops the image encoder 29, the image input unit 30, and the camera 32.

  In this way, since all the blocks that perform the reception-side processing and the blocks that perform the transmission-side processing are stopped, it is possible to suppress unnecessary current consumption (power consumption).

  In this stopped state, when the image decoder 27 detects the partner image stream from the input frame processing unit 21, the image decoder 27 resumes operation and outputs the partner image data to the face recognition determination unit 40. As a result, the face recognition determination unit 40 can determine whether or not there is a face in the partner image data.

  The control unit 50 determines transmission side processing and reception side processing based on the determination result of the face recognition determination unit 40 (see FIG. 2). Specifically, if the partner image has a face, normal processing is executed as the transmission side processing, and normal display, LCD stop, and LCD & image decoder stop are executed as the reception side processing. On the other hand, if there is no face in the partner image, the image encoder stop or image encoder & camera stop is executed as the transmission side process, and the normal display or LCD stop is executed as the reception side process.

  As a modification of the reception side process, only the image decoder 27 may be stopped. In this case, since the image output unit 28 and the LCD 31 are operating, the control unit 50 can display a message such as “no image” or “no other party image data” on the LCD 31 via the image output unit 28. The image data for displaying the message may be stored in the storage unit 60 in advance.

  As a modification on the transmission processing side, normal processing may be used instead of camera stop. When the transmission makes the process a normal process, the mobile terminal 10 outputs its own terminal stream including the own image stream. If the partner terminal is ready to receive the self-portrait stream, an image captured by the camera 32 of the mobile terminal 10 is displayed on the image display means of the partner terminal. In this way, by transmitting the self-portrait stream, it is possible to inform the user of the counterpart terminal of information such as whether or not there is a user in front of the mobile terminal 10.

(Additional notes)
Each block of the mobile terminal 10 described above may be realized in hardware by a logic circuit formed on an integrated circuit (IC chip), or may be realized in software using a CPU (Central Processing Unit). Also good.

  In the latter case, the apparatus includes a CPU that executes instructions of a program that realizes each function, a ROM (Read Only Memory) that stores the program, a RAM (Random Access Memory) that expands the program, the program, and various data. A storage device (recording medium) such as a memory for storing the. An object of the present invention is to provide a recording medium in which a program code (execution format program, intermediate code program, source program) of a control program for each device, which is software for realizing the functions described above, is recorded in a computer-readable manner. This can also be achieved by supplying to the apparatus and reading and executing the program code recorded on the recording medium by the computer (or CPU or MPU).

  Examples of the recording medium include tapes such as magnetic tapes and cassette tapes, magnetic disks such as floppy (registered trademark) disks / hard disks, and disks including optical disks such as CD-ROM / MO / MD / DVD / CD-R. IC cards (including memory cards) / optical cards, semiconductor memories such as mask ROM / EPROM / EEPROM / flash ROM, PLD (Programmable logic device), FPGA (Field Programmable Gate Array), etc. Logic circuits can be used.

  Further, each of the above devices may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited as long as it can transmit the program code. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication network, and the like can be used. The transmission medium constituting the communication network may be any medium that can transmit the program code, and is not limited to a specific configuration or type. For example, even with wired lines such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL (Asymmetric Digital Subscriber Line) line, infrared rays such as IrDA and remote control, Bluetooth (registered trademark), IEEE 802.11 wireless, HDR ( It can also be used by radio such as High Data Rate (NFC), Near Field Communication (NFC), Digital Living Network Alliance (DLNA), mobile phone network, satellite line, and digital terrestrial network.

  The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims, and embodiments obtained by appropriately combining technical means disclosed in different embodiments. Is also included in the technical scope of the present invention.

  INDUSTRIAL APPLICABILITY The present invention can be used as a portable terminal that transmits / receives an image stream between the own terminal and a partner terminal, and a method for controlling the portable terminal.

10 mobile terminal 15 wireless processing unit (reception means and transmission means)
20 Image / Audio Processing Unit 21 Input Frame Processing Unit 22 Audio Decoder 23 PCM Output Unit 24 Audio Encoder 25 PCM Input Unit 26 Output Frame Processing Unit 27 Image Decoder (Image Decoding Unit)
28 Image output unit (image output means)
29 Image encoder (image encoding means)
30 Image input unit (image input means)
31 LCD (image display means)
32 Camera (imaging means)
33 Speaker 34 Microphone 40 Face recognition determination unit (face recognition determination means)
50 Control unit (stopping means and returning means)
60 storage unit

Claims (15)

Image display means;
Imaging means for imaging the direction in which the image display means is facing;
Receiving means for receiving the partner image stream transmitted by the partner terminal;
Image decoding means for decoding the partner image stream into partner image data;
Image output means for outputting the counterpart image data to the image display means;
Image input means for inputting self-portrait data via the imaging means;
Image encoding means for encoding the self-image data into a self-image stream;
Transmitting means for transmitting the self-portrait stream to the counterpart terminal;
Face recognition determination means for determining whether or not a face is included in the partner image represented by the partner image data, and determining whether or not a face is included in the host image represented by the host image data;
When the face recognition determining means determines that at least one of the counterpart image and the self-portrait does not include a face, the image display means, the image decoding means, the imaging means, and the image encoding means A portable terminal comprising stop means for stopping at least one of them.   When the face recognition determining means determines that the self-portrait does not include a face and the partner image includes a face, the stop means stops the image display means. The mobile terminal according to claim 1.   The portable terminal according to claim 2, wherein the stopping unit stops the image decoding unit in addition to the image display unit.   When the face recognition determination unit determines that the self-portrait includes a face and the partner image does not include a face, the stop unit stops the image encoding unit. The portable terminal according to any one of claims 1 to 3.   5. The portable terminal according to claim 4, wherein the stopping unit stops the imaging unit in addition to the image encoding unit.   When the face recognition determining unit determines that the self-portrait does not include a face and the partner image does not include a face, the stop unit stops the image encoding unit. The mobile terminal according to any one of claims 1 to 5.   When the face recognition determination unit determines that the self-portrait does not include a face and the partner image does not include a face, the stop unit stops the image display unit. The portable terminal according to any one of claims 1 to 6.   The portable terminal according to any one of claims 1 to 7, wherein the stop unit stops the imaging unit when the receiving unit does not receive the partner image stream.   The mobile terminal according to any one of claims 1 to 8, wherein when the receiving unit does not receive the partner image stream, the stopping unit stops the image decoding unit.   The portable terminal according to claim 9, wherein the stopping unit stops the image display unit in addition to the image decoding unit.   The said receiving means receives an other party image stream via a wireless communication line, and the said transmission means transmits a self-image stream via a wireless communication line. Mobile device.   When at least one of the image display unit, the image decoding unit, the imaging unit, and the image encoding unit is stopped, the face recognition determination unit includes a new face in the self-image. Or when it is determined that a face is newly included in the counterpart image, the restoration means is all means of the image display means, the image decoding means, the imaging means, and the image encoding means. The mobile terminal according to claim 1, wherein the mobile terminal is operated. An image display process;
An imaging step of imaging a direction in which an image is displayed in the image display step;
A receiving step of receiving a partner image stream transmitted by the partner terminal;
An image decoding step of decoding the partner image stream into partner image data;
An image output step for outputting the counterpart image data to the image display step;
An image input step of inputting self-portrait data through the imaging step;
An image encoding step of encoding the self-image data into a self-image stream;
A transmission step of transmitting the self-portrait stream to the counterpart terminal;
A face recognition determination step of determining whether or not a face is included in the partner image represented by the partner image data, and determining whether or not a face is included in the host image represented by the host image data;
When it is determined by the face recognition determination step that at least one of the counterpart image and the self-portrait does not include a face, the image display step, the image decoding step, the imaging step, and the image encoding step And a stop step of stopping at least one of the means. A method for controlling a portable terminal.   A program for operating the mobile terminal according to any one of claims 1 to 12, wherein the program causes a computer to function as each of the above means. The computer-readable recording medium which has recorded the program of Claim 14.