JP2008060734A - Portable terminal - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a portable terminal having a mechanism by which a caller (user) wants to adjust during hands-free communications at a video telephone mode, so that distance and right/left and up/down positions between the portable terminal and the caller become optimal without the need for a special distance sensor. <P>SOLUTION: In hands-free communication status by a microphone 8 and a speaker 6, there is a 3-dimensional (XYZ direction) caller aptitude position P which is proper for the caller in sound structure. The caller aptitude position P is a specific point within a visual angle range of a camera 4. Accordingly, it is checked beforehand in experiment etc. which part of the own image by the camera 4 corresponds to the caller aptitude position P, and the size and position data of the own image are memorized at a caller aptitude position storage memory 12 in a portable terminal 100. If the own image data of the camera 4 moves from the caller aptitude position P, the own image is emphasized and corrected to the moving direction and displayed in an own image display 52 area. Then, the caller looks at it, and wants to adjust the position so that it may be displayed proper. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a portable terminal for obtaining an optimal call voice during a hands-free call in a videophone mode.

(Background Technology 1)
There is a portable wireless communication device that controls the amplification factor of received signal sound and transmission signal sound according to the distance between the device and a user (sender) during a hands-free call (for example, see Patent Document 1). In this patent document 1, it has an ultrasonic distance sensor, measures the distance between an apparatus and a user (speaker), and controls received signal voice (received voice) and transmitted signal voice (transmitted voice). ing.

(Background Technology 2)
During videophone calls, there is a videophone that controls the amplification factor of the audio signal sent from the other party's telephone according to the distance between the videophone and the speaker (speaker) (for example, see Patent Document 2). . In Patent Document 2, a distance between a video phone and a speaker (speaker) is estimated based on the size of the face of the speaker (speaker) photographed by the camera of the video phone, and a voice signal (received voice) Is controlling.
Japanese Unexamined Patent Publication No. 2004-165962 (pages 3 to 4, FIGS. 1 and 2) Japanese Patent Laid-Open No. 5-64181 (pages 2 and 3, FIG. 1)

In the portable wireless communication device of the conventional background art 1, a dedicated ultrasonic distance sensor for measuring the distance between the device and the user (speaker) is required. In addition, there is a limit depending on the distance between the device and the user (speaker). For example, if the distance is too far, there is a limit even if the amplification factor of the voice is increased. Further, only the distance is measured, and there is no description about the left and right vertical positions between the device and the user (speaker).
Similarly, in the conventional video phone of the background art 2, if the distance between the video phone and the speaker (speaker) is too far, there is a limit even if the voice signal (received voice) is increased. Further, only the distance is measured, and there is no description about the left and right vertical positions between the device and the user (speaker).

  The present invention has been made to solve the above-described problems. In hands-free calling in the videophone mode, the distance between the portable terminal and the sender, and the left, right, up, and down directions are not required without a special distance sensor. It is an object of the present invention to provide a portable terminal having a mechanism that the sender himself / herself wants to adjust so that the position becomes the optimum position.

  In order to achieve the above object, a mobile terminal of the present invention includes a camera that captures a self-portrait of a sender, a display unit that displays the self-portrait, a microphone and a speaker for hands-free calling, and a hands-free call. When the sounder's sound proper position is converted into the image position of the self-image and stored in advance, the speaker's sound proper position storage means, and when the image position of the self-image taken by the camera is out of the sound proper position, And control means for emphasizing and correcting the self-portrait in the direction away from the self-image.

  According to the present invention, there is a mechanism that the speaker himself / herself wants to adjust so that the distance between the portable terminal and the speaker and the left / right / up / down / up / down positions are optimal without requiring a special distance sensor. It becomes possible.

  Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is an external view of a mobile terminal according to an embodiment of the present invention. (A) is a front view, (b) shows a side view. X direction (left and right direction), Y direction (up and down direction), and Z direction (thickness direction) are defined as follows based on the position when the user holds the portable terminal 100 in the hands-free call state in the video phone mode. To do.
The portable terminal 100 has a foldable structure, and an upper housing 1 and a lower housing 2 are engaged with a hinge 3 so as to be freely rotatable. A camera 4, a display unit 5, a speaker 6, and the like are mounted on the upper housing 1. On the lower housing 2, an operation key 7, a microphone 8, and the like are mounted.

  The camera 4 captures a self-portrait of the user of the mobile terminal 100, that is, the transmitter in the videophone mode. The camera 4 has a viewing angle, and the viewing angle in the vertical direction is indicated by a viewing angle V. Although there is a viewing angle in the left-right direction, illustration is omitted. In the videophone mode, the display unit 5 displays the image of the communication partner in the area of the partner image display 51, and displays the sender's self-portrait in the area of the self-image display 52.

  In the hands-free call state in the videophone mode, the talker talks while looking at the display unit 5 at a position slightly away from the portable terminal 100. The transmitter's transmitted sound S is picked up by the microphone 8. The received sound R from the communication partner is output from the speaker 6 and reaches the transmitter.

  The sound input to the microphone 8 includes a sneak echo E from the received sound R, ambient noise N, and the like in addition to the transmitter's transmitted sound S. The wraparound echo E becomes an unpleasant sound because the communication partner hears the spoken word as an echo because the reception sound R spoken by the communication partner is picked up by the microphone 8 and is delayed and reaches the communication partner. End up. In addition, since ambient noise N reaches the communication partner, the sound is hard to hear for the communication partner.

  As a countermeasure, an echo canceller circuit and a noise canceller circuit are mounted inside the mobile terminal 100. In addition, the acoustic structure is designed in consideration of the directivity D structure of the speaker 8 and the microphone 8 based on the normal position of the speaker in the hands-free call state in the videophone mode. In other words, there is an appropriate three-dimensional position of the speaker on the acoustic structure in the hands-free call state. The position is shown as a speaker suitable position P within a one-dot chain line. The area of the speaker appropriate position P is defined as an appropriate position in each of the XYZ directions.

  Further, the speaker appropriate position P is a specific part within the viewing angle range of the camera 4. Therefore, which part of the image of the camera 4 corresponds to the speaker's aptitude position P is confirmed in advance by experiments or the like, and the position data of the image is stored in the speaker's aptitude position storage memory 12 inside the portable terminal 100 (FIG. 2). ).

  FIG. 2 is a diagram for explaining a speaker aptitude position storage memory of a mobile terminal according to an embodiment of the present invention. (A) is a figure explaining the appropriate distance from the Z direction, ie, the portable terminal 100, to a transmitter. (B) is a figure explaining the appropriate up-and-down position of a speaker facing the Y direction, ie, the portable terminal 100. FIG. (C) is a figure explaining the X direction, ie, the right-and-left position of the speaker facing the portable terminal 100. FIG.

  In (a), a thick solid line represents an image obtained by extracting the outline of the face of the sender photographed by the camera 4. The large face outline means that the distance between the portable terminal 100 and the transmitter is short. The fact that the face outline is small means that the distance between the portable terminal 100 and the transmitter is long. Therefore, the size of the face contour of the camera image corresponding to the far limit position in the Z direction of the speaker suitability position P is confirmed by experiment etc. and defined by the size of the far limit 121 shown by the inner one-dot chain line. This is stored in advance in the speaker aptitude position storage memory 12.

  In addition, the size of the face outline of the camera image corresponding to the limit position close to the Z direction of the speaker aptitude position P is confirmed by experiment etc., and defined by the size of the near limit 122 indicated by the outer one-dot chain line. This is stored in advance in the speaker aptitude position storage memory 12. Since the size of the speaker's face varies from person to person, check the average size. Therefore, if the size of the face contour of the camera image is between the distance limit 121 and the near limit 122, the speaker is within the appropriate distance.

  In (b), the thick solid line is a self-portrait of the transmitter photographed by the camera 4. An appropriate vertical position is defined based on, for example, the position of the mouth of the sender's own image. Mouth detection may be performed by detecting mouth characteristics from the self-portrait, or may be defined as a position 30% below the top and bottom intervals of the face contour after extracting the face contour, for example. Then, the position of the mouth of the face of the camera image corresponding to the upper limit position in the Y direction of the speaker suitability position P is confirmed by experiments or the like, and defined by the upper limit 123 indicated by the upper one-dot chain line. Is stored in advance in the speaker aptitude position storage memory 12.

  Further, the position of the mouth of the face of the camera image corresponding to the lower limit position in the Y direction of the talker aptitude position P is confirmed by an experiment or the like, and defined by the lower limit 124 indicated by the lower one-dot chain line, This position is stored in advance in the speaker aptitude position storage memory 12. Therefore, if the position of the mouth of the face of the camera image is between the upper limit 123 and the lower limit 124, the speaker is within the appropriate vertical range.

  In (c), the thick solid line is a self-portrait of the transmitter photographed by the camera 4. For example, the right and left positions are defined with reference to the position of the mouth of the sender's own image. Then, the position of the mouth of the face of the camera image corresponding to the limit position on the right side in the X direction of the talker aptitude position P is confirmed by an experiment or the like, and is defined by the right limit 125 indicated by the one-dot chain line on the right. Is stored in advance in the speaker aptitude position storage memory 12.

  Further, the position of the mouth of the face of the camera image corresponding to the limit position on the left side in the X direction of the speaker suitable position P is confirmed by an experiment or the like, and is defined by the left limit 126 indicated by the one-dot chain line on the left side. Is stored in advance in the speaker aptitude position storage memory 12. Therefore, if the position of the mouth of the face of the camera image is between the right limit 125 and the left limit 126, the speaker is within the proper left and right range.

  Although the appropriate distance is defined based on the size of the face outline, for example, both eyes may be detected from the self-image and defined by the interval between the eyes. In addition, the proper vertical position and horizontal position are based on the mouth position of the self-portrait, but may be any one point within the face outline.

  FIG. 3 is a block diagram of relevant portions of the mobile terminal according to the embodiment of the present invention. In addition to the camera 4, the display unit 5, the speaker 6, the microphone 8, and the like described in FIG. 1, the mobile terminal 100 includes an antenna 9, a communication unit 10, an image processing unit 11, a speaker proper position storage memory 12, an echo canceller. 13, an audio processing unit 14, a control unit 15, and the like. The control unit 15 further includes a contour extraction unit 16, a shift emphasis control unit 17, and the like.

  The antenna 9 transmits and receives radio waves to and from a base station (not shown). The communication unit 10 performs wireless signal processing and the like. The image processing unit 11 performs processing of an image captured by the camera 4 and output of a received image sent from a communication partner of the mobile terminal 100 via the base station to the display unit 5. The speaker appropriate position storage memory 12 stores in advance the speaker appropriate position described with reference to FIG. The echo canceller 13 calculates the received sound data sent from the communication partner via the base station and the wraparound echo from the microphone 8 to remove the wraparound echo. The voice processing unit 14 performs processing of a transmitted sound from the microphone 8 and processing of received sound data.

  The control unit 15 performs overall control of the mobile terminal 100. The contour extracting unit 16 extracts the contour of the face from the self-portrait of the sender photographed by the camera 4 and confirms the size thereof. The shift emphasis control unit 17 compares the size of the face profile of the self-portrait and the vertical and horizontal positions thereof with the speaker's aptitude position in the speaker's aptitude position storage memory 12, and corrects the self-image so as to emphasize the shift. Then, the data is output to the display unit 5 via the image processing unit 11.

Next, the detailed operation of the control unit 15 will be described.
FIG. 4 is an operation flowchart regarding self-portrait display in the videophone mode of the control unit of the mobile terminal according to the embodiment of the present invention.
FIG. 5 is a diagram illustrating a self-portrait display screen of the mobile terminal according to the embodiment of the present invention. This will be described with reference to both figures. When the video phone mode is set, the control unit 15 extracts the face outline based on the self-portrait data from the camera 4. The face contour extraction is performed by searching for a change point in the spectrum of the self-portrait data, or by extracting features of both eyes and estimating the size of the face. Then, the size of the face outline and the position of the mouth of the face are extracted (step S1).

  If this extraction is not possible in step S1 or if the sender is facing the camera 4 of the portable terminal 100, the image from the camera 4 is displayed as it is (step S7). If the face contour can be extracted in step S1, the size of the face contour and the position of the mouth of the face are compared with the speaker appropriate position in the speaker appropriate position storage memory 12 (steps S2 and S3). .

  In the comparison in step S3, if the size of the face contour is not between the size of the far limit 121 and the near limit 122 of the transmitter aptitude position memory 12 (FIG. 2A), the appropriate distance is set. As a result, the image from the camera 4 is highlighted and displayed in a direction shifted in distance (step S4). For example, in the case of the face contour of the camera image indicated by the dotted line in FIG. 5A, it is determined that this is smaller than the far limit 121 in FIG. Further, the emphasis correction is performed so as to be farther away, and the display image is displayed as a solid line.

  The speaker can recognize that the self-portrait is too small by looking at the display image of the self-portrait. Therefore, as a general transmitter action, the transmitter himself / herself can make the display image of the self-portrait large in the portable terminal 100. It will approach. This leads to an acoustically optimal distance for hands-free calls in the videophone mode.

  In the comparison of step S3, if the position of the mouth of the self-portrait is not between the upper limit 123 and the lower limit 124 (FIG. 2 (b)) of the transmitter aptitude position storage memory 12, the camera is regarded as outside the appropriate upper and lower ranges. The image from 4 is displayed with emphasis in the vertically shifted direction (step S5). For example, in the case of the camera image shown by the dotted line in FIG. 5B, this is above the upper limit 123 of FIG. Further, it is corrected as emphasized and displayed as a display image indicated by a solid line.

  Since the speaker can recognize that the self-portrait display image is shifted too much by looking at the display image of the self-portrait, as a general action of the speaker, the self-portrait display image goes down. Will move downward with respect to the portable terminal 100. This leads to an acoustically optimal vertical position for a hands-free call in the videophone mode.

  In the comparison in step S3, if the position of the mouth of the self-portrait is not between the right limit 125 and the left limit 126 (FIG. 2 (c)) of the talker aptitude position storage memory 12, the camera is regarded as outside the proper left and right range. The image from 4 is highlighted and displayed in the left and right shifted directions (step S6). For example, in the case of the camera image indicated by the dotted line in FIG. 5C, it is determined that the image is from the right limit 125 in FIG. Further, the correction is made so as to be on the right, and the display image is displayed as a solid line.

  The speaker can see that the self-portrait display image is shifted to the right by looking at the display image of the self-portrait, so that the self-portrait display image goes to the left as a general transmitter action. Will move to the left with respect to the portable terminal 100. As a result, the acoustically optimal left and right positions of the hands-free call in the videophone mode are guided.

  If both are within the appropriate range in the comparison in step S3, the image from the camera 4 is displayed as it is (step S7). Thus, the display image indicated by the solid line in FIG. 5D, that is, the camera image is displayed as it is.

  According to the embodiment of the present invention, the speaker himself / herself wants to adjust the distance between the portable terminal and the speaker and the right / left / up / down position to an optimum position without requiring a special distance sensor. It becomes possible to have a mechanism.

The external view of the portable terminal which concerns on the Example of this invention. The figure explaining the speaker suitable position storage memory of the portable terminal which concerns on the Example of this invention. The block diagram of the relevant part of the portable terminal which concerns on the Example of this invention. The operation | movement flowchart regarding the self-portrait display in the videophone mode of the control part of the portable terminal which concerns on the Example of this invention. The figure explaining the self-portrait display screen of the portable terminal which concerns on the Example of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Upper housing | casing 2 Lower housing | casing 3 Hinge 4 Camera 5 Display part 51 Other party image display 52 Self-image display 6 Speaker 7 Operation key 8 Microphone 9 Antenna 10 Communication part 11 Image processing part 12 Speaker aptitude position storage memory 13 Echo canceller 14 Audio processing unit 15 Control unit 16 Outline extraction unit 17 Deviation emphasis control unit portable terminal 100

Claims (2)

A camera that takes a self-portrait of the talker,
Display means for displaying the self-portrait;
A microphone and speaker for hands-free calling;
Speaker aptitude position storage means for preliminarily storing the sound proper position of the speaker at the time of a hands-free call converted into the image position of the self-portrait,
And a control unit that, when the image position of the self-portrait photographed by the camera deviates from the appropriate acoustic position, includes a control unit that emphasizes and corrects the self-portrait in the deviating direction and displays the image on the display unit. Terminal. A camera that takes a self-portrait of the talker,
Display means for displaying the self-portrait;
A microphone and speaker for hands-free calling;
Speaker aptitude position storage means for storing in advance the sound proper three-dimensional position of the speaker at the time of a hands-free call as vertical position appropriate data, left and right position appropriate data, and face contour size appropriate data of the self-image,
A face contour extracting means for extracting a face contour of a self-portrait photographed by the camera;
The vertical position of the self-portrait photographed by the camera is compared with the appropriate data for the vertical position, the left-right position of the self-portrait photographed by the camera is compared with the right-and-left position appropriate data, and the face contour of the face contour extracting means and the size of the face contour A portable terminal comprising: a control unit that compares appropriate data and displays the self-portrait on the display unit by emphasizing and correcting the self-portrait in the deviating direction when the appropriate data is deviated from the appropriate data.

Images