JP2015159461A - Communication device, communication system, image segmentation method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a communication device, communication system, image segmentation method, and program for easily determining an image area to be segmented as an object to be displayed on a communication device of the opposite party.SOLUTION: A control unit 23 of each digital signage device 1 detects the direction of a voice acquired from a voice acquisition unit 31, determines an image area to be segmented from a photographed image of a participant in communication facing an image forming unit on the basis of the detected direction of the voice, and segments the determined image area from the photographed image and transmits the image area to the digital signage device 1 of the opposite party with a communication unit 26.

Description

  The present invention relates to a communication device, a communication system, an image clipping method, and a program.

  2. Description of the Related Art Conventionally, an image output device connected to an image supply device, a reflection member, and a screen, output light that projects content from the image output device is reflected by the reflection member, and output light reflected by the reflection member Is projected on a screen formed in the shape of the outline of the content, and there is known a video output device-equipped device that can perform a display with a higher impression on the viewer (for example, see Patent Document 1). ).

JP 2011-150221 A

  However, the video output device-equipped device described in Patent Document 1 captures an image of a person or the like to be displayed for content display (a still image or a moving image captured without moving as much as possible), and accordingly, as a display unit I was creating a screen. For this reason, such a device is not suitable for displaying an image of a speaker sent from a communication device on the other side as a communication device for a video call. In particular, since only one image is displayed on the screen, for example, when there are a plurality of persons participating in a call, it becomes a problem which one is to be cut out as a display target.

  An object of the present invention is to make it possible to easily determine an image region to be cut out as a display target on a counterpart communication device.

In order to solve the above-mentioned problem, the invention described in claim 1
A communication device that transmits and receives images and sound to and from a communication device on the other side,
Photographing means for photographing an image to be transmitted to the communication device on the other side;
Audio acquisition means for acquiring audio;
Detecting means for detecting the direction of the sound acquired by the sound acquiring means;
Determining means for determining an image region to be cut out from the captured image acquired by the imaging means based on the direction of the detected sound;
Is provided.

  According to the present invention, it is possible to easily determine an image region to be cut out as a display target on the counterpart communication device.

It is a figure which shows the example of whole structure of the communication system in this embodiment. It is a block diagram which shows the functional structure of the digital signage apparatus of FIG. It is a figure which shows schematic structure of the screen part of FIG. It is a flowchart which shows the telephone call process performed by the control part of FIG. (A) is a figure which shows the range divided | segmented by the several direction when detecting an audio | voice, (b) is a figure which shows the area | region corresponding to each of the range divided | segmented when detecting the audio | voice in a picked-up image. It is. It is a figure for demonstrating a peeping operation | movement. It is a flowchart which shows the movement detection process performed by the control part of FIG.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The present invention is not limited to the illustrated example.

[Configuration of Communication System 100]
FIG. 1 is a diagram showing an overall configuration of a communication system 100 according to an embodiment of the present invention. As shown in FIG. 1, the communication system 100 is configured such that a plurality of digital signage devices 1 can be connected via a communication network N. In the present embodiment, the communication network N is described as being the Internet, but is not limited thereto, and may be a telephone line, for example.

[Configuration of Digital Signage Device 1]
FIG. 2 is a block diagram showing a main control configuration of the digital signage device 1 as a communication device in the present embodiment. The digital signage device 1 can make a video call with another digital signage device 1 via the communication network N.

  As shown in FIG. 2, the digital signage apparatus 1 includes a projection unit 21 that irradiates content image light, and a screen unit 22 that receives the image light emitted from the projection unit 21 at the back and projects it onto the front. Yes.

First, the projection unit 21 will be described.
The projection unit 21 includes a control unit 23, a projector 24, a storage unit 25, and a communication unit 26. The projector 24, the storage unit 25, and the communication unit 26 are connected to the control unit 23 as shown in FIG.

  The control unit 23 includes a CPU (Central Processing Unit) that executes various programs stored in the storage unit 25 to perform predetermined calculations and control of each unit, and a memory that is a work area when the program is executed. (Both not shown). The control unit 23 functions as a detection unit, a determination unit, a cutout unit, a transmission control unit, and a voice output control unit in cooperation with a program stored in the program storage unit 251 of the storage unit 25. In addition, it functions as a movement detection unit in cooperation with the imaging unit 30.

  The projector 24 is a projection device that converts the image data output from the control unit 23 into video light and irradiates it toward the back of the screen unit 22. For example, the projector 24 performs a display operation by individually turning on / off each tilt angle of a plurality of micromirrors arranged in an array (in the case of XGA, horizontal 1024 pixels × vertical 768 pixels) at high speed. A DLP (Digital Light Processing) (registered trademark) projector using a DMD (digital micromirror device), which is a display element that forms a light image with the reflected light, is applicable.

  The storage unit 25 is configured by an HDD (Hard Disk Drive), a nonvolatile semiconductor memory, or the like. As shown in FIG. 1, the storage unit 25 is provided with a program storage unit 251, a telephone directory storage unit 252, and the like.

The program storage unit 251 stores a system program executed by the control unit 23, various processing programs, data necessary for executing these programs, and the like.
The telephone book storage unit 252 stores a telephone number and a name registered in advance in association with each other.

  The communication unit 26 includes a modem, a router, a network card, and the like, and performs communication with an external device.

Next, the screen unit 22 will be described.
FIG. 3 is a front view illustrating a schematic configuration of the screen unit 22. As shown in FIG. 3, the screen unit 22 includes an image forming unit 27 and a pedestal 28 that supports the image forming unit 27.

  The image forming unit 27 is formed by laminating a film-like Fresnel lens on a single translucent plate 29 formed in a human shape such as an acrylic plate, which is disposed so as to be substantially orthogonal to the image light irradiation direction. The screen is configured by attaching a rear projection film screen. The image forming unit 27 constitutes a display unit.

  An imaging unit 30 such as a camera is provided on the upper part of the image forming unit 27. The imaging unit 30 captures an image of a space facing the image forming unit 27 to generate a captured image, and outputs the captured image to the control unit 23. Although not shown, the imaging unit 30 includes a camera including an optical system and an imaging element, and an imaging control unit that controls the camera. The imaging element is an image sensor such as a charge coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS), and converts an optical image that has passed through the optical system into a two-dimensional image signal. The imaging unit 30 functions as a photographing unit.

  The pedestal 28 is provided with an audio acquisition unit 31, an operation unit 32, and an audio output unit 33.

  The sound acquisition unit 31 includes a plurality of directional microphones, acquires sound by each microphone, converts the sound into an electric signal, and outputs the electric signal to the control unit 23.

  The operation unit 32 includes a call button 32a, a response button 32b, a call end button 32c, an operation panel 32d, and the like. The transmission button 32a is a button for instructing transmission of a video call. The response button 32b is a button for instructing a response to the video call. The call end button 32c is a button for instructing the end of the video call. The operation panel 32d is composed of a display unit such as an LCD (Liquid Crystal Display) and a touch panel in which transparent electrodes are arranged in a grid pattern so as to cover the surface of the LCD, and is stored in the telephone directory storage unit 252. , Etc., and the position information of the telephone number selected by the finger or the like is detected and output to the control unit 23.

  The audio output unit 33 includes speakers 33 a to 33 e as audio output means, and outputs audio according to instructions from the control unit 23.

  The imaging unit 30, the audio acquisition unit 31, the operation unit 32, and the audio output unit 33 are connected to the control unit 23 as illustrated in FIG.

[Operation of Communication System 100]
Next, the video call operation in the digital signage apparatus 1 of the communication system 100 will be described.
In any digital signage apparatus 1, when the telephone number of the other party of the video call is selected by the operation panel 32d and the call button 32a is pressed, the control unit 23 causes the communication unit 26 to communicate with the other party via the communication network N. A connection request is transmitted to the previous digital signage apparatus 1.
When the connection request is received (received) by the communication unit 26 in the other party's digital signage apparatus 1, the control unit 23 causes the voice output unit 33 to output a ring tone for notifying the incoming call. When the response button 32 b is pressed, the control unit 23 transmits a connection response to the transmission source via the communication network N by the communication unit 26. As a result, a session is established between the two digital signage devices 1, a call line is connected, and a video call state is established. In the video call state, the call processing described later is executed, and the call participants on the transmission side and the reception side face the image forming unit 27 of each digital signage device 1 on the other side of the image forming unit 27. Make a call while watching the call participants.

  When the call end button 32c is pressed in any of the digital signage devices 1 to which the call line is connected, the control unit 23 causes the communication unit 26 to issue a line disconnect request via the communication network N to the partner digital signage device. 1 to send. When a line disconnection response arrives at the other party's digital signage device 1, the session is terminated and the call line is disconnected.

  FIG. 4 shows a flowchart of call processing executed in the digital signage device 1 on the transmission side and reception side in the video call state. Call processing is executed in each digital signage device 1 by cooperation of the control unit 23 and the program stored in the program storage unit 251 of the storage unit 25.

  In addition, in the present application, “cut out” includes, in addition to actually cutting out an area, including enabling to specify a region to be cut out in the digital signage device 1 on the other side. It includes replacement with black or associating position information of a region to be cut out with a captured image.

  First, the control unit 23 causes the imaging unit 30 to acquire a captured image (step S1), and performs a process of acquiring the initial position of the call participant based on the acquired captured image (step S2). Specifically, the face area is recognized from the acquired photographed image, and when there is one recognized face area, the center of the face area is stored in the memory as the initial position of the call participant. When a plurality of face areas are recognized, the center of the face area detected from the position closest to the center of the captured image is stored in the memory as the initial position of the call participant. Note that the recognition of the face area and the recognition of the person area including the face area can be performed using a known image processing technique, and thus description thereof is omitted.

  Next, the control unit 23 determines whether or not the partner image and / or partner voice has been received by the communication unit 26 (step S3). Here, the other party image is an image photographed by the other party digital signage device 1 (that is, a photographed image of a call participant facing the image forming unit 27 of the other party digital signage device 1). It is the image which cut out the area | region of the call participant for one call participant as a display object.

  When it is determined that the other party image and / or the other party voice is not received by the communication unit 26 (step S3; NO), the control unit 23 proceeds to the process of step S5. When it is determined that the other party image and / or the other party voice has been received by the communication unit 26 (step S3; YES), the control unit 23 fits the received other party image to match the shape of the image forming unit 27. Then, the projector 24 displays (projects) the image on the image forming unit 27, and the received voice of the other party is output by the audio output unit 33 (step S4), and the process proceeds to step S5.

  Here, in the communication system 100, the control unit 23 of each digital signage device 1 in step S <b> 8 described later, as shown in FIG. 5A, a predetermined range (front side 180) around the voice acquisition unit 31. The range (°) is divided into a plurality of (here, five) directions, and the level of the sound level (hereinafter referred to as the sound level) for each of the divided ranges is detected. Then, as will be described later in step S9, when audio is transmitted to the digital signage device 1 on the other side, information indicating the audio level for each of the plurality of ranges is also transmitted. That is, the digital signage apparatus 1 on the other side transmits information indicating the sound level for each of a plurality of ranges in which sound is detected, that is, for each direction, along with the sound. Therefore, when outputting the sound in step S4, the control unit 23 gives a bias to the sound volume output from the speakers 33a to 33e based on the sound levels of the plurality of ranges in which the sound is detected. Adjust and output. Specifically, the speakers 33a to 33e correspond to a plurality of ranges (ranges 1 to 5) in which audio is detected in the digital signage device 1 on the other side, and the control unit 23 sets the range where the audio level is high. The corresponding speaker outputs the received sound at a louder volume.

In step S5, the control unit 23 causes the imaging unit 30 to acquire a captured image (step S5) and executes a movement detection process (step S6).
Here, as described above, for example, when there are a plurality of call participants on the other side and a person who is not at the center is a speaker, in step S4, the speaker is in the direction of the range where the speaker exists. A loud sound is output from the corresponding speaker. Then, since only one image of the other party's call participant is displayed on the image forming unit 27, the call participant on the own device side in front of the image forming unit 27 moves at least the face (head). An operation of looking into a person in the direction in which a larger sound is output is performed to see the person in that direction.
Therefore, in step S6, a movement detection process is performed, and based on the photographed image acquired in step S5, a call participant facing the image forming unit 27 (in the case of a plurality of persons, a face region at the center of the photographed image). It is detected whether the center of the face of the call participant whose center is closest is moved from the initial position acquired in step S1. When movement is detected, by transmitting to the digital signage device 1 on the other side, angle change information (details will be described later) indicating the movement amount and movement direction from the initial position as information indicating that the peeping operation has occurred. When the other party's digital signage device 1 cuts out a display target area, a call participant (that is, a speaker) existing in the voice direction (range where the voice level is the highest) is cut out.
In the present embodiment, as shown in FIG. 6, when a loud voice is heard from the right side of the image forming unit 27 (left side as viewed from the person A looking at the image forming unit 27), the person A is at least a face. However, the present invention is not limited to this.

  FIG. 7 shows a flowchart of the movement detection process executed in step S6. The movement detection process is executed in cooperation with the control unit 23 and the program stored in the program storage unit 251.

  First, the control unit 23 recognizes the face area from the captured image acquired by the imaging unit 30, and detects the center position of the recognized face area (step S601). When a plurality of face areas are recognized, the center position of the face area closest to the center of the captured image is detected. Here, the center position of the face area of the call participant whose initial position has been acquired in step S2 is detected.

  Next, the control unit 23 determines whether the movement of the call participant (call participant whose initial position has been acquired) has been detected based on the detected center position of the face and the initial position stored in the memory. (Step S602). Specifically, the control unit 23 detects that the call participant has moved when the detected center position of the face is shifted from the initial position stored in the memory.

  If it is determined that the movement of the call participant has been detected (step S602; YES), the control unit 23 calculates the shift amount (movement amount) of each component in the vertical and horizontal directions of the detected center position of the face with respect to the initial position. The angle θ formed by three points of the initial position, the center of the lens of the imaging unit 30 and the center position of the face detected in step S601 is calculated and acquired as angle change information (step S603). The vertical angle θ is positive when moving upward and negative when moving downward. The angle θ in the left-right direction is positive when moving in the right direction as viewed from the imaging unit 30 and negative when moving in the left direction. The angle θ in each direction can be obtained based on the center of the captured image, the coordinates of each position, the size and angle of view of the captured image, and the like.

  And the control part 23 transmits angle change information to the digital signage apparatus 1 of the other party by the communication part 26 (step S604), complete | finishes a movement detection process, and transfers to step S7 of FIG.

  On the other hand, when it is determined that the movement of the call participant is not detected (step S602; NO), the control unit 23 transmits information notifying the movement to the digital signage device 1 on the other side by the communication unit 26 ( In step S605), the movement detection process is terminated, and the process proceeds to step S7 in FIG.

In step S7 of FIG. 4, the control unit 23 causes the microphones of the sound acquisition unit 31 to acquire a sound signal (step S7), detects the direction of the sound based on the acquired sound signal (step S8), and acquires the sound signal. The communication unit 26 transmits the audio data based on the received audio signal and the information indicating the audio level for each direction to the digital signage device 1 on the other side (step S9).
Here, in step S8, the control unit 23, based on the audio signal acquired by each microphone of the audio acquisition unit 31, for example, around the audio acquisition unit 31 as shown in FIG. A sound level is detected for each range obtained by dividing the (front side) range into a plurality of (here, five) ranges depending on the direction. Then, the direction of the range having the highest audio level is detected as the audio direction. Note that the number of ranges to be divided when detecting the direction of voice is determined based on the number of directional microphones of the voice acquisition unit 31 and the like.

  Next, the control unit 23 determines whether or not a sound equal to or higher than a predetermined value (predetermined sound level) has been acquired (step S10). When it is determined that a sound of a predetermined value or more has been acquired (step S10; YES), the control unit 23 divides the captured image acquired in step S5 so as to correspond to a plurality of ranges in which sound is detected ( Step S11).

  For example, when the range in which the direction of sound is detected is in the range 1 to 5 as shown in FIG. 5A, the captured image corresponds to each of the ranges 1 to 5 as shown in FIG. It is divided into areas 1 to 5 (in which each of the ranges 1 to 5 is captured). In addition, although the numerical value shown below shows the range of each division | segmentation area | region 1-5 with the angle which made the center of the lens of the imaging part 30 0 degree, it is an example and is not limited to this. .

  Next, the control unit 23 performs face recognition processing on the divided area corresponding to the direction of sound in the captured image, that is, corresponding to the range with the highest sound level, and determines whether or not the face area is included. (Step S12). When it is determined that the face area is included in the divided area of the captured image corresponding to the voice direction (step S12; YES), the control unit 23 proceeds to step S14. When it is determined that the face area is not included in the divided area of the captured image corresponding to the voice direction (step S12; NO), the control unit 23 performs face recognition processing on the divided area adjacent to the divided area, It is determined whether or not there is a face area in the adjacent divided area (step S13). When it is determined that there is a face area in the adjacent divided area (step S13; YES), the control unit 23 proceeds to the process of step S14.

  In step S <b> 14, the control unit 23 cuts out a human area including a face area recognized from a divided area corresponding to the voice direction or its adjacent area (when no face area exists in the divided area corresponding to the voice direction). It determines as a position candidate and memorize | stores in memory (step S14). When a plurality of face areas are recognized, any one, for example, a person area including a face area close to the center of the captured image is determined as a cutout position candidate.

  Next, based on whether or not the angle change information is received from the other-side digital signage apparatus 1 by the communication unit 26, the control unit 23 determines whether or not the other-party call participant has performed a peeping operation. (Step S15). When angle change information is received from the other party's digital signage device 1, it is determined that the other party's call participant has performed a peeping operation.

When it is determined that the call participant on the other side has performed the peeping operation (step S15; YES), the control unit 23 sets the peeping direction to the divided region including the cutout position candidate based on the received angle change information. It is determined whether or not the direction is a corresponding direction (step S16).
For example, when the angle change information in the left-right direction is positive, when the divided area including the cutout position candidate in the captured image is the area 4 or the area 5, the peeping direction corresponds to the divided area including the cutout position candidate. It is determined that the direction. When the angle change information in the left-right direction is negative, when the divided region including the cutout position candidate in the captured image is the region 1 or the region 2, the peeping direction is a direction corresponding to the divided region including the cutout position candidate. It is judged that there is.

When it is determined that the peeping direction is the direction corresponding to the divided region including the cutout position candidate (step S16; YES), the control unit 23 includes the peeping angle (that is, the movement amount) including the cutout position candidate. It is determined whether or not the divided area has been reached (step S17).
For example, the angle θ in the left-right direction in the angle change information received from the other-side digital signage device 1 is an angle corresponding to or within the range of the divided region including the cutout position candidate in the captured image. If it exceeds, it is determined that the viewing angle has reached the divided area including the cutout position candidate. For example, when the divided region including the cut-out position candidate is a divided region corresponding to the direction of the voice, the peeping angle exceeds the minimum angle (an angle closer to 0) of the divided region corresponding to the direction of the voice. Then, it is determined that the peeping angle has reached the divided area including the cutout position candidate.

  When it is determined that the peeping angle has reached the divided region including the cutout position candidate (step S17; YES), the control unit 23 determines the cutout position candidate as the cutout position (step S18), and proceeds to step S20. Transition.

  On the other hand, when it is determined in step S10 that there is no sound whose sound level is equal to or higher than the predetermined value (step S10; NO), when it is determined in step S13 that no face area exists in the adjacent divided area (step S13; NO). ) When it is determined in step S15 that there is no peeping operation of the other party on the call (step S15; NO), in step S16, the peeping direction is in the direction corresponding to the divided area including the cutout position candidate. If it is determined that there is not (step S16; NO), or if it is determined in step S17 that the peeping angle has not reached the divided region including the cutout position candidate (step S17; NO), the control unit 23 , The area of the person closest to the center of the captured image (including the face area closest to the center of the captured image Pass) was determined as the cut-out position (step S19), and proceeds to step S20.

In step S20, the control unit 23 cuts out the image at the cut-out position determined from the photographed image, transmits the cut-out position image to the other-side digital signage device 1 through the communication unit 26 (step S20), and returns to the process of step S3.
Until the communication with the digital signage device 1 on the other side is disconnected, the control unit 23 repeatedly executes the processes of step S3 to step S20.

  As described above, according to the communication system 100, each control unit 23 of the digital signage apparatus 1 detects the direction of the sound acquired by the sound acquisition unit 31, and based on the detected direction of the sound, An image area to be cut out from the captured image of the call participant facing the image forming unit 27 is determined, and the determined image area is cut out from the captured image and transmitted to the digital signage device 1 on the other side by the communication unit 26.

  Therefore, it is possible to easily determine an image area to be cut out as a display target on the other party's digital signage device 1 based on the direction of sound.

Specifically, the control unit 23 performs face recognition processing on an area corresponding to the detected voice direction in the captured image, and when the face area is recognized, the image area of the person including the recognized face area Is determined as a region to be cut out from the captured image.
More specifically, the control unit 23 divides a predetermined range around the sound acquisition unit 31 by a plurality of directions, detects the level of sound level for each of the divided plurality of ranges, and performs shooting. The image is divided into a plurality of areas corresponding to the divided ranges. Then, face recognition processing is performed on an area corresponding to the range where the sound level in the photographed image is the highest, and when the face area is recognized, an image area of a person including the recognized face area is cut out from the photographed image. decide.
Accordingly, it is possible to easily determine a person area existing in an area corresponding to the direction of the voice, that is, an image area to be cut out as a display target on the digital signage apparatus 1 on the other side.

  Further, when the face area is not recognized from the area corresponding to the range where the level of the sound volume in the captured image is the largest, the control unit 23 performs face recognition processing on the area adjacent to the area, Is recognized, the person's image area including the recognized face area is determined as an area to be cut out from the captured image. Therefore, even if a person is not detected from the direction of the voice, it is possible to determine an adjacent region as a region to be cut out.

  In addition, each digital signage device 1 has a plurality of speakers 33a to 33e provided corresponding to each of a plurality of ranges in which audio is detected by the partner digital signage device 1, and the control unit 23 For each of the speakers 33a to 33e, the sound received from the digital signage device 1 on the other side is output at a volume corresponding to the level of the sound volume in the corresponding range. Therefore, it is possible to reproduce the sound with a bias depending on the direction of the sound acquired by the digital signage device 1 on the other side.

Also, each digital signage device 1 detects the moving direction and moving amount of the call participant facing the image forming unit 27, and sends information on the detected moving direction and moving amount to the other digital signage device 1. Send. Therefore, it is possible to notify the other party's digital signage device 1 that the call participant has performed the peeping operation.
Further, each digital signage device 1 corresponds to a direction in which the moving direction of the call participant of the other-side digital signage device 1 corresponds to a range in which the level of the volume of the voice acquired by the voice acquisition unit 31 of the own apparatus is the highest. And when the amount of movement exceeds a predetermined threshold according to the direction of the sound, the image region of the person included in the region corresponding to the range where the level of the sound level in the captured image is the largest Is determined as a region to be cut out from the captured image. Therefore, when the other party performs a look-in operation or the like and desires to switch the display target to the speaker corresponding to the voice direction, the display target can be switched according to the voice direction.

  In addition, the description content in the said embodiment is a suitable example of the digital signage apparatus which concerns on this invention, and is not limited to this.

  For example, in the above embodiment, when there is a peeping operation in the detected voice direction and the peeping angle exceeds a threshold corresponding to the voice direction (the peeping angle is the voice direction) The person area existing in the divided area corresponding to the direction of the sound in the captured image is determined as the cutout position. However, the present invention is not limited to this. For example, a person region existing in a divided region corresponding to the detected voice direction may be simply determined as a cutout position regardless of whether or not a peeping operation is performed. Further, when the direction of the peeping operation corresponds to the direction of the voice, the person area existing in the divided area corresponding to the detected voice direction may be determined as the cut-out position. Even in this case, when the other party performs a look-in operation or the like and desires to switch the display target to the speaker corresponding to the voice direction, the display target can be switched according to the voice direction.

In the above embodiment, when a face area is not recognized from the divided area corresponding to the direction of the sound in the captured image, the face area existing in the adjacent divided area is determined as a cutout position candidate. When the face area is not recognized from the divided area corresponding to, the face area may be recognized from the other divided areas in descending order of the sound level, and the recognized face area may be set as a cutout position candidate.
In the above embodiment, when a face area is recognized in a divided area corresponding to the direction of sound in the captured image, the person's image area including the recognized face area is determined as an area to be cut out from the captured image. However, it is good also as an area | region cut out according to the shape of the screen part 22 so that a face area | region may be included.

  In the above embodiment, the case where the present invention is applied to a digital signage apparatus that displays an image by projecting an image from a projector onto a screen has been described as an example. Even when applied to this display device, the same effect can be obtained, and the present invention is not limited to this example.

  In addition, the detailed configuration and detailed operation of each device constituting the communication system can be changed as appropriate without departing from the spirit of the invention.

Although several embodiments of the present invention have been described, the scope of the present invention is not limited to the above-described embodiments, but includes the scope of the invention described in the claims and equivalents thereof.
The invention described in the scope of claims attached to the application of this application will be added below. The item numbers of the claims described in the appendix are as set forth in the claims attached to the application of this application.
[Appendix]
<Claim 1>
A communication device that transmits and receives images and sound to and from a communication device on the other side,
Photographing means for photographing an image to be transmitted to the communication device on the other side;
Audio acquisition means for acquiring audio;
Detecting means for detecting the direction of the sound acquired by the sound acquiring means;
Determining means for determining an image region to be cut out from the captured image acquired by the imaging means based on the direction of the detected sound;
A communication device comprising:
<Claim 2>
Clipping means for cutting out the determined image region from the captured image;
Transmission control means for transmitting the clipped image area to the counterpart communication device;
The communication apparatus according to claim 1.
<Claim 3>
The determination unit performs face recognition processing on an area corresponding to the detected voice direction in the captured image, and when a face area is recognized, an image area of a person including the recognized face area is The communication device according to claim 1, wherein the communication device is determined as an area to be cut out from a captured image.
<Claim 4>
The detection means divides a predetermined range around the sound acquisition means by a plurality of directions, detects a level of sound volume for each of the divided ranges,
The determination unit divides the photographed image into a plurality of regions corresponding to the plurality of divided ranges, and performs face recognition processing on a region corresponding to a range where the level of the sound volume in the photographed image is the largest. The communication apparatus according to claim 2, wherein when a face area is recognized, an image area of a person including the recognized face area is determined as an area to be cut out from the captured image.
<Claim 5>
When the face area is not recognized from the area corresponding to the range where the level of the volume of the sound in the captured image is the largest, the determining means performs face recognition processing on an area adjacent to the area, The communication apparatus according to claim 4, wherein when an image is recognized, an image area of a person including the recognized face area is determined as an area cut out from the captured image.
<Claim 6>
The transmission control means transmits information indicating the level of the loudness level for each of the plurality of ranges detected by the detection means to the communication apparatus on the other side together with the acquired voice,
A plurality of voice output means provided corresponding to each of the plurality of ranges in which voice is detected by the counterpart communication device;
A voice output control means for causing each of the plurality of voice output means to output the voice received from the counterpart communication device at a volume corresponding to the level of the voice in a range corresponding to the voice output means;
The communication device according to claim 4 or 5.
<Claim 7>
Display means for displaying an image received from the counterpart communication device;
A movement detecting means for detecting a moving direction of the person facing the display means;
The transmission control means transmits information on the moving direction detected by the movement detecting means to the communication apparatus on the other side,
The determining means detects, by the detecting means, the moving direction of the person facing the display means of the partner communication device specified based on the information on the moving direction received from the partner communication device. Face recognition processing is performed on an area corresponding to the range where the level of the loudness level in the photographed image is the largest in the direction corresponding to the range where the volume level of the generated voice is the largest. The communication apparatus according to claim 6, wherein when an image is recognized, an image area of a person including the recognized face area is determined as an area to be cut out from the captured image.
<Claim 8>
The movement detection means further detects the movement amount of the person facing the display means,
The transmission control means transmits information on the movement direction and the movement amount detected by the movement detection means to the communication apparatus on the other side,
The determining means detects, by the detecting means, the moving direction of the person facing the display means of the partner communication device specified based on the information of the movement direction received from the partner communication device. And the direction corresponding to the range in which the loudness level is the largest, and the amount of movement exceeds a predetermined threshold according to the range in which the loudness level is the largest. , When face recognition processing is performed on an area corresponding to a range where the level of the volume of the sound in the photographed image is the largest, and when a face area is recognized, an image area of a person including the recognized face area is The communication device according to claim 7, wherein the communication device is determined as an area to be cut out from a captured image.
<Claim 9>
A communication system in which a plurality of communication devices that transmit and receive images and sounds can be connected via a telephone line,
Each of the communication devices
Display means for displaying an image received from the communication device on the other side;
Photographing means for photographing an image to be transmitted to the communication device on the other side;
Audio acquisition means for acquiring audio;
Detecting means for detecting the direction of the sound acquired by the sound acquiring means;
Determining means for determining an image region to be cut out from the captured image acquired by the imaging means based on the direction of the detected sound;
Clipping means for cutting out the determined image region from the captured image;
Transmission control means for transmitting the clipped image area to the counterpart communication device;
A communication system comprising:
<Claim 10>
An image clipping method in a communication device that transmits and receives images and sound to and from a communication device on the other side,
Capturing an image for transmission to the counterpart communication device;
Obtaining audio,
Detecting the direction of the acquired voice;
Determining an image region to be cut out from the captured image acquired by the imaging unit based on the detected direction of the sound;
Image clipping method including
<Claim 11>
A computer used for a communication apparatus that includes an imaging unit that captures an image to be transmitted to a communication device on the other side and a sound acquisition unit that acquires sound, and that is used for the communication device that transmits and receives images and sound to and from the other side communication device.
Detecting means for detecting the direction of the sound acquired by the sound acquiring means;
Determining means for determining an image region to be cut out from the captured image acquired by the imaging means based on the direction of the detected sound;
Program to function as.

DESCRIPTION OF SYMBOLS 1 Digital signage apparatus 21 Projection part 22 Screen part 23 Control part 24 Projector 25 Memory | storage part 251 Program memory | storage part 252 Telephone directory memory | storage part 26 Communication part 27 Image formation part 28 Base 29 Translucent board 30 Imaging part 31 Voice acquisition part 32 Operation part 33 Audio output unit

Claims (11)

A communication device that transmits and receives images and sound to and from a communication device on the other side,
Photographing means for photographing an image to be transmitted to the communication device on the other side;
Audio acquisition means for acquiring audio;
Detecting means for detecting the direction of the sound acquired by the sound acquiring means;
Determining means for determining an image region to be cut out from the captured image acquired by the imaging means based on the direction of the detected sound;
A communication device comprising: Clipping means for cutting out the determined image region from the captured image;
Transmission control means for transmitting the clipped image area to the counterpart communication device;
The communication apparatus according to claim 1.   The determination unit performs face recognition processing on an area corresponding to the detected voice direction in the captured image, and when a face area is recognized, an image area of a person including the recognized face area is The communication device according to claim 1, wherein the communication device is determined as an area to be cut out from a captured image. The detection means divides a predetermined range around the sound acquisition means by a plurality of directions, detects a level of sound volume for each of the divided ranges,
The determination unit divides the photographed image into a plurality of regions corresponding to the plurality of divided ranges, and performs face recognition processing on a region corresponding to a range where the level of the sound volume in the photographed image is the largest. The communication apparatus according to claim 2, wherein when a face area is recognized, an image area of a person including the recognized face area is determined as an area to be cut out from the captured image.   When the face area is not recognized from the area corresponding to the range where the level of the volume of the sound in the captured image is the largest, the determining means performs face recognition processing on an area adjacent to the area, The communication apparatus according to claim 4, wherein when an image is recognized, an image area of a person including the recognized face area is determined as an area cut out from the captured image. The transmission control means transmits information indicating the level of the loudness level for each of the plurality of ranges detected by the detection means to the communication apparatus on the other side together with the acquired voice,
A plurality of voice output means provided corresponding to each of the plurality of ranges in which voice is detected by the counterpart communication device;
A voice output control means for causing each of the plurality of voice output means to output the voice received from the counterpart communication device at a volume corresponding to the level of the voice in a range corresponding to the voice output means;
The communication device according to claim 4 or 5. Display means for displaying an image received from the counterpart communication device;
A movement detecting means for detecting a moving direction of the person facing the display means;
The transmission control means transmits information on the moving direction detected by the movement detecting means to the communication apparatus on the other side,
The determining means detects, by the detecting means, the moving direction of the person facing the display means of the partner communication device specified based on the information on the moving direction received from the partner communication device. Face recognition processing is performed on an area corresponding to the range where the level of the loudness level in the photographed image is the largest in the direction corresponding to the range where the volume level of the generated voice is the largest. The communication apparatus according to claim 6, wherein when an image is recognized, an image area of a person including the recognized face area is determined as an area to be cut out from the captured image. The movement detection means further detects the movement amount of the person facing the display means,
The transmission control means transmits information on the movement direction and the movement amount detected by the movement detection means to the communication apparatus on the other side,
The determining means detects, by the detecting means, the moving direction of the person facing the display means of the partner communication device specified based on the information of the movement direction received from the partner communication device. And the direction corresponding to the range in which the loudness level is the largest, and the amount of movement exceeds a predetermined threshold according to the range in which the loudness level is the largest. , When face recognition processing is performed on an area corresponding to a range where the level of the volume of the sound in the photographed image is the largest, and when a face area is recognized, an image area of a person including the recognized face area is The communication device according to claim 7, wherein the communication device is determined as an area to be cut out from a captured image. A communication system in which a plurality of communication devices that transmit and receive images and sounds can be connected via a telephone line,
Each of the communication devices
Display means for displaying an image received from the communication device on the other side;
Photographing means for photographing an image to be transmitted to the communication device on the other side;
Audio acquisition means for acquiring audio;
Detecting means for detecting the direction of the sound acquired by the sound acquiring means;
Determining means for determining an image region to be cut out from the captured image acquired by the imaging means based on the direction of the detected sound;
Clipping means for cutting out the determined image region from the captured image;
Transmission control means for transmitting the clipped image area to the counterpart communication device;
A communication system comprising: An image clipping method in a communication device that transmits and receives images and sound to and from a communication device on the other side,
Capturing an image for transmission to the counterpart communication device;
Obtaining audio,
Detecting the direction of the acquired voice;
Determining an image region to be cut out from the captured image acquired by the imaging unit based on the detected direction of the sound;
Image clipping method including A computer used for a communication apparatus that includes an imaging unit that captures an image to be transmitted to a communication device on the other side and a sound acquisition unit that acquires sound, and that is used for the communication device that transmits and receives images and sound to and from the other side communication device.
Detecting means for detecting the direction of the sound acquired by the sound acquiring means;
Determining means for determining an image region to be cut out from the captured image acquired by the imaging means based on the direction of the detected sound;
Program to function as.

Images