JP2015119424A - Communication display device, communication display method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow a user to easily grasp, when using a television telephone function, at which position a self image is appropriately transmitted to a communication partner.SOLUTION: The device comprises: an antenna 21 and a radio communication section 22 which receive from a destination device image data; projection systems 23 to 26, 11 which project image in response to the received image data; a photographing section IM which photographs an image of a user; a CPU 28 which detects deviation from a subject position with the photographed image of the user and notifies in the image to be projected the detected deviation.

Description

  The present invention relates to a communication display device having a videophone function, a communication display method, and a program.

  Conventionally, a technique for projecting video content such as advertisements on a humanoid display panel has been considered. (For example, Patent Document 1)

JP 2011-150221 A

  Recently, including the technology described in the above-mentioned patent document, it is placed on a storefront of a store, etc., and an advertisement image of a product is projected in an arbitrary shape, for example, a human-shaped translucent plate as a screen from its rear side Signage devices are becoming increasingly popular.

  It is considered that this type of digital signage device is provided with an imaging function and a communication function, and is used as a videophone device that is permanently installed in a welfare facility for the elderly, for example.

  When a videophone function is realized using a device having an irregular display unit as described above, the display volume is not as large as when a wide range is displayed by a device having a general rectangular display unit. For this reason, it is necessary to accurately align and display the image of the other party. However, what is displayed on the display unit on the own device side is the image of the other party, and the user has no way of directly confirming how his / her own image is displayed on the display unit of the other device. Therefore, a method for grasping and adjusting the photographing position as easily as possible is being sought.

  The present invention has been made in view of the above circumstances, and the purpose of the present invention is to easily grasp where a user is at when the user is in a videophone function to correctly transmit his / her image to the other party. Another object of the present invention is to provide a communication display device, a communication display method, and a program that can be used.

  One embodiment of the present invention includes a receiving unit that receives image data from a communication destination device, a display unit that displays an image corresponding to the image data received by the receiving unit, and an image that acquires image data to be transmitted. Based on the acquisition unit, the image data acquired by the image acquisition unit, a shift detection unit that detects a shift in the subject position in the image, and the display unit displays the shift according to the shift detected by the shift detection unit. Informing means for informing in the image is provided.

  According to the present invention, it is possible to easily grasp at which position the user is at the time of the videophone function to correctly transmit his / her image to the other party.

The perspective view which shows the external appearance structure of the video telephone apparatus which concerns on one Embodiment of this invention. The figure which looked at the structure of the projection optical system provided in the apparatus housing | casing upper part of the video telephone apparatus based on the embodiment from the right side surface. The figure which shows the screen projection area | region in the rectangular projectable area | region which concerns on the same embodiment. FIG. 3 is a block diagram for mainly explaining a functional configuration of an electronic circuit of the videophone device according to the embodiment. The flowchart which shows the content of the reception process when there exists a call from the standby state which concerns on the same embodiment. The flowchart which shows the content of the transmission process in the case of making a call from the video telephone apparatus side concerning the embodiment. 7 is a flowchart showing detailed contents of a subroutine for call processing in step S108 in FIG. 5 and step S209 in FIG. 6 according to the embodiment. The flowchart which shows the detailed content of the subroutine of the right-and-left deviation alerting | reporting process of FIG.7 S306 which concerns on the same embodiment. The flowchart which shows the detailed content of the subroutine of the back-and-front deviation alerting | reporting process of step S306 of FIG. 7 which concerns on the embodiment. The figure which imitated the shape of the signage board of the other party apparatus acquired from the other party apparatus which concerns on the same embodiment, and the detection range of deviation | shift. The figure which imitated the shape of the signage board of the other party apparatus acquired from the other party apparatus which concerns on the same embodiment, and the detection range of deviation | shift.

Hereinafter, an embodiment in which the present invention is applied to a digital signage videophone apparatus will be described in detail with reference to the drawings. Digital signage is described as a virtual mannequin using a human-type screen.
FIG. 1 is a perspective view showing an external configuration of a videophone device 10 according to the present embodiment. A replaceable signage board SB is erected on the upper front end side of the apparatus housing 10A. This signage board SB has a translucent plate-like configuration having an arbitrary shape, which is installed so as to be within the originally rectangular projectable area of the videophone device 10.

  An imaging unit IM is provided at the front lower center of the signage board SB, and acquires an obliquely upper image on the front side. Further, a pair of speaker units SP and SP and a pair of microphone units MC and MC are provided below the photographing unit IM, and it is possible to receive and transmit stereo sound.

  Further, a mirror MR is erected on the rear end side of the upper surface of the apparatus housing 10A, and a light image emitted from a projection lens unit, which will be described later, is totally reflected on the front side and formed on the signage board SB.

FIG. 2 is a view of the configuration of the projection optical system provided on the upper portion of the apparatus housing 10A of the videophone apparatus 10 as viewed from the right side.
The projected light beam emitted obliquely upward and rearward from the projection lens unit 11 is reflected forward by the mirror MR and then deflected by the Fresnel lens sheet 12 into a parallel light beam in the horizontal direction. 13 is projected.

  A transparent board 14 is integrally formed on the front side of the rear project screen 13. The three layers of the Fresnel lens sheet 12, the rear project screen 13, and the transparent board 14 constitute the signage board SB.

  As shown in FIG. 1, the signage board SB has an indefinite shape such as a human figure, while a display element for forming a light image emitted from the projection lens unit 11, for example, DLP (Digital Light Processing) (registered trademark). ) Type projection apparatus, the micromirror element has a rectangular display area. For this reason, it is necessary to accurately recognize the shape of the signage board SB in order to suppress emission of unnecessary projection light to an area outside the signage board SB.

  In this embodiment, information indicating which signage board SB corresponds to which of a plurality of shape types registered in advance is stored in a storage medium that temporarily stores image data to be transmitted. The image data and the information on the shape type of the signage board SB are transmitted together to the other party's device.

  Therefore, the image acquired by the device on the other end of the call is processed in the device on the other end based on the transmitted shape type information, and then sent to the videophone device 10 at that time. The received image data that matches the shape of the signage board SB mounted on the telephone device 10 may be projected as it is.

  FIG. 3 shows a state in which the CPU 28 described later recognizes the shape of the signage board SB within the rectangular projectable area PA and sets it as the screen projection area SA.

  Next, the functional configuration of mainly the electronic circuit of the videophone device 10 will be described with reference to FIG. In the figure, the wireless communication technology corresponds to, for example, a fourth generation mobile phone system. Here, the radio communication unit 22 receives call data from the base station of the telephone system (not shown) via the antenna 21. The wireless communication unit 22 demodulates the received data and separates it into image data, audio data, and other control data. The obtained image data is sent to the projection image drive unit 23 via the bus B.

  The projection image drive unit 23 multiplies a frame rate according to a predetermined format, for example, 120 [frames / second], the number of color component divisions, and the number of display gradations, in accordance with the transmitted image data. The micromirror element 24, which is a display element, is driven to display by high-speed time division driving.

  The micromirror element 24 performs a display operation by individually turning on / off each tilt angle of a plurality of micromirrors arranged in an array, thereby forming an optical image with the reflected light.

  On the other hand, R, G, B primary color lights are emitted cyclically from the light source unit 25 in a time-sharing manner. The light source unit 25 includes an LED, which is a semiconductor light emitting element, and repeatedly emits primary colors of R, G, and B in a time division manner. The LED included in the light source unit 25 may include an LD (semiconductor laser) or an organic EL element as an LED in a broad sense.

  Moreover, it is good also as what uses the primary color light from which a wavelength differs from the original light excited by irradiating the light radiate | emitted from LED to a fluorescent substance. The primary color light from the light source unit 25 is totally reflected by the mirror 26 and applied to the micromirror element 24.

  Then, an optical image is formed by the reflected light from the micromirror element 24, and the formed optical image is reflected on the signage board SB after being reflected by the mirror MR (not shown) through the projection lens unit 11. The

  The audio data demodulated and separated by the wireless communication unit 22 is transmitted to the audio processing unit 27 via the bus B. The sound processing unit 27 has a sound source circuit such as a PCM sound source, for example, reproduces an analog sound signal based on the transmitted sound data, and emits sound from the speaker units SP and SP.

  On the other hand, an analog audio signal input from the microphones MC and MC that become a receiver during a call on the videophone device 10 is sampled by the audio processing unit 27, converted into digital audio data, and compressed. . The audio data is transmitted from the wireless communication unit 22 via the bus B from the antenna 21 to the device of the call partner.

  The CPU 28 controls all the operations of the above circuits. The CPU 28 is directly connected to the main memory 29 and the program memory 30. The main memory 29 is constituted by an SRAM, for example, and functions as a work memory for the CPU 28. The program memory 30 is composed of an electrically rewritable non-volatile memory such as a flash ROM, and stores an operation program executed by the CPU 28, various fixed data, and the like.

  The CPU 28 controls the videophone device 10 by reading out the operation program, fixed data, and the like stored in the program memory 30, expanding and storing them in the main memory 29, and executing the program. Control.

  The CPU 28 performs various projection operations according to operation signals from the operation unit 31. This operation unit 31 provides key operation signals of several operation keys including an on-hook key, an off-hook key, and dial keys (ten key, “*” key, “#” key) provided in the main body of the videophone device 10 to the CPU 28. To send.

The CPU 28 is further connected to the captured image processing unit 32 and the image memory 33 via the bus B.
The captured image processing unit 32 scans and drives a solid-state imaging device, for example, a CMOS image sensor 35, which is disposed at the in-focus position of the imaging lens unit 34 that constitutes a part of the imaging unit IM. An image signal obtained by shooting is obtained.

  The image signal output from the CMOS image sensor 35 is digitized by the A / D conversion circuit 36 and then sent to the photographed image processing unit 32. The captured image processing unit 32 temporarily stores the obtained image data in the image memory 33 as appropriate, recognizes a person's face from the image using face recognition technology described later, cuts out the range, The process which detects the deviation | shift amount of is performed.

  Further, the captured image processing unit 32 compresses the image data including the cut face portion into a file, and sends the file to the wireless communication unit 22 via the bus B. In the wireless communication unit 22, under the control of the CPU 28, the image data from the photographed image processing unit 32 is transmitted to the device on the other party side of the call in the same manner as the audio data and control data from the audio processing unit 27.

Next, the operation of the above embodiment will be described.
The operation described below is executed after the CPU 28 develops the operation program read from the program memory 30 in the main memory 29 as described above. The operation program and the like stored in the program memory 30 are not only those stored in the program memory 30 at the time of factory shipment of the videophone device 10, but also a version upgrade program after the user purchases the videophone device 10. Includes the contents of the installation.

  FIG. 5 shows a processing procedure of reception processing by the videophone function when there is a call from the device of the other party in the standby state. In the standby state, the CPU 28 waits for a call to be made by repeatedly determining whether or not a call is received from the other party's device (step S101).

  When it is determined that the call is received when the call is made, the CPU 28 determines whether or not the user of the videophone device 10 has responded to the call by determining whether or not the off-hook key of the operation unit 31 is operated next. If there is no such operation, the process returns to the standby state from step S101.

  When the user operates the off-hook key from the called state, the CPU 28 determines the operation in the above step S102, and then reads the shape information of the signage board SB mounted at that time from the image memory 33, and wirelessly The communication unit 22 transmits the call to the other party's device (step S103). If the user operates the off-hook key in step S102 and the shape information is added from the communication partner device, the shape information is received.

  Thereafter, the CPU 28 causes the captured image processing unit 32 to scan and drive the CMOS image sensor 35 to start the photographing operation (step S104). For example, the captured image processing unit 32 starts moving image capturing with a CMOS image sensor 35 at an image size of VGA (vertical 480 pixels × horizontal 640 pixels) and a frame rate of 30 [frame / second], and sequentially obtains the obtained image data. Stored in the memory 33.

  Here, the CPU 28 projects the image from the other party's device when the off-hook is determined in the step S102, in the same manner as the shape information of the signage board SB of the videophone device 10 is transmitted in the step S103. It is determined whether or not the shape information related to is received (step S105).

  If it is determined that shape information has been received from the other party's device, the received shape information is set on the assumption that the other party has an irregular projection target as in the case of the videophone device 10 (step). S106).

  If it is determined in step S106 that shape information has not been received from the other party's device, it is assumed that the other party does not have an irregular projection target, unlike this videophone device 10, etc. Is set to a rectangular shape (step S107).

  After the setting in step S106 or S107, the CPU 28 performs a call process with the other party's device based on the setting contents (step S108), and determines whether or not the on-hook key of the operation unit 31 has been operated. (Step S109) If not operated, the call processing from Step S108 is continued.

  By repeatedly executing the processes of steps S108 and S109 in this way, the operation waits for the operation of the on-hook key for terminating the call while continuing the call operation.

  When the on-hook key is operated, it is determined that the call is terminated in step S109, the call is terminated, and the process returns to the process from step S101 in order to shift to the next standby state.

  Next, FIG. 6 shows a processing procedure of outgoing call processing by the video phone function when a call is made from the video phone device 10 side. At the beginning of the process, the CPU 28 repeatedly determines whether or not a destination input for specifying the call partner has been made by operating the off-hook key and the dial key (step S201), and waits for the start of the calling process.

  If it is determined that the destination has been input, the CPU 28 reads out the shape information of the signage board SB mounted at that time from the image memory 33 and transmits it by the wireless communication unit 22 together with information on the telephone number for identifying the other party ( Step S202).

  Thereafter, the CPU 28 repeatedly determines whether or not there is a response from the other party's device after the telephone line is connected (step S203) and whether or not the on-hook key has been operated (step S204). Or whether the user of the videophone device 10 gives up starting the call.

  If it is determined in step S204 that the on-hook key has been operated, the CPU 28 determines that the call has been given up and returns to the processing from step S201 again.

  If it is determined in step S203 that there is a response from the other party's device, the CPU 28 causes the captured image processing unit 32 to scan and drive the CMOS image sensor 35 to start the capturing operation (step S205). For example, the captured image processing unit 32 starts moving image capturing with a CMOS image sensor 35 at an image size of VGA (vertical 480 pixels × horizontal 640 pixels) and a frame rate of 30 [frame / second], and sequentially obtains the obtained image data. Stored in the memory 33. In addition, when there is a response from the communication partner device in step S203 and shape information is added from the communication partner device, the shape information is received.

  Here, the CPU 28 determines whether or not the shape information regarding the object on which the image is to be projected has been received from the device of the other party (step S206).

  If it is determined that shape information has been received from the other party's device, the received shape information is set on the assumption that the other party has an irregular projection target as in the case of the videophone device 10 (step). S207).

  If it is determined in step S206 that shape information has not been received from the other party's device, the other party does not have an irregular projection target (in the form of a rectangle) unlike this videophone device 10 or the like. In step 208, a rectangular shape is set as a projection target.

  After the setting in step S207 or S208, the CPU 28 determines whether or not the on-hook key of the operation unit 31 has been operated while executing a call process with the other party's device based on the setting content (step S209). (Step S210) If it is not operated, the call processing from Step S209 is continued.

  By repeatedly executing the processes in steps S209 and S210 in this way, the on-hook key operation for terminating the call is waited while continuing the call operation.

  When the on-hook key is operated, it is determined that the call is terminated in step S210, and the call is terminated, and the process returns to step S201 to shift to the next standby state.

  Next, with reference to FIG. 7, the detailed contents of the call processing subroutine in step S108 in FIG. 5 and step S209 in FIG. 6 will be described.

  Initially, the CPU 28 projects the sound based on the sound data received from the other party's device via the wireless communication unit 22 from the speaker units SP and SP by the sound processing unit 27 and based on the image data by the projection image driving unit 23. Images are output from the projection lens unit 11 (step S301).

  Thereafter, the CPU 28 first causes the wireless communication unit 22 to transmit a compressed audio data file based on the audio input from the microphone units MC and MC to the device of the other party (step S302).

  Next, the CPU 28 creates image data so as to match the shape information and image size obtained from the other party's device, creates an image data file, and transmits it to the other party's device from the wireless communication unit 22 (step S303). ).

  Here, the CPU 28 performs a face recognition process on the image data that has been transmitted after the transmitted image data and has not yet been subjected to the clipping process, and waits for the completion of the process (step S304).

  When it is determined in step S304 that the face recognition processing has been completed, the CPU 28 next detects a human torso area that continues downward from the recognized face part area (step S305).

  Regarding the calculation of the area of the person other than the face area, the CPU 28 periodically acquires image data obtained from the photographing unit IM at the installation position of the videophone device 10 in the absence of a temporary subject such as a person. Thus, it is possible to determine the area of the person other than the face-recognized area by storing it as background image data in the image memory 33 in advance and comparing it with the captured image of the imaging unit IM at the time of the videophone function. good.

  The right and left shift notification processing (step S306) and the front and rear shift notification processing (step S306) for displaying the self image at the correct position on the other party's device by using the face area and the non-face area of the person thus obtained. S307) is executed, and necessary notification processing is performed by superimposing it on the projection image on the signage board SB. Then, the subroutine of FIG. 7 is terminated, and the reception processing of FIG. Return to outgoing call processing.

  FIG. 8 is a subroutine showing details of the notification process for the left / right deviation in step S306 of FIG. At the beginning of the process, the CPU 28 alerts that the position of the person's area in the image is set at a threshold that is set in advance from a range that can be displayed on the display unit of the other party's device. It is determined whether or not the attention line is exceeded (step S401).

  FIG. 10 schematically shows the shape of the signage board SB of the counterpart device acquired from the counterpart device, and the area corresponding to the person's face and the area below it are rectangular. Represents. In each region, the position of several% at the left and right ends, for example, each 15% of the width of the region is set as the attention line AL, and it is determined whether or not the attention line AL is exceeded by the above recognition.

  If it is determined in step S401 that the area of the person in the image does not exceed the attention line AL, the CPU 28 is within the central area sandwiched by the attention line AL, and the right and left The processing of FIG. 8 is terminated on the assumption that there is no deviation and no notification is required.

  If it is determined in step S401 that the area of the person in the image exceeds the attention line AL, then the CPU 28 determines whether the area of the person in the image is outside the range that can be displayed on the display unit. Thus, it is determined whether or not the display frame of the other party's device is protruding (step S402).

  Here, when it is determined that the area of the person in the image protrudes from the display frame of the other party's device, the CPU 28 recognizes the protruding position and then displays an image serving as a warning corresponding to the position in the projected image. Superimposed by the drive unit 23 and projected on the signage board SB in the own apparatus (step S403), the process of FIG.

  At this time, for example, on the signage board SB of the videophone device 10, the CPU 28 projects the image of the user of the device on the other end of the call, and at the position and direction of the area of the face that is determined to protrude and the area other than the face. In response, an area corresponding to the outside of the attention line AL shown in FIG. 10 on the image is projected with a warning color, for example, red.

  Therefore, the user of this videophone device 10 projects his / her captured image by coloring the face area of the other party's image projected on the signage board SB or the side of the other area in red. It is possible to understand very visually that the other party is protruding.

  In step S402, it is determined that the area of the person in the image exceeds the attention line AL on the display unit, but is not out of the displayable range, and the display frame of the other party's device does not protrude. In this case, the CPU 28 recognizes a position that exceeds the attention line AL, and then superimposes an image that requires attention according to the position on the signage board SB in the device itself by superimposing the image on the projection image driving unit 23 ( In step S404), the processing of FIG.

  At this time, for example, on the signage board SB of the videophone device 10, the CPU 28 projects the image of the user of the device on the other end of the call, and determines the positions of the determined facial region exceeding the attention line AL and the region other than the face Depending on the direction, an area corresponding to the outside of the caution line AL shown in FIG. 10 on the image is superimposed and projected with a caution color, for example, yellow.

  Therefore, the user of the videophone device 10 projects his or her captured image by coloring the face area of the other party image projected on the signage board SB or the side of the other area in yellow. It can be understood very visually that the other party is about to overflow.

  Next, a subroutine showing details of the notification process for the back and forth deviation in step S307 of FIG. 7 will be described with reference to FIG. At the beginning of the process, the CPU 28 alerts that the position of the person's area in the image is set at a threshold that is set in advance from a range that can be displayed on the display unit of the other party's device. It is determined whether or not the upper caution line is exceeded (step S501).

  FIG. 11 schematically shows the shape of the signage board SB of the partner device acquired from the partner device. The region corresponding to the face of the person and the region below it are rectangular. Represents. In each area, the position of several% of the upper end, for example, each 15% of the upper side of the vertical width of the area is set as the upper caution line AL. It is determined whether ten users are too close to the photographing unit IM.

  If it is determined in step S501 that the area of the person in the image does not exceed the upper attention line AL, the CPU 28 is too close to both the face area and the non-face area in the image. If the position of the person area in the image is not set to the preset value from the range that can be displayed on the display unit of the other party, a warning will be issued. It is determined whether or not the lower attention line is exceeded (step S501).

  In FIG. 11, in the area corresponding to the face of a person, the position of several% from the lower end, for example, the lower 15% of the vertical width of the area is the lower attention line AL. Further, in the lower region other than the person's face, for example, a substantially central height position is set as a lower caution line AL.

  For each of the lower attention lines AL, the CPU 28 determines whether the user of the videophone device 10 is in accordance with whether or not the lower side attention line AL is straddled from the recognition result of the person. It is determined whether it is too far away from the imaging unit IM (step S505).

  Here, when the CPU 28 determines that the person recognition result does not straddle the lower attention line AL in each region and the user of the videophone device 10 is not too far from the photographing unit IM, The processing of FIG. 9 is terminated on the assumption that there is no deviation and no notification is required.

  If it is determined in step S505 that a person recognition result straddles the lower attention line AL in at least one of the areas, and the user of the videophone device 10 is too far from the photographing unit IM. Since the CPU 28 is displayed in a small size on the partner device, the CPU 28 superimposes an image for calling attention below the lower attention line AL in each area in the projected image driving unit 23 to signage board in the own device. Projection is performed on SB (step S506), and the processing of FIG.

  At this time, the CPU 28 projects, for example, the image of the user of the other party device on the signage board SB of the videophone device 10, and further below the lower attention line AL in both the face area and the non-face area. Are projected with the attention color, for example, yellow, superimposed.

  Therefore, the user of this videophone device 10 projects his / her captured image small by projecting the lower side of the face area of the other party image projected on the signage board SB or the lower side of the other area in yellow. I can understand what is reflected very visually.

  If it is determined in step S501 that the person area in the image exceeds the upper attention line AL, the CPU 28 next starts from the upper side of the range in which the person area in the image can be displayed on the display unit. It is determined whether or not the display frame on the upper side of the device on the other end of the caller is protruding (step S502).

  If the CPU 28 determines that the area of the person in the image protrudes from the display frame on the upper side of the other party's device, the CPU 28 superimposes an image that becomes a warning on being too close by the projected image driving unit 23, and 9 is projected on the signage board SB (step S503), and the processing of FIG. 9 is completed.

  At this time, for example, on the signage board SB of the videophone device 10, the CPU 28 projects the image of the user of the other device of the other party, and the attention lines on the upper side of the face area and the non-face area that are determined to be protruding. An area above AL is projected with a warning color, for example, red.

  Therefore, the user of the videophone device 10 projects his / her captured image by coloring the face area of the other party's image projected on the signage board SB and the upper side of the other area in red. It is possible to understand very visually that the other side of the call is protruding too close to the other side.

  In step S502, although the area of the person in the image exceeds the upper attention line AL on the display unit, it does not deviate from the upper side of the displayable range, and the upper display frame of the other party's device is If it is determined that the image does not protrude, the CPU 28 recognizes the position exceeding the upper caution line AL, and superimposes an image that becomes a caution on being too close by the projection image drive unit 23, thereby signage board in the device itself. Projection is performed on the SB (step S504), and the processing of FIG.

  At this time, for example, on the signage board SB of the videophone device 10, the CPU 28 projects the image of the user of the other party's device on the attention line AL above each of the face area and the non-face area in FIG. A region above the corresponding position is projected with a caution color, for example yellow, superimposed.

  Therefore, the user of the videophone device 10 projects his or her captured image by coloring the face area of the other party image projected on the signage board SB or the upper side of the other area in yellow. You can understand very visually that the other party is approaching too close.

  As described above in detail, according to the present embodiment, it is possible to easily grasp at which position the user is at the time of the videophone function to correctly transmit his image to the other party.

  In the above embodiment, the face recognition process is performed from the photographed Yusa's own image to recognize the face area and the other area, and the shift is detected, so that the shift can be detected more accurately. Can do.

  Furthermore, according to the above-described embodiment, it is easy to understand by the user by performing visual notification according to at least one of the position, the direction, and the degree of deviation in the image to be displayed or projected, and the immediate deviation is corrected. It becomes possible to shift to the operation to do.

  In the above-described embodiment, the case where the region of the corresponding direction is displayed or projected by superimposing yellow as a caution color and red as a warning color according to the direction and degree of deviation is illustrated, but the present invention does so. For example, the position where the shift is generated may be expressed by a spot-colored image (a plurality of continuous spots when the shift is long), or the shift is corrected. The moving direction for this purpose may be expressed by superimposing an arrow image.

  When the image of the arrow is superimposed, the degree can be expressed by changing the color of the arrow or the size on the image depending on the magnitude of the deviation.

  Moreover, you may alert | report a warning and warning about the shift | offset | difference before and behind with a character.

  Also, change the notice method of warning and warning of left and right deviation and front and rear deviation, for example, the right and left deviation is reported using caution color and warning color, and the front and rear deviation is reported by letters This often makes it easier for the user to know how to move.

  Moreover, although the said embodiment demonstrated the case where the digital signage apparatus which performs a projection of a DLP (trademark) system was applied to the soft videophone apparatus, the system of projection, an apparatus, etc. are not limited.

  Furthermore, for example, when making a videophone call using a communication terminal such as a smartphone, it can be expressed by the same processing that the orientation of the smartphone being held is tilted, and the inclination of the smartphone is changed with respect to the user. Or, you can instruct to change the perspective position of the smartphone.

  In addition, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the invention in the implementation stage. Further, the functions executed in the above-described embodiments may be combined as appropriate as possible. The above-described embodiment includes various stages, and various inventions can be extracted by an appropriate combination of a plurality of disclosed constituent elements. For example, even if some constituent requirements are deleted from all the constituent requirements shown in the embodiment, if the effect is obtained, a configuration from which the constituent requirements are deleted can be extracted as an invention.

Hereinafter, the invention described in the scope of claims of the present application will be appended.
According to the first aspect of the present invention, receiving means for receiving image data from a communication destination device, display means for displaying an image corresponding to the image data received by the receiving means, and data of an image to be transmitted are acquired. Based on the image acquisition means, the image data acquired by the image acquisition means, a deviation detection means for detecting the deviation of the subject position in the image, and the display means according to the deviation detected by the deviation detection means. And informing means for informing in the image that is present.

  According to a second aspect of the present invention, in the first aspect of the present invention, the shift detection means detects a shift of the subject position from the image based on face recognition processing.

  According to a third aspect of the present invention, in the first or second aspect of the present invention, the notifying unit notifies in accordance with at least one of a position, a direction, and a degree of deviation in an image displayed by the display unit. It is characterized by performing.

  According to a fourth aspect of the present invention, in the first aspect of the invention, the image processing apparatus includes a background image acquisition unit that acquires a background image, and a subject position detection unit that detects a subject position by comparing the background image. The shift detection means detects a shift in the image based on the detection result of the subject position detection means.

  The invention according to claim 5 is the invention according to any one of claims 1 to 4, wherein the communication destination device is a projection device including a projection panel, and the image data to be transmitted is the projection panel. The shift detection means detects a shift of the subject position with respect to the image to be transmitted projected on the projection panel.

  According to a sixth aspect of the present invention, there is provided a caution line setting means for setting a caution line with respect to a display frame of an image to be transmitted displayed on the communication destination device. Further, the shift detection means detects a shift based on the attention line and the position of the subject.

  The invention according to claim 7 is the invention according to any one of claims 1 to 6, wherein the deviation detecting means detects a deviation in the left-right direction and a deviation in the front-rear direction.

  The invention described in claim 8 is characterized in that, in the invention described in claim 7, the notification means performs different notification for the shift in the left-right direction and the shift in the front-rear direction.

  According to the ninth aspect of the present invention, a receiving step for receiving image data from a communication destination device, a display step for displaying an image corresponding to the image data received in the receiving step, and image data to be transmitted are acquired. Based on the image acquisition process, the image data acquired in the image acquisition process, a shift detection process for detecting a shift in the subject position in the image, and the display process according to the shift detected in the shift detection process. And an informing step for informing within the image.

  According to a tenth aspect of the present invention, there is provided a program executed by a computer built in a communication destination device and a device that transmits and receives image data, wherein the computer receives image data from the communication destination device, and Display means for displaying an image according to the image data received by the receiving means, image acquiring means for acquiring data of the image to be transmitted, and the displacement of the subject position in the image by the image data acquired by the image acquiring means. It is made to function as a detection means for detecting, and a notification means for making a notification in the image displayed by the display means in accordance with the detection detected by the detection means.

  DESCRIPTION OF SYMBOLS 10 ... Video telephone apparatus, 10A ... Apparatus housing, 11 ... Projection lens part, 12 ... Fresnel lens sheet, 13 ... Rear project screen, 14 ... Transparent board, 21 ... Antenna, 22 ... Wireless communication part, 23 ... Projection image drive , 24 ... micromirror element, 25 ... light source unit, 26 ... mirror, 27 ... sound processing unit, 28 ... CPU, 29 ... main memory, 30 ... program memory, 31 ... operation unit, 32 ... photographed image processing unit, 33 ... Image memory, 34 ... Shooting lens unit, 35 ... CMOS image sensor, 36 ... A / D conversion circuit, AL ... Caution line, B ... Bus, IM ... Shooting unit, MC ... Microphone unit, MR ... Mirror, PA ... Projection Possible area, SA ... Screen projection area, SB ... Signage board, SP ... Speaker part.

Claims (10)

Receiving means for receiving image data from a communication destination device;
Display means for displaying an image according to the image data received by the receiving means;
Image acquisition means for acquiring image data to be transmitted;
A deviation detecting means for detecting a deviation of the subject position in the image based on the image data obtained by the image obtaining means;
A communication display device comprising: an informing means for informing an image displayed on the display means according to the deviation detected by the deviation detecting means.   2. The communication display device according to claim 1, wherein the shift detection unit detects a shift of the subject position from the image based on face recognition processing.   The communication display device according to claim 1, wherein the notification unit performs notification according to at least one of a shifted position, direction, and degree in an image displayed by the display unit. Background image acquisition means for acquiring a background image;
Subject position detecting means for detecting the subject position by comparing with the background image,
2. The communication display apparatus according to claim 1, wherein the shift detection unit detects a shift in the image based on a detection result of the subject position detection unit. The communication destination device is a projection device including a projection panel,
The image data to be transmitted is processed into the shape of the projection panel,
The communication display device according to claim 1, wherein the shift detection unit detects a shift of a subject position with respect to the image to be transmitted projected on the projection panel. A caution line setting means for setting a caution line for the display frame of the image to be transmitted displayed on the communication destination device,
The communication display device according to claim 1, wherein the shift detection unit detects a shift based on the attention line and the position of the subject.   The communication display device according to claim 1, wherein the shift detection unit detects a shift in the left-right direction and a shift in the front-rear direction.   8. The communication display device according to claim 7, wherein the notification means performs different notifications for the left-right direction shift and the front-rear direction shift. A receiving step of receiving image data from a communication destination device;
A display step of displaying an image according to the image data received in the reception step;
An image acquisition step of acquiring image data to be transmitted;
A shift detection step for detecting a shift in the subject position in the image based on the image data acquired in the image acquisition step;
A communication display method comprising: a notification step of notifying in the image displayed in the display step according to the shift detected in the shift detection step. A program executed by a computer built in a communication destination device and a device that transmits and receives image data, and the computer is
Receiving means for receiving image data from a communication destination device;
Display means for displaying an image according to the image data received by the receiving means;
Image acquisition means for acquiring image data to be transmitted;
Based on the image data acquired by the image acquisition means, a shift detection means for detecting a shift of the subject position in the image, and a notification in the image displayed by the display means according to the shift detected by the shift detection means A program characterized in that it functions as a notification means.

Images