JP2008060864A - Interpretation system with privacy protection function - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an interpretation system with simple structure by which privacy protection of a caller is sufficiently attained in a system for providing sign language interpretation or foreign language interpretation service by images and sound utilizing a network. <P>SOLUTION: The privacy protection of the caller is attained by combining a means for preventing incorporation of a part of a real image of the caller as an image using a lens filter and an infrared irradiation function, a processing means for performing work processing of the incorporated image by a simple means and a means for storing communication data in a storage device at remote place from locations of a user and an interpreter. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an interpreting system having a function for protecting a caller's privacy in a sign language interpreting or foreign language interpreting service using a terminal device connected via a network.

  Conventionally, it has been difficult to transmit a large amount of data at high speed in communication between remote locations, so clear images cannot be transmitted and received. We were unable to provide sign language services. In addition, there is a system that provides a foreign language interpreting service from a remote location, but it is not an interpreting service using a video phone or the like having a poor image quality, but a voice interpreting service using a telephone or the like is common.

However, with the development of communication technology over the network, it has become possible to transmit and receive clear images, and communication using a terminal device that can transmit and receive video and audio bidirectionally, such as videophones, has been performed, Recently, a service for providing a sign language interpreter or a foreign language interpreter using a terminal device such as a videophone has been proposed.

  However, the provision of interpreting services using terminal devices such as videophones provides sufficient privacy for users and interpreters because the images of users and interpreters are displayed as they are and the data is stored as they are. The problem of being unprotected is occurring.

Therefore, in order to protect the privacy of users and interpreters, videophone devices and interpreting systems that display proxy images such as character images and animation images instead of actual images of callers have been proposed.
JP 2003-37826 A JP 2003-16475 A

  However, when the caller's real image is converted into a proxy image such as an animation, the caller's real video is not transmitted, so anonymity is high. However, the caller's facial expression, mouth movement, and hand movement are sufficient. It is difficult to express. In particular, in sign language interpretation, the movement of the mouth and hand is important, and it is difficult to faithfully represent this with a proxy image.

In addition, there is a problem that data of actual video captured through the camera is held in the system for a long time. This is because it is necessary to constantly monitor facial expressions in order to observe emotions, and data collection and analysis must be performed simultaneously when providing interpretation services.

Furthermore, these devices and systems require a large amount of data processing, and there are problems that the device and system configuration becomes complicated and that smooth communication becomes difficult.

  The present invention has been made in view of the above problems, and its purpose is to sufficiently protect the privacy of a caller in a video and audio interpretation system using a videophone or the like connected via a network. However, it is to provide a simple interpretation system that can provide a smooth and reliable interpretation service.

  In order to solve the above problems, the present invention provides a user terminal device used by a user who wants to provide a service in an interpreter system that provides sign language interpretation or a foreign language interpretation service using video and audio, and a service center. An interpreter system in which an interpreter terminal device installed and used by an interpreter is connected via a network, and each terminal device provides an interpreting service by bidirectionally transmitting and receiving image data and audio data. It is characterized by having a privacy protection means for protecting the privacy of the user or interpreter.

  Each terminal device includes an image input unit, an audio input unit, an image data and audio data transmission / reception device, an image and character information display unit, and an audio output unit, and performs image processing to process image data as privacy protection means. Or at least one processing means for processing the voice data.

  As the privacy protection means, the image input unit has a function of not capturing a part of the real image of the caller as an image. Thereby, it is possible to reduce the transmission data capacity and enable smooth communication.

  As a function of not capturing a part of the real image of the caller as an image, the image input unit is provided with a filter lens in which a shaded color or black color is applied within a predetermined range.

  As a function of not capturing a part of the real image of the caller as an image, the image input unit has an infrared irradiation function.

  As privacy protection means, it has a face area detection means for detecting the caller's face and each part from the image acquired from the image input unit, and a setting means for setting a reference point based on the detected face area. Image processing means for deleting at least one of a certain range or facial parts from the reference point immediately after acquisition is provided.

  As a privacy protection means, an image of the same part or part of a third party's face or a CG image stored in advance in the part deleted in claim 6 and having a size smaller than the deleted range is used. An image processing means for pasting is provided.

  As privacy protection means, image processing means for applying a special effect such as inversion to the image acquired from the image input unit is provided, and a blurred image due to the special effect is transmitted and received.

  As the privacy protection means, a voice filter for processing the acquired voice is provided in the voice input unit of each terminal device, and the voice of the caller is processed and transmitted / received.

  As privacy protection means, it has a storage device for storing information of users and interpreters connected via a network, and a management server for managing interpreting services. By the management server, user terminal devices and interpreter terminals The device is divided into areas according to the area where the device is installed, and the transmission from the user terminal device belonging to the first area is received by the interpreter terminal device belonging to the second area that is geographically remote. The user and interpreter information and communication data are stored in a storage device belonging to a third area other than the first and second areas.

  Each terminal device has a face area detecting means for detecting the face of the caller. When the face area detected by the face area detecting means is out of the preset coordinate range, the setting of the image input unit is initial. It switches to a setting and image acquisition is performed again, It is characterized by the above-mentioned.

  The interpreter terminal device includes a confirmation signal transmission means for transmitting a signal indicating whether or not the interpretation content has been confirmed, and the user terminal device transmits a response signal for replying whether or not the content is understood with respect to the confirmation signal. Means are provided.

  Each terminal device includes character information input means. When character information is input from the character information input means, the character information is displayed on the display unit of the receiving terminal device.

  According to the interpreting system of the present invention, the caller's privacy is protected by processing and transmitting and receiving part of the face image and sound without displaying the actual video of the caller as it is and It is possible to provide an interpretation service that makes full use of video by displaying the movements of hands and hands as they are.

  In addition, the actual video of the caller is stored in the system because part of the video is acquired in a blurred or deleted state, or is processed immediately after acquisition. The privacy of the caller can be sufficiently protected.

  In addition, the communication from the user is received by the interpreter terminal device in another area far away, and the communication information of the user and the interpreter is stored in the storage device belonging to the other third area. Therefore, it is possible to sufficiently protect the privacy of the caller.

  Since the interpreting system of the present invention is constructed with a simple configuration, the terminal device and the interpreting service can be provided at low cost, and the image processing is simple such as deletion, so that the communication data is large. Therefore, it is possible to provide smooth bidirectional communication, that is, smooth interpretation service.

  FIG. 1 is a diagram showing an outline of an interpreting system according to an embodiment of the present invention. The interpreting system of the present invention includes at least one user terminal device 2 and at least one service center 3. These are connected via the network 1.

  FIG. 2 is a diagram illustrating a configuration of each terminal device. The user terminal device 2 is a terminal device used by a person who is provided with an interpreting service, and is installed at a place where interpretation is required, such as a shop or a public office, or a deaf or foreigner's home. Using the user terminal device 2, a deaf person or a foreigner himself / herself or a person corresponding to a deaf person or a foreigner at a window or the like can be provided with an interpretation service.

  The user terminal device 2 includes an image input unit 11, a voice input unit 12, a display unit 13, a voice output unit 14, an image processing unit 15, a voice processing unit 16, a transmission / reception unit 17, a response signal transmission unit 18, and a character information input unit. 19 is provided. The image input unit 11 is a camera or the like that captures the user's video, and the audio input unit 12 is a microphone or the like that captures the user's voice. The voice processing unit 16 processes the acquired voice of the caller. The display unit 13 displays the image and character information transmitted from the interpreter terminal device 4. The response signal transmission unit 18 transmits a response signal for returning whether or not the interpretation content transmitted from the interpreter terminal device 4 is understood in response to the confirmation signal indicating whether or not the interpretation content is understood. The character information input unit 19 is, for example, a keyboard, a scanner, or the like, and captures characters that are typed on the keyboard or an image of a document that is read by the scanner.

  The service center 3 includes a server 5, a storage device 6, and a plurality of interpreter terminal devices. An interpreter is resident in the service center 3, and the interpreter uses the interpreter terminal device 4 to provide an interpretation service. The server 5 connects communication from the user terminal device 2 to an arbitrary interpreter terminal device 4, and the storage device 6 stores various information such as the area and sex of the user and the interpreter, image data, and audio. Communication information including data is stored.

  The interpreter terminal device 4 includes an image input unit 21, an audio input unit 22, an image output unit 23, an audio output unit 24, an image processing unit 25, an audio processing unit 26, a transmission / reception device 27, a confirmation signal transmission unit 28, and character information input. A portion 29 is provided. The confirmation signal transmission unit 29 transmits a signal for confirming whether or not the user understands the interpreted content. Other configurations are the same as those of the user terminal device 2.

The user terminal device 2 is provided with a service request transmission means for receiving the provision of an interpreting service. Whether the interpreting service desired is a sign language interpreter, a foreign language interpreter, or a foreign language interpreter. The management server 5 communicates and connects the interpreter terminal device 4 with the interpreter capable of interpreting and the user terminal device 2 based on the service request signal. The interpreting service may be started.

  Next, image processing of the acquired image in the image processing units 15 and 25 of each terminal device will be described. Since each terminal device performs the same processing, description will be made based on the user terminal device 2 here.

FIG. 4 is a flowchart showing processing of the acquired image. First, the image of the caller is read from the image input unit 11 (step 101). From the captured image, the face area detection unit detects the face area and each part of the face (step 102). Next, it is determined whether or not it has been detected (step 103). If it has not been detected, the process returns to step 101, and image capture is performed again. If it can be detected, for example, the middle point of the eye is recorded as a reference point (step 104). Next, image processing is performed based on the reference point (step 105). If the reference point is within the preset coordinate range on the screen, the image data is transmitted. If the reference point is not within the coordinate range, the image transmission is interrupted and the process returns to step 101 and again. Image acquisition is performed (step 106). The above processing is repeated during a call.

FIG. 5 is a diagram showing the display screen 30 when the actual image is acquired as it is, and FIG. 6 is a diagram showing the display screen 30 after the image processing in step 105 described above. This image processing deletes a part of the face, for example, the eye part based on the detected face area, each part of the face, and the set center coordinates, and deletes the white deleted part 32. The image 33 of the same part of a third party stored in advance is pasted. Since the image 33 to be pasted is an image having a size smaller than that of the deleted portion 32, a white boundary is provided between the actual image and the image 33. This makes it difficult to determine who the caller is.

FIG. 8 is a diagram showing a lens filter 31 attached to the image input unit 11 and an image captured in this case. The lens filter 31 has a predetermined range, for example, an upper portion that is black or light and shaded as shown in the figure. The lens filter 31 is provided in the image input unit 11 instead of or in addition to the image processing unit. By wearing 31, it is possible to prevent a predetermined portion of the caller's face, for example, a portion above the eyes from being captured as an image.

  FIG. 9 is a diagram showing an acquired image when the image input unit 11 is provided with an infrared irradiation function in place of the lens filter 31. The irradiated portion 34 is obtained by irradiating a predetermined portion of the caller with far infrared rays. Since it becomes white, it is not captured as a real image.

  FIG. 7 is a flowchart showing image acquisition when the image input units 11 and 21 have a lens filter and an infrared irradiation function. Since the user terminal device 2 and the interpreter terminal device 4 perform the same processing, description will be made based on the user terminal device 2. First, the caller's image is read from the image input unit 11 (step 201). At this time, a part of the image is not acquired as a real image by the lens filter 31 or infrared irradiation. Next, in a state where a portion to be displayed as an image is not displayed (deleted), an intermediate point of the eye at this time is recorded as a reference point (step 202). If the reference point is within the preset coordinate range on the screen, image data is transmitted. If the reference point is not within the coordinate range, the image transmission is interrupted and the process returns to step 201. Image acquisition is performed again (step 203).

  In this way, by capturing an image without including a part of the real image such as a part of the caller's face, there is no concern that the caller's face is held in the terminal device as it is. Compared with the case where the image data is acquired and processed and transmitted, the image data capacity can be reduced, and smooth communication is possible.

  In addition, it is possible to further improve the caller's anonymity by performing an effect process on the acquired image after capturing the image without including a part of the real image using a lens filter or an infrared irradiation function. .

Each terminal device may have a voice processing unit 16 and 26, and may be configured to process and transmit the voice of the caller input from the voice input units 12 and 22, or transmit the caller's voice as it is. And it is good also as a structure which processes the audio | voice in an other party's terminal device, and outputs the audio | voice processed by the audio | voice output part 13 or 23. FIG.

  FIG. 3 is a diagram showing the communication relationship between the user terminal device 2 and the service center 3 and the relationship between the recording device storing the communication information. The user terminal device 2 and the interpreter terminal device 4 are divided into areas such as area 1, area 2... According to the area where they are installed, and transmission from the user terminal device belonging to the first area is The interpreter terminal device belonging to the second area that is geographically remote is received, and the information and communication data of the user and interpreter are stored in the storage device belonging to the third area other than the first and second areas. The For example, when there is a transmission from the user terminal device 2 belonging to the area 1, the transmission is performed by the management server 5, and the transmission is performed by an interpreter in the service center 3 belonging to the area 3 which is geographically remote from the area 1. The terminal device 4 receives it. The communication data is stored in the storage device 6 in the service center 3 belonging to the area 6 other than the areas 1 and 2.

In this way, interpreters who are geographically remote from the user interpret the user so that the user and the interpreter are not in the same area, and the communication data is stored and held in another area. This makes it possible to protect the privacy sufficiently.

As communication management by the management server 5, it is conceivable to connect to the interpreter terminal device 4 belonging to the geographically remote area based on the telephone number information transmitted from the user terminal device 2. It is also possible to manage communication based on

  The interpreting system of the present invention allows the user to select a stepwise privacy protection level by combining the various privacy protection means described above.

It is a book figure which shows the structure of one Embodiment of this invention. It is the figure which showed the structure of each terminal device which concerns on this invention. It is a figure which shows the relationship between the user terminal device which concerns on this invention, the communication relationship of a service center, and the recording device in which communication information is stored. It is a flowchart which shows the process of an acquired image. It is the figure which showed the display screen at the time of acquiring a real image as it is. It is the figure which showed the display screen after image processing. It is a flowchart which shows image acquisition in the case of acquiring an image in the state which does not include a part of real image. It is the figure which showed the lens filter which concerns on this invention, and its display screen. It is a figure which shows the acquired image at the time of providing the infrared irradiation function which concerns on this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Network 2 User terminal device 3 Service center 4 Interpreter terminal device 5 Management server 6 Storage device 11, 21 Image input part 12, 22 Voice input part 13, 23 Display part 14, 24 Voice output part 15, 25 Image processing part 16, 26 Voice processing unit 17, 27 Transmission / reception unit 18 Response signal transmission unit 19, 29 Character information input unit 28 Confirmation signal transmission unit

Claims (13)

  In an interpretation system for providing sign language interpretation or foreign language interpretation services by video and audio, a user terminal device used by a user who desires service provision, and an interpreter terminal device installed in a service center and used by an interpreter Is an interpreter system in which each terminal device provides an interpreting service by bidirectionally transmitting and receiving image data and audio data, and the privacy of the user or interpreter in the system is provided. Interpretation system characterized by providing protection means.   Each terminal device includes an image input unit, an audio input unit, an image data and audio data transmission / reception device, an image and character information display unit, and an audio output unit, and performs image processing to process image data as privacy protection means. 2. The interpreting system according to claim 1, further comprising at least one processing means among means or voice processing means for processing voice data.   The interpreting system according to claim 1 or 2, wherein the image input unit has a function of not taking a part of a real image of a caller as an image as privacy protection means.   4. The function of not capturing a part of the real image of the caller as claimed in claim 3 as an image, wherein the image input unit includes a lens filter in which a shaded color or black color is applied within a predetermined range. Interpretation system.   4. The interpreting system according to claim 3, wherein the image input unit has an infrared irradiation function as a function of not capturing a part of the real image of the caller of claim 3 as an image.   As privacy protection means, it has a face area detection means for detecting the caller's face and each part from the image acquired from the image input unit, and a setting means for setting a reference point based on the detected face area. 6. The interpreting system according to claim 1, further comprising image processing means for deleting at least one of a certain range or facial parts from the reference point immediately after acquisition.   As a privacy protection means, an image of the same part or part of a third party's face or a CG image stored in advance in the part deleted in claim 6 and having a size smaller than the deleted range is used. 7. The interpreting system according to claim 6, further comprising image processing means for pasting.   8. The interpreting system according to claim 1, wherein image processing means for applying a special effect such as inversion to the image acquired from the image input unit is provided as privacy protection means, and a blurred image due to the special effect is transmitted and received. .   9. The interpreter according to claim 1, wherein a voice filter for processing the acquired voice is provided in the voice input unit of each terminal device as privacy protection means, and the voice of the caller is processed and transmitted / received. system.   As a privacy protection means, the interpreting system of claim 1 includes a storage device for storing information of users and interpreters connected via a network, and a management server for managing the interpreting service. The user terminal device and the interpreter terminal device are divided into areas according to the area where the user terminal device and the interpreter terminal device are installed, and transmission from the user terminal device belonging to the first area belongs to the second area that is geographically remote. 10. The interpreter terminal device receives the information and communication data of users and interpreters and is stored in a storage device belonging to a third area other than the first and second areas. Interpretation system.   Each terminal device has a face area detecting means for detecting the face of the caller. When the face area detected by the face area detecting means is out of the preset coordinate range, the setting of the image input unit is initial. The interpreting system according to claim 1, wherein the image acquisition is performed again after switching to the setting.   The interpreter terminal device includes a confirmation signal transmission means for transmitting a signal indicating whether or not the interpretation content has been confirmed, and the user terminal device transmits a response signal for replying whether or not the content is understood with respect to the confirmation signal. The interpreting system according to claim 1, further comprising means.   13. Each terminal device includes character information input means, and when the character information is input from the character information input means, the character information is displayed on the display unit of the receiving terminal device. Interpretation system.

Images