KR20040010945A - Image communication - Google Patents

Image communication Download PDF

Info

Publication number
KR20040010945A
KR20040010945A KR1020020043988A KR20020043988A KR20040010945A KR 20040010945 A KR20040010945 A KR 20040010945A KR 1020020043988 A KR1020020043988 A KR 1020020043988A KR 20020043988 A KR20020043988 A KR 20020043988A KR 20040010945 A KR20040010945 A KR 20040010945A
Authority
KR
South Korea
Prior art keywords
information
hand
picture
sign language
characteristic information
Prior art date
Application number
KR1020020043988A
Other languages
Korean (ko)
Inventor
김유호
지석만
Original Assignee
엘지전자 주식회사
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 엘지전자 주식회사 filed Critical 엘지전자 주식회사
Priority to KR1020020043988A priority Critical patent/KR20040010945A/en
Publication of KR20040010945A publication Critical patent/KR20040010945A/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724—User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72475—User interfaces specially adapted for cordless or mobile telephones specially adapted for disabled users
    • H04M1/72478—User interfaces specially adapted for cordless or mobile telephones specially adapted for disabled users for hearing-impaired users
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M2201/00—Electronic components, circuits, software, systems or apparatus used in telephone systems
    • H04M2201/50—Telephonic communication in combination with video communication

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Signal Processing (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Acoustics & Sound (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Telephonic Communication Services (AREA)
  • Telephone Function (AREA)

Abstract

PURPOSE: A method for picture phone communication recognizing hand movements is provided to realize smooth communication between the deaf and normal persons. CONSTITUTION: Hand movements and picture information of a transmitter are inputted through a camera. A movement recognition module extracts characteristic information of the finger language to the picture information. The characteristic information of the finger language and the picture information are compressed by using a compression codec. The compressed characteristic information and picture information are transmitted to a receiving terminal. The receiving terminal receives the compressed data for extracting the picture information and the characteristic information. The restored characteristic information of the finger language are converted into character information. The converted character information and the picture information are displayed.

Description

손동작인식 화상 전화 통신방법{IMAGE COMMUNICATION}Hand gesture recognition video telephony method {IMAGE COMMUNICATION}

본 발명은 손동작 인식화상전화 통신방법에 관한것으로, 특히 화상전화 통신시스템에서 화상정보와 함께 수화동작에 대한 특징정보를 압축하여 전송하고 상대방 전화기에서 상기 특징정보를 문자정보로 변환하여 화상정보와 함께 모니터상에 디스플레이 하므로서 청각장애인과 청각 장애인 혹은 청각장애인과 일반인 사이의 원활한 의사소통을 위한 통신방법의 구현에 있다.The present invention relates to a hand gesture recognition video telephone communication method. In particular, the video telephone communication system compresses and transmits feature information about a sign language operation together with image information, and converts the feature information into text information in the other party's phone and together with the image information. The present invention is to implement a communication method for smooth communication between the deaf and hearing impaired or the deaf and the public by displaying on a monitor.

일반적으로 화상전화기는 송신측에서 카메라에 의하여 촬영된 송신자의 영상데이터와 마이크를 통한 음성데이터를 모뎀을 통하여 전송하고, 수신측에서 모뎀으로 수신된 영상 데이터 및 음성데이터를 복조하고, 각각 영상신호 처리수단 및 음성신호 처리수단을 통하여 신호처리한후에 수신된 송신자의 화상신호는 모니터에디스플레이 하고, 음성신호는 스피커를 통하여 출력하여 상대방의 영상을 보면서 음성통화를 하였다.In general, a video telephone transmits video data of a sender photographed by a camera and audio data through a microphone at a transmitting side through a modem, demodulates video data and audio data received by a modem at a receiving side, and processes a video signal, respectively. After signal processing through the means and the audio signal processing means, the image signal of the sender is displayed on the monitor, and the audio signal is output through the speaker to make a voice call while viewing the image of the other party.

그러나, 청각 장애자가 화상전화기를 이용하는 경우에, 송신자의 화상신호는 모니터를 통하여 볼 수 있으나, 스피커를 통하여 출력되는 음성신호는 청취할 수 없거나, 일반인이 수신할 경우 수화의 동작을 인식할 수 없으므로 의사소통에 문제점이 있었다.However, in the case where the hearing impaired person uses a video telephone, the sender's video signal can be seen through the monitor, but the audio signal output through the speaker cannot be heard or when the general public receives the sign language operation. There was a problem with communication.

본 발명이 이루고자 하는 기술적 과제는 상술한 문제점을 해결하기 위하여 화상전화 통신시스템에서 손의 동작인식을 통하여 송화자의 손의 동작의 특징정보의 전송과 더불어 화상정보를 압축하여 송신함으로써 상대방이 청각장애인이거나 정상인이거나 모두 의사소통이 가능하도록 하는데 있다.The technical problem to be solved by the present invention is to solve the above-mentioned problems by transmitting the feature information of the operation of the talker's hand through the recognition of the hand in the video telephone communication system, and compressing and transmitting the image information to the other party is deaf or hearing impaired It is to make sure that all or normal people can communicate.

제1도는 본 발명에 의한 송신자 단말기의 손동작 인식 및 송신방법을 나타낸1 is a diagram illustrating a hand gesture recognition and a transmission method of a sender terminal according to the present invention.

신호흐름도Signal flow chart

제2도는 본 발명에 의한 수신자 단말기의 손동작 특징정보에 대한 문자변환2 is a text conversion of the hand gesture feature information of the receiver terminal according to the present invention

및 수신방법에 대한 신호흐름도And signal flow diagram for receiving method

도1은 본 발명에 대한 화상정보 및 손동작의 인식을 통한 특징정보추출을 위한 송신단의 데이터 처리 흐름도를 나타내며, 도2는 상기 송신단에서 보내져온 화상정보 및 손동작의 특징정보를 모니터를 통하여 디스플레이하는 동작에 대한 수신단의 데이터 처리 흐름도를 나타내고 있다.1 is a flowchart illustrating a data processing of a transmitter for extracting feature information through recognition of image information and hand gestures according to the present invention, and FIG. 2 is an operation of displaying image information and feature information of a hand gesture sent from the transmitter through a monitor. The data processing flowchart of the receiving end is shown.

이하 도1 및 도2를 참조하여 설명하기로 한다.Hereinafter, a description will be given with reference to FIGS. 1 and 2.

먼저 카메라를 통해 송신자의 손동작 및 화상정보가 입력되며(S11), 상기 입력된 화상정보 및 수화시의 손동작에 대한 정보는 동작인식 모듈로 입력되며, 입력된 사용자의 입력영상으로 부터 동작정보를 인식하여 특징정보를 추출한다(S12).상기 특징정보추출은 수화 인식을 위한 손모양에 대한 인식알고리듬과 손의 궤적모양을 인식하기 위한 알고리듬을 이용하여 추출된다.수화에 사용되는 손의 모양에 따른 의미는 미리 데이터 베이스의 형태로 단말기내 메모리에 저장되어 있고, 각 모양의 인식은 패턴인식 알고리듬을 사용한다. 본 발명에서는 LDA(Linear Discrimineant Analysis)를 사용하여 각 모양을 패턴벡터의 형태로 변환한다. 또한 메모리에 저장되어 있는 손의 모양에 따른 의미 또한 인덱스로 패턴벡터의 형태를 가진다. 실제 사용자의 손 모양이 입력 되었을때 단말기내에 구현된 LDA알고리듬은 입력영상을 패턴벡터의 형태로 변환하고 메모리에 저장된 미리 계산된 패펀벡터값과 비교하여 가장 에러가 적은값을 선택하여 수신측에 전송하게 된다. 만약 정지된 손의 동작이 아니라 손의 궤적에 따른 의미를 파악하기 위해서는 손의 트렉킹과 손의 모양을 동시에 인식해야 한다. 손의 동작을 추적하기 위해서 먼저 손영역만을 컬러정보를 이용하여 분할한 후 빠른 계산을 위하여 칼라필터를 사용하여 손동작의 좌표를 계산한다. 계산된 손의 궤적의 좌표와 패턴벡터의 형태로 계산된 손의 모양의 의미를 동시에 수신단에 전달하게 된다.이어 상기 추출된 특징정보와 화상정보는 MPEG4와 같은 압축코덱에서 압축되며(S13) 수신단으로 송신된다(S14).First, the hand gesture and the image information of the sender are input through the camera (S11), and the input image information and the information on the hand gesture at the time of sign language are input to the motion recognition module, and the motion information is recognized from the input image of the user. The feature information extraction is extracted using a recognition algorithm for a hand shape for sign language recognition and an algorithm for recognizing a track shape of a hand. The meaning is stored in the memory of the terminal in the form of a database in advance, and the recognition of each shape uses a pattern recognition algorithm. In the present invention, LDA (Linear Discrimineant Analysis) is used to convert each shape into a pattern vector. In addition, the semantics according to the shape of the hand stored in the memory also have the form of a pattern vector as an index. When the actual user's hand is input, the LDA algorithm implemented in the terminal converts the input image into a pattern vector form and compares it with the pre-calculated Pfunn vector value stored in memory to select the lowest error value and transmit it to the receiver. Done. If you want to know the meaning of the hand's trajectory rather than the motion of the stationary hand, you have to recognize the hand tracking and the shape of the hand at the same time. In order to track the motion of the hand, first, only the hand region is segmented using color information, and then for the quick calculation, the coordinates of the hand motion are calculated using a color filter. The calculated coordinates of the hand and the meaning of the shape of the hand calculated in the form of a pattern vector are simultaneously transmitted to the receiver. The extracted feature information and the image information are compressed in a compression codec such as MPEG4 (S13). Is transmitted (S14).

수신단에서는 상기 송신단에서 보내져온 압축데이터를 수신하여(S21), 영상정보및 특징정보를 추출하며(S22), 상기 추출된 동작정보는 문자정보로 변환하여(S23)The receiving end receives the compressed data sent from the transmitting end (S21), extracts the image information and the characteristic information (S22), and converts the extracted motion information into text information (S23).

단말기의 모니터를 통하여 화상정보 및 문자정보를 디스플레이 하게된다(S24).The image information and the character information are displayed through the monitor of the terminal (S24).

이상에서 살펴본 바와같이, 본 발명은 손동작 인식 화상전화 통신방법에 관한것으로, 특히 화상전화 통신시스템에서 화상정보와 함께 수화동작에 대한 특징정보를 압축하여 전송하고 상대방 전화기에서 상기 특징정보를 문자정보로 변환하여 화상정보와 함께 모니터상에 디스플레이 하므로서 청각장애인과 청각 장애인 혹은 청각장애인과 일반인 사이의 원활한 의사소통을 위한 통신방법의 구현에 있다.As described above, the present invention relates to a hand gesture recognition video telephone communication method. In particular, the video telephone communication system compresses and transmits characteristic information on a sign language operation together with image information, and transmits the characteristic information to character information in a counterpart phone. By converting and displaying image information on a monitor, the present invention is to implement a communication method for smooth communication between the deaf and hearing impaired or the deaf and the general public.

Claims (2)

카메라를 통해입력된 화상정보에 대하여 동작인식 모듈을 통해 수화에 대한 특징정보를 추출하는 단계와,Extracting feature information on the sign language from the motion recognition module with respect to the image information input through the camera; 상기 수화에 대한 특징정보와 화상정보를 압축코덱을 사용하여 압축 및 전송하는 단계와Compressing and transmitting the feature information and the image information of the sign language using a compression codec; 수신된 수화의 특징정보와 화상정보를 수신 및 복원하는 단계와,Receiving and restoring characteristic information and image information of the received sign language; 상기 복원된 수화의 특징정보를 문자정보로 변환하는 단계와,Converting feature information of the restored sign language into text information; 상기 변환된 문자정보와 화상정보를 디스플레이 하는 단계로 구성된 것을 특징으로 하는 손동작 인식 화상 통신방법.And displaying the converted character information and image information. 제1항에 있어서The method of claim 1 특징정보추출은 수화 인식을 위한 손모양에 대한 인식알고리듬과 손의 궤적모양을 인식하기 위한 알고리듬을 이용하여 추출되는 것을 특징으로 하는 손동작 인식 화상 통신방법.Characteristic information extraction is extracted using a recognition algorithm for the hand shape for sign language recognition and an algorithm for recognizing the trajectory shape of the hand.
KR1020020043988A 2002-07-25 2002-07-25 Image communication KR20040010945A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020020043988A KR20040010945A (en) 2002-07-25 2002-07-25 Image communication

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020020043988A KR20040010945A (en) 2002-07-25 2002-07-25 Image communication

Publications (1)

Publication Number Publication Date
KR20040010945A true KR20040010945A (en) 2004-02-05

Family

ID=37319278

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020020043988A KR20040010945A (en) 2002-07-25 2002-07-25 Image communication

Country Status (1)

Country Link
KR (1) KR20040010945A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100687690B1 (en) * 2006-01-24 2007-02-27 엠텍비젼 주식회사 Digital processing apparatus capable of inputting data using camera and inputting method thereof

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0775075A (en) * 1993-08-31 1995-03-17 Tamura Electric Works Ltd Video telephone system
KR100235194B1 (en) * 1992-09-16 1999-12-15 윤종용 Recognition system
KR20000017756A (en) * 1999-11-12 2000-04-06 황병익 Apparatus for Translating of Finger Language
US6181778B1 (en) * 1995-08-30 2001-01-30 Hitachi, Ltd. Chronological telephone system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100235194B1 (en) * 1992-09-16 1999-12-15 윤종용 Recognition system
JPH0775075A (en) * 1993-08-31 1995-03-17 Tamura Electric Works Ltd Video telephone system
US6181778B1 (en) * 1995-08-30 2001-01-30 Hitachi, Ltd. Chronological telephone system
KR20000017756A (en) * 1999-11-12 2000-04-06 황병익 Apparatus for Translating of Finger Language

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100687690B1 (en) * 2006-01-24 2007-02-27 엠텍비젼 주식회사 Digital processing apparatus capable of inputting data using camera and inputting method thereof

Similar Documents

Publication Publication Date Title
EP2574220B1 (en) Hand-held communication aid for individuals with auditory, speech and visual impairments
US5890120A (en) Matching, synchronization, and superposition on orginal speaking subject images of modified signs from sign language database corresponding to recognized speech segments
US20090313013A1 (en) Sign language capable mobile phone
US20050288063A1 (en) Method for initiating voice recognition mode on mobile terminal
WO1997008895A1 (en) Chirological telephone system
CA2416592A1 (en) Method and device for providing speech-to-text encoding and telephony service
KR20070006336A (en) Method for controlling image in wireless terminal
KR20100026701A (en) Sign language translator and method thereof
US20180293230A1 (en) Multifunction simultaneous interpretation device
KR102299571B1 (en) System and Method for Providing Simultaneous Interpretation Service for Disabled Person
KR101400754B1 (en) System for providing wireless captioned conversation service
CN111128180A (en) Auxiliary dialogue system for hearing-impaired people
KR100735415B1 (en) Method for performing video telephone call of multilateral in?wireless terminal
KR20040010945A (en) Image communication
JP2932027B2 (en) Videophone equipment
KR100703354B1 (en) Method for transmitting image data in video telephone mode of wireless terminal
JP2004015250A (en) Mobile terminal
KR20010107877A (en) Voice Recognized 3D Animation Sign Language Display System
JP2004254218A (en) Cellular phone, communication method, and communication program
JP2000004304A (en) Speech communication device enabling communication with different means
JP2006139138A (en) Information terminal and base station
JP2004248022A (en) Mobile phone and communicating method using the same
JP3062080U (en) Telephone with screen
JPH08307841A (en) Pseudo moving image video telephone system
KR100735378B1 (en) Method for performing video telephone of sign language in wireless terminal

Legal Events

Date Code Title Description
A201 Request for examination
E902 Notification of reason for refusal
E601 Decision to refuse application