KR101979605B1 - Wireless Terminal and Information Processing Method - Google Patents
Wireless Terminal and Information Processing Method Download PDFInfo
- Publication number
- KR101979605B1 KR101979605B1 KR1020110136908A KR20110136908A KR101979605B1 KR 101979605 B1 KR101979605 B1 KR 101979605B1 KR 1020110136908 A KR1020110136908 A KR 1020110136908A KR 20110136908 A KR20110136908 A KR 20110136908A KR 101979605 B1 KR101979605 B1 KR 101979605B1
- Authority
- KR
- South Korea
- Prior art keywords
- information
- wireless terminal
- application
- text information
- vehicle
- Prior art date
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W88/00—Devices specially adapted for wireless communication networks, e.g. terminals, base stations or access point devices
- H04W88/02—Terminal devices
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L13/00—Speech synthesis; Text to speech systems
- G10L13/08—Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/26—Speech to text systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M1/00—Substation equipment, e.g. for use by subscribers
- H04M1/60—Substation equipment, e.g. for use by subscribers including speech amplifiers
- H04M1/6033—Substation equipment, e.g. for use by subscribers including speech amplifiers for providing handsfree use or a loudspeaker mode in telephone sets
- H04M1/6041—Portable telephones adapted for handsfree use
- H04M1/6075—Portable telephones adapted for handsfree use adapted for handsfree use in a vehicle
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/02—Services making use of location information
- H04W4/021—Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/12—Messaging; Mailboxes; Announcements
Abstract
The present invention relates to a wireless terminal, an information processing method, and a recording medium. The wireless terminal according to the present invention includes a short range communication unit for connecting a short distance communication channel with a communication device or a hands-free device provided in a vehicle, and a short range communication unit through the short range communication unit. A confirmation unit for confirming whether the wireless terminal is located in the vehicle based on the communication channel connection information, and a first result for transmitting / receiving character information provided in the wireless terminal when the wireless terminal is located in the vehicle, To voice information converted by the conversion unit 1 through a short-range communication channel connected through the short-range communication unit; and the conversion unit 1 for converting text information received through one of the N-th applications into voice information. A processing unit for transmitting the voice information through the hands-free device or the communication device for a vehicle by transmitting a signal; And an input unit for receiving voice information of a wireless terminal user input to the hands-free device or a vehicle communication device through a short range communication channel, and a conversion unit 2 for converting the input voice information into text information. Is processed so that the converted text information is transmitted through the text information transmission and reception application.
Description
The present invention confirms that the location of the wireless terminal is located in the vehicle, converts the text information received by the wireless terminal into the voice information and outputs, and converts the voice information input through the wireless terminal into the text information to be processed will be.
In recent years, as mobile messengers become more popular, various IM (Instant Messenger) applications such as KakaoTalk and other e-mail applications, short message service (SMS) applications, etc. have been established as means for receiving text information. It has several types of text information receiving means instead of ~ 2.
However, such a number of text information receiving means has been mass-produced a problem that when the user is driving, the driver's eyes are dispersed and may act as a risk factor when driving the vehicle as the text information is sometimes received.
On the other hand, thanks to the development of the Text to Speech (TTS) engine that converts text information into voice information and the STT (Speech to Text) engine that converts voice information into text information, users can receive text information received from wireless terminals. The voice information can be output and listened to, and when the voice information is input, it can be converted into text information and transmitted.
Patent Publication No. 10-2001-0086402 (voice recognition device) describes a device and method for recognizing a user's voice, and Patent Publication No. 10-2002-0094988 (voice synthesis method and implementation thereof) Voice synthesizer) describes a device and a method for synthesizing a specific sentence into an arbitrary voice.
However, even if the TTS engine and the STT engine are provided in the wireless terminal, when receiving text information, the user must drive the corresponding TTS engine and the STT engine one by one, and drive the TTS engine and the STT engine while driving the vehicle. If you forget, it's a shameless feature.
An object of the present invention for solving the above problems, after checking whether or not the wireless terminal is located in the vehicle, when the wireless terminal is located in the vehicle, automatically driving the TTS engine and STT engine, the converted voice information By outputting the signal through a handsfree device or a communication device of a vehicle connected to a wireless terminal through a short-range communication channel, the TTS engine during driving while solving the inconvenience of having to process the driving of the TTS engine and the STT engine directly on the vehicle. Computer that records information processing methods and programs that perform information processing in wireless terminals and wireless terminals that forgets to drive the STT engine and blocks the driver's eyes and actions from dangerous situations when they receive text information. The present invention provides a recording medium that can be read.
According to an aspect of the present invention, a wireless terminal includes a short range communication unit for connecting a short distance communication channel with a communication device or a hands-free device included in a vehicle, and a short distance communication channel connection information through the short range communication unit. A confirmation unit for confirming whether or not, and when the confirmation unit confirms that the wireless terminal is located in the vehicle, voice information received through one of the first to Nth applications for text information transmission and reception provided in the wireless terminal is voiced. And converts the voice information converted by the converter 1 through a short-range communication channel connected through the short-range communication unit and the hands-free device or the vehicle communication device. A processing unit which processes the output unit and the hands-free device or the vehicle barrel through the short range communication channel; And an input unit for receiving voice information of a user of a wireless terminal input to a device, and a converting unit 2 for converting the input voice information into text information, wherein the processing unit transmits and receives the text information. To be transmitted through the application.
According to one side, the confirmation unit may be activated at the time when the text information is received through one of the first to N-th application for transmitting and receiving the text information, the wireless terminal and the from the communication device provided in the vehicle Receive short range communication connection information between communication devices, when the wireless terminal logs in to the communication device, receives a signal corresponding to the login information of the wireless terminal from the communication device, or the Bluetooth device and the wireless terminal provided in the vehicle Bluetooth devices provided in the device are paired, directly receiving vehicle boarding information of the wireless terminal from the communication device, or receiving information connected to a wireless terminal charging terminal on a power supply device provided in the vehicle through the communication device. In addition, the wireless terminal may check whether the vehicle is located in the vehicle.
According to another aspect, the wireless terminal, and the interface unit for providing an interface for selecting one or more of the first to N (N = 2,3..N) applications for transmitting and receiving text information provided in the wireless terminal; When the character information is received by the registration unit for storing and registering information of at least one character information transmission / reception application selected by the wireless terminal user through the interface unit in a memory, and the first to Nth applications for character information transmission and reception, If the text information transmission / reception application for receiving the text information is an application for transmitting / receiving text information registered in the memory, the text information received or received by the text information transmission / reception application previously stored in the memory or the It may further include an extraction unit received through the application for transmitting and receiving text information. All.
According to another aspect, the interface unit may be voice information among text information received by a selected application other than an application selection function among the first to Nth applications (N = 2,3..N) applications for transmitting and receiving text information. The item information to be converted and output may be further selected or received, and the registration unit may be further stored and registered in a memory by connecting the item information selected or input through the interface with the selected application information for transmitting and receiving text information. The extracting unit may check the item information connected to the application information for transmitting and receiving the character information on the memory, and selectively extract or receive only character information corresponding to the item information from the character information. The item information may include the short message service (SMS), In the case of a message transmission / reception application including one or more Long Message Service (LMS) and Multimedia Message Services (MMS), the message transmission / reception application may include one or more item information of message sender information and message content information. It may include one or more item information of the email sender information, email subject information and email body content information, and if the application is an instant messenger application or an SNS application, and includes one or more item information of the user ID information and content information. can do.
An information processing method in a wireless terminal according to the present invention includes a connection step of connecting a short distance communication channel with a communication device or a hands-free device provided in a vehicle, and based on the short distance communication channel connection information through the short distance communication unit, A confirmation step of confirming whether the terminal is located in the vehicle, and as a result of the confirmation through the confirmation step, when the wireless terminal is located in the vehicle, the application of one of the first to N-th application for transmitting and receiving text information provided in the wireless terminal A voice information conversion step of converting text information received through voice information into voice information; and checking whether the hands-free device or the vehicle communication device connected to the wireless terminal through the short-range communication is activated, and if it is deactivated, activating it. And establish a short range communication channel with the activated hands free device or vehicle communication device. The voice information processing step of transmitting the converted voice information through the hands-free device or the vehicle communication device to output the voice information, and the voice information of the wireless terminal user input through the hands-free device or the vehicle communication device An input step received through a short range communication channel, a text information conversion step of converting the input voice information into text information, and a text information processing step of processing the converted text information to be transmitted through the text information transmission and reception application; It includes.
According to one side, the checking step,
The method may further include: first checking whether text information is received through one of the first to Nth applications for transmitting and receiving text information, and in response to the first verification result, confirming whether the wireless terminal is located in the vehicle. Can be.
According to another aspect, the checking step, when receiving the short-range communication connection information between the wireless terminal and the communication device from the communication device provided in the vehicle, or when the wireless terminal logs into the communication device, from the communication device Receive a signal corresponding to the login information of the wireless terminal, the Bluetooth device provided in the vehicle and the Bluetooth device provided in the wireless terminal is paired, or directly receives the vehicle boarding information of the wireless terminal from the communication device, The communication device may receive information connected to a charging terminal of a wireless terminal on a power supply provided in the vehicle to check whether the wireless terminal is located in the vehicle.
According to another aspect, the information processing method in the wireless terminal, the interface for selecting one or more of the first to N-th (N = 2,3..N) applications for transmitting and receiving text information provided in the wireless terminal; A step of providing an interface, a registration step of storing and registering information on at least one character information transmission / reception application selected by a wireless terminal user through the interface, in a memory, and the first to Nth applications for character information transmission and reception. When text information is received, if the text information transmission / reception application for receiving the text information is previously stored in the memory and registered text information transmission / reception application, the previously stored text information transmission / reception application in the memory is received. Extract text information or transmit it through the text information transmission and reception application It may further include an extraction step.
According to another aspect, the interface providing step, the voice information of the text information received by the selected application other than the selection of one or more applications of the first to N (N = 2,3..N) applications for transmitting and receiving the text information May further select or receive item information to be converted and output, and the registration step may further include storing the item information selected or input through the interface with the application information for transmitting / receiving the selected text information, in a memory. In the extracting step, after checking the item information connected to the application information for transmitting and receiving the character information on the memory, only the character information corresponding to the item information among the character information may be selectively extracted or received. .
According to the present invention, there is also provided a computer-readable recording medium characterized by recording a program for executing each of the above steps.
In addition, according to the present invention, the text information transmission and reception application, a message transmission and reception application including one or more Short Message Service (SMS), Long Message Service (LMS), Multimedia Message Services (MMS), an email transmission and reception application, It may include an instant messenger application and one or more Social Networking Service (SNS) application, the text information, the message sender information and message content when the application is a message transmission and reception application including one or more SMS, LMS, MMS And at least one piece of information, and if the application is an email sending / receiving application, including at least one email sender information, an email subject information, and an email body content information. If the application is an instant messenger application, messenger user information and messenger content information. To Or it may include more.
According to the present invention, by checking only whether the radio terminal is located in the vehicle to automatically drive the TTS engine and STT engine, while solving the inconvenience that the user must directly handle the driving of the TTS engine and STT engine on board the vehicle one by one Forgetting the operation of the TTS engine and the STT engine while driving has the effect of blocking the dangerous situation where the driver's eyes and actions are focused on the wireless terminal at the time of receiving text information.
In addition, the converted voice information is output through the hands-free device or the communication device provided in the vehicle, not the wireless terminal speaker, and the user's reply voice information is input through the hands-free device or the communication device provided in the vehicle instead of the wireless terminal microphone. As a result, a smooth service is possible even when the wireless terminal is placed in a pocket of clothes, and the quality of service can be greatly improved by using a speaker or microphone that is superior to the performance of the wireless terminal speaker or microphone.
In addition, the wireless terminal user can set and register the application to convert the text information received from the various types of text information transmission and reception applications to the voice information and the item information to be converted through the single UI (User Interface). If the wireless terminal is located in the vehicle, it is easier and more convenient to use the voice information conversion output service of the text information, and through this, do not separately install a TTS engine or STT engine for each application for sending and receiving text information It has economic effects that do not have to.
The following drawings, which are attached to this specification, illustrate preferred embodiments of the present invention, and together with the detailed description of the present invention serve to further understand the technical spirit of the present invention. It should not be construed as limited to.
1 is a diagram showing an overall communication network configuration according to an embodiment of the present invention.
2 is a diagram illustrating a detailed configuration of a user wireless terminal according to an embodiment of the present invention.
3 is a diagram illustrating a user interface (UI) according to an embodiment of the present invention.
4 is a diagram illustrating a process of converting text information into voice information and outputting the same after confirming whether a wireless terminal is located in a vehicle according to an exemplary embodiment of the present invention.
5 is a diagram illustrating a process of confirming whether a wireless terminal is located in a vehicle according to an exemplary embodiment of the present invention.
6 is a diagram illustrating a process of extracting text information corresponding to pre-registered application information according to an embodiment of the present invention.
7 is a diagram illustrating a process of converting text information corresponding to pre-registered application and item information into voice information according to an embodiment of the present invention and outputting the voice information.
8 is a diagram illustrating a process of converting a user's reply voice information into text information and transmitting the same according to an embodiment of the present invention.
9 is an embodiment diagram according to an embodiment of the present invention.
Hereinafter, with reference to the accompanying drawings and description will be described in detail the operating principle of the preferred embodiment of the present invention. However, the drawings and the following description shown below are for the preferred method among various methods for effectively explaining the features of the present invention, the present invention is not limited only to the drawings and description below. In addition, in the following description of the present invention, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted. Terms to be described later are terms defined in consideration of functions in the present invention, which may vary according to intentions or customs of users or operators. Therefore, the definition should be made based on the contents throughout the present invention.
As a result, the technical spirit of the present invention is determined by the claims, and the following examples are one means for efficiently explaining the technical spirit of the present invention to those skilled in the art to which the present invention pertains. It is only.
1 is a diagram showing the overall communication network configuration according to an embodiment of the present invention.
In more detail, FIG. 1 shows that a user wireless terminal 100 is connected to at least one text information relay server 110 corresponding to a plurality of text information transmission and reception applications provided in the wireless terminal 100 through a communication network. Receives the text information transmitted through the information transmitting and receiving terminal 120, and after confirming that the wireless terminal 100 is located in the vehicle through the short-range communication channel connection with the vehicle communication device 130 connected through the local area network, the text information Shows a configuration for outputting the converted voice information through the vehicle communication device 130 connected through a local area network.
The text information relay server 110 according to the present invention relays the text information transmitted through the text information transmission and reception terminal 120 to the user wireless terminal 100, and the text transmitted through the user wireless terminal 100. It relays and transmits information to the text information transmission and reception terminal 120.
According to the present invention, the text information relay server 110, the text information transmission and reception application provided in the user wireless terminal 100 is SMS (Short Message Service), LMS (Long Message Service), MMS (Multimedia Message Services) In the case of a message transmission and reception application including one or more, it may be a message service center server on the communication network, when the text information transmission and reception application provided in the user wireless terminal 100 is an email application, the email relay server 110 on the communication network If the text information transmission and reception application provided in the user wireless terminal 100 is an instant messenger application, it may be a relay server 110 for supplying and operating each messenger application, the user wireless terminal 100 If the provided text information transmission and reception application is a Social Networking Service (SNS) application, Each of the SNS application can be a relay server (110) that supplies and operations.
The text information transmission and reception terminal 120 according to the present invention transmits a message or an e-mail including one or more SMS, LMS and MMS to the user wireless terminal 100, or communicates a messenger or SNS with the user wireless terminal 100. Means the terminal to be executed, and includes all terminals capable of transmitting and receiving text such as mobile phones, smartphones, tablet PCs, PCs, telematics.
The user wireless terminal 100 according to the present invention receives the short-range communication connection information between the wireless terminal 100 and the communication device 130 from the communication device 130 provided in the vehicle, or the wireless terminal 100 ) Receives a signal corresponding to the login information of the wireless terminal 100 from the communication device 130, or to the Bluetooth device and the wireless terminal 100 provided in the vehicle The provided Bluetooth device is paired or directly receives the vehicle boarding information of the wireless terminal 100 from the communication device 130, or the wireless terminal on the power supply device provided in the vehicle through the communication device 130. (100) After receiving the information connected to the charging terminal, after confirming that the wireless terminal 100 is located in the vehicle, when the wireless terminal 100 is located in the vehicle, transmitting and receiving text information provided in the wireless terminal 100 1st to Nth The vehicle communication device 130 (navigation device, telematics device) converting text information received through one of the applications into voice information, and converting the converted voice information into the wireless terminal 100 through a short range communication channel. And a vehicle communication device 130 in which the voice information of the user of the wireless terminal 100 is connected to the wireless terminal 100 through a short-range communication channel. When received through, after converting the received voice information into text information, the converted text information to be transmitted to the text information transmission and reception terminal that has transmitted the text information to the wireless terminal 100 through the application for text information transmission and reception It performs the role of processing.
In addition, the wireless terminal 100 provides a user with an interface for selecting one or more of the first to Nth (N = 2,3..N) applications for text information transmission and reception provided in the wireless terminal 100. And store and register information on at least one character information transmission / reception application selected by the user of the wireless terminal 100 through the interface in advance on the memory 220, and whether the wireless terminal 100 is located in the vehicle. After confirming, and determining the activation or deactivation of the extraction function of the character information received through the application for transmitting and receiving the character information, previously stored and stored in the memory 220 in response to the confirmation result, When the text information is received by the N application, the text information transmission and reception application for receiving the text information is previously stored and registered in the memory 220. If the beam for transmitting and receiving applications, and extracts or perform further serves to receive the character information transmitted through the transmitting and receiving applications for the reservoir, which is for the registered character information transmitting and receiving application receives text information in the memory 220.
In addition, the wireless terminal 100 of the text information received by the selected application other than one or more application selection function of the first to N-th (N = 2,3..N) applications for transmitting and receiving the text information through the interface The mobile terminal 100 may further perform a role of selecting or inputting item information to be converted and outputted into voice information. In this case, the wireless terminal 100 transmits and receives the selected text information to the selected item information. Is connected to the application information for storing and registering on the memory 220, and when the character information is received in the first to N-th application for transmitting and receiving the character information, the application for transmitting and receiving the character information on the memory 220 After checking the item information connected with the information, only the character information corresponding to the item information among the text information is selected and added. Or delivery can be performed more acts recipient.
Here, the item information may be one of message sender information and message content information when the application is a message transmission / reception application including one or more Short Message Service (SMS), Long Message Service (LMS), and Multimedia Message Services (MMS). The above item information may be included. When the application is an e-mail transmission / reception application, the information may include at least one item information among e-mail sender information, e-mail subject information, and e-mail body content information. When the application is an instant messenger application, One or more item information among messenger user information and messenger content information may be included.
Here, according to the present invention, the text information transmission and reception application, a message transmission and reception application including one or more Short Message Service (SMS), Long Message Service (LMS), Multimedia Message Services (MMS), email transmission and reception application, It may include one or more instant messenger application and SNS application, wherein the text information, if the application is a message transmission and reception application including one or more SMS, LMS, MMS, and includes one or more message sender information and message content information When the application is an e-mail transmission and reception application, and includes at least one email sender information, email subject information and email body content information, if the application is an instant messenger application, and includes at least one messenger user information and messenger content information, Group may include an application SNS if the application, one or more of the user ID information and contents information.
According to the present invention, the vehicle communication device 130 is connected to the user wireless terminal 100 through short-range communication (all short-range communication including Bluetooth communication, Wi-Fi communication, etc.), the wireless terminal 100 to the Transmitting short-range communication connection information between communication devices 130, or when the wireless terminal 100 logs in to the communication device 130, transmits a signal corresponding to the login information to the wireless terminal 100, or the vehicle The pairing information between the Bluetooth device provided in the wireless device 100 and the Bluetooth device provided in the wireless terminal 100 is transmitted to the wireless terminal 100 or the vehicle boarding information of the wireless terminal 100 is directly transmitted to the wireless terminal 100. Or, by transmitting the information connected to the wireless terminal 100 charging terminal on the power supply provided in the vehicle to the wireless terminal 100, so as to determine whether the wireless terminal 100 is located in the vehicle. It plays a role.
In addition, the vehicle communication device 130 receives and outputs voice information converted from text information received through the wireless terminal 100 through a short range communication channel and a reply voice of a user corresponding to the voice information. After receiving the information, and serves to transmit to the wireless terminal 100 through the short-range communication channel.
According to the present invention, the vehicle communication device 130 includes all devices capable of short-range communication, such as telematics provided in a vehicle, an audio device provided in the vehicle, a Bluetooth device provided in the vehicle, and a handsfree device provided in the vehicle. It can include one or more.
2 is a diagram illustrating a detailed configuration of a user wireless terminal 100 according to an embodiment of the present invention.
2 is applied to the information processing according to the present invention, the wireless terminal 100 operating in a mobile communication or communication system, although not shown in detail in the drawing, the wireless terminal 100 in terms of hardware It may include an external body, a speaker, a microphone, a keypad, a liquid crystal display (LCD), an antenna, a battery, and the like, and may include all the functions and configurations of the wireless terminal 100 that can communicate now or in the future. Can be.
Preferably, the main functional configuration of the wireless terminal 100 according to the present invention with reference to the above-described hardware characteristics, as shown, the
For example, one of the application for text information transmission and reception, which is one of the embodiments of the present invention, is provided with a function of directly converting text information into voice information and outputting the voice information into text information. The configuration of the
Referring to FIG. 2, the
That is, the
In addition, the
Here, the item information may be one of message sender information and message content information when the application is a message transmission / reception application including one or more Short Message Service (SMS), Long Message Service (LMS), and Multimedia Message Services (MMS). The above item information, and when the application is an e-mail transmission / reception application, the e-mail sender information, the e-mail subject information, and the e-mail body content information includes one or more item information. At least one item of messenger content information is included.
3 is a temporary view of an interface screen according to the present invention. As shown in FIG. 3, the
For example, in the case of an SMS application, when a user receives text information such as an SMS, the user may select the sender information and the message content information to be converted to voice. In addition, the e-mail sender information and the e-mail subject information can be selected as an item to be converted into voice. In the case of M & Messenger application, only the sender information which is a messenger conversation partner can be selected to be converted into voice and output.
Here, the sender information may be sender name information recorded in the contact information, Calling Identification Display (CID) information, ID information, or e-mail address information.
The registration unit 215 according to the embodiment of the present invention mutually exchanges information of at least one character information transmission / reception application selected by the user of the wireless terminal 100 and item information corresponding to each selected application through the
The memory 220 according to the present invention serves to store application information for transmitting / receiving text information selected by a user stored and registered through the registration unit 215 and item information corresponding to each selected application.
Of course, the memory 220 is when a predetermined program routine (or code) and / or program data (eg, program routine (or code)) for controlling the overall operation of the wireless terminal 100 is performed A general term for a memory 220 element that stores input and output information and / or data, and includes a read only memory (ROM) corresponding to the read only memory 220 and a flash memory 220 capable of reading / writing (Flash). Memory (FM), and EEPROM (Electrically Erasable and Programmable Read Only Memory), etc., and may further serve to store a variety of applications provided in the wireless terminal (100).
The short range communication unit 265 according to the present invention serves to connect a short range communication channel between the communication device 130 provided in the vehicle and the wireless terminal 100.
The confirmation unit 235 according to the present invention serves to determine whether the wireless terminal 100 is located in a vehicle, and is provided in the vehicle through a short range communication channel connected through the short range communication unit 265. When receiving the short-range communication connection information between the wireless terminal 100 and the communication device 130 from the communication device 130, or when the wireless terminal 100 logs in to the communication device 130, the communication device ( Receive a signal corresponding to the login information of the wireless terminal 100 from the 130, or the Bluetooth device provided in the vehicle and the Bluetooth device provided in the wireless terminal 100 is paired, or from the communication device 130 Receiving the vehicle boarding information of the wireless terminal 100 directly, or via the communication device 130 receives the information connected to the wireless terminal 100 charging terminal on the power supply provided in the vehicle, the wireless terminal ( 100) Make sure the amount of your location.
In addition, according to another embodiment of the present invention, when the character information is received through the communication unit 230, the confirmation unit 235, the application for receiving the character information is pre-stored in the memory 220 It further performs the role of checking whether the registered text information transmission and reception application.
The
That is, the
The control unit according to the present invention controls to activate or deactivate the function of the extraction unit 240 according to the determination result of the
The communication unit 230 according to the present invention serves to receive various text information received by the wireless terminal 100 and to transmit various text information from the wireless terminal 100.
After the extraction unit 240 according to an embodiment of the present invention is activated by the control unit, text information is received through the communication unit 230 to the first to Nth applications for transmitting and receiving text information, and the confirmation unit When it is determined whether the wireless terminal 100 is located in the vehicle through 235, it extracts the text information received by the text information transmission / reception application or receives the text information through the text transmission / reception application. do.
In addition, the extraction unit 240 is connected to the application for transmitting and receiving the text information on the memory 220, when there is previously stored and registered item information, the character information corresponding to the item information of the received text information (For example, sender information or e-mail subject information, etc.) may be extracted or received through the application for transmitting and receiving text information.
The conversion unit 245 according to the present invention serves to convert the text information extracted or received through the extraction unit 240 into voice information.
In addition, when the voice information corresponding to the reply of the user is received through the vehicle communication apparatus 13 connected to the wireless terminal 100 through the short range communication in the subsequent processing, the conversion unit 245 receives the received voice information. It converts into text information.
According to the present invention, although the configuration of the conversion unit 245 is shown as a single configuration on the drawing, the conversion unit 245 (1) and the voice information to convert the text information to the voice information according to the method of the skilled person and the text information It may be configured by dividing into a conversion unit (245) (2) to convert to.
According to the present invention, the conversion unit 245 is '^^', '~~', ');' In the case of a character that includes emotions expressed through various special characters, such as 'haha', which is voice information corresponding to the emotion, the character may be converted to 'haha'. When it is converted into character information, it can be converted into '^^', which is an emotional expression containing a special character.
Of course, the conversion unit 245 may change the size of the converted character according to the size of the voice when converting the emotional expression containing such a special character to the voice or text, the size of the voice according to the size of the received text Can be different.
Text to Speech (TTS) function for converting text information, which is the main function of the conversion unit 245, according to the present invention, and STT (Speech to Text) function for converting voice information to text information, are the same as those of the conventional TTS engine. The functions of the STT engine or the functions of the TTS engine and the STT engine that can be developed in the future may be used, and detailed descriptions thereof will be omitted since they are not the main idea of the present invention.
The processing unit 250 according to the present invention serves to process the voice information converted through the conversion unit 245 to be output through the vehicle communication apparatus 130 connected to the wireless terminal 100 through a short range communication channel. Do this.
In addition, the processor 250 receives voice information corresponding to the reply of the user through the vehicle communication device 130 connected to the wireless terminal 100 through a short range communication channel, and through the conversion unit 245. When the received voice information is converted into text information, the voice information is further processed to be transmitted to the text information transmitting and receiving terminal 120.
The output unit 255 according to the present invention transmits the voice information to the vehicle communication device 130 provided in the vehicle and processes the output signal, and the input unit 260 receives voice information from the vehicle communication device 130. Do this.
According to the present invention, all or part of the functions of the respective components provided in the wireless terminal 100 may be implemented in the form of an application which is a kind of program provided in the wireless terminal 100.
In this case, the application may include a function of providing an interface for selecting at least one of the first to Nth (N = 2,3..N) applications for transmitting and receiving text information provided in the wireless terminal 100, A function of storing and registering information on at least one character information transmitting / receiving application selected by the user of the wireless terminal 100 through the interface on the memory 220, and checking whether the wireless terminal 100 is located in the vehicle; A function for determining activation or deactivation of the extraction function of the character information received through the character information transmission / reception application previously stored and registered in the memory 220 in response to the confirmation result, and the first to Nth applications for transmission and reception of the character information; When the text information is received in response to the determination result, the text information transmission and reception application for receiving the text information is stored in the memory (2). 20) if the application for transmitting and receiving text information registered and stored in the memory 220, the text information received by the application for transmitting and receiving text information registered and stored in the memory 220 or received through the application for transmitting and receiving text information A function, a function of converting the extracted or received text information into voice information, and a process of outputting the converted voice information through the vehicle communication apparatus 130 connected to the wireless terminal 100 through short-range communication. And a function of receiving voice information of a user of the wireless terminal 100 through the vehicle communication apparatus 130, converting the received voice information into text information, and converting the text information into the text. Implement one or more of the functions to process the information to be transmitted through the application for transmitting and receiving information.
Hereinafter, each information processing process according to the present invention will be described in more detail.
4 is a diagram illustrating a process of converting text information into voice information and outputting the text information after confirming whether the wireless terminal 100 is located in a vehicle according to an exemplary embodiment of the present invention.
First, the wireless terminal 100 connects a near field communication channel with a vehicle communication device 130 or a hands-free device provided in a vehicle through near field communication (S410).
Although the process (S410) is illustrated as being preceded by the process (S420) on the drawing, after the process (S420), it may be included in any process before transmitting the converted voice information to the vehicle communication device 130 or the hands-free device. Do.
Thereafter, the wireless terminal 100 checks whether the wireless terminal 100 is located in the vehicle through the identification unit 235 (S420).
Here, the process of confirming whether or not the wireless terminal 100 is located in the vehicle through the identification unit 235, from the communication device 130 provided in the vehicle to the wireless terminal 100 and the communication device 130 Receive near field communication connection information, or when the wireless terminal 100 logs into the communication device 130, receives a signal corresponding to the login information of the wireless terminal 100 from the communication device 130 or The Bluetooth device provided in the vehicle and the Bluetooth device provided in the wireless terminal 100 are paired or directly receive vehicle boarding information of the wireless terminal 100 from the communication device 130 or the communication device 130. Receive information connected to the charging terminal of the wireless terminal 100 on the power supply provided in the vehicle to determine whether the wireless terminal 100 is located in the vehicle.
When it is determined whether the wireless terminal 100 is located in the vehicle through the process (S420) (S430), the wireless terminal 100 corresponds to all or some item information received from the wireless terminal 100. The item is converted into voice information (S440).
If it is not determined whether the wireless terminal 100 is located in the vehicle through the process (S420) (S450), the wireless terminal 100 outputs the text information received by the wireless terminal 100 on the screen. do.
Thereafter, the wireless terminal 100 processes the voice information converted in step S440 to be output through the hands-free device or the in-vehicle communication device 130 through the short-range communication channel connected through step S410 (S460). ).
5 is a diagram illustrating a process of selecting and registering a voice information conversion target application and item information among a plurality of text information transmission and reception applications according to an embodiment of the present invention.
First, in response to a user request, the user wireless terminal 100 may transmit a first application to Nth (N = 2,3) for text information transmission / reception provided in the wireless terminal 100 through a specific application or the
Of course, in step S510, the interface screen may output an interface for selecting item information to be converted and outputted from voice text information received through an application selected outside the application selection through a single screen.
When the user selects an application in operation S510, the user may determine whether item information to be output is converted or input from text information received by the selected application to be further selected or input in operation S530.
If the user does not select an application in step S510, the interface screen ends and waits for a restart (S540).
In operation S530, when the user further selects or inputs item information to be converted and output from text information received by the selected application (S550), the wireless terminal 100 may use a specific application or registration unit 215. The information of the application for transmitting / receiving the selected character information through the interface and the item information corresponding to each application are connected and stored and registered in the memory 220 (S560).
If the separate item information is not selected or input in step S530 (S570), the wireless terminal 100 transmits the information of the text information transmission / reception application selected through the interface through a specific application or the registration unit 215. Is stored on the memory 220 and registered (S580).
The step S580 may correspond to a case in which the item information is previously set as a default or when all text information received without separate item classification is converted into voice information.
6 is a diagram illustrating a process of extracting text information corresponding to pre-registered application information according to an embodiment of the present invention.
First, the wireless terminal 100 checks whether the wireless terminal 100 is located in the vehicle through the identification unit 235 (S610).
In operation S610, when it is determined that the wireless terminal 100 is located in the vehicle (S620), the application or the control unit of the wireless terminal 100 is previously stored and registered in the memory 220 for transmitting and receiving text information. Activate the extraction function of the text information received through the application (S630).
If, in the process (S610), the wireless terminal 100 is not confirmed to be located in the vehicle (S640), extraction of the character information received through the application for transmitting and receiving the character information previously stored in the memory 220, Keep the function disabled.
Thereafter, the wireless terminal 100 receives various text information through the communication unit 230 (S650).
Thereafter, the wireless terminal 100 checks whether a specific application or an application that receives the text information through the verification unit 235 is an application for transmitting and receiving text information that is previously stored and registered in the memory 220 (S660).
As a result of the check through the step (S660), when the application for receiving the text information is an application for transmitting and receiving registered text information in the memory 220 (S670), the wireless terminal 100 is a specific application or extraction Character information received by the character information transmission / reception application previously stored and registered in the memory 220 through the unit 240 may be extracted or received through the character information transmission / reception application (S680).
If, as a result of the check through the step (S660), if the application for receiving the text information is not stored in the memory 220, the application for transmitting and receiving the registered text information (S690), the process before the process (S650) do.
7 is a diagram illustrating a process of converting text information corresponding to pre-registered application and item information into voice information according to an embodiment of the present invention and outputting the voice information.
First, the wireless terminal 100 checks the item information connected to the application information for transmitting and receiving the character information previously stored and registered in the memory 220 by the application that receives the character information through the memory 220 (S710). ).
The process (S710) is carried out after receiving or receiving the entire text information through the (S680) process, or proceeds before the process (S680) according to another embodiment of the person skilled in the art, the process in the (S680) Only the character information corresponding to the item information identified through the process (S710) can be selected to be extracted or delivered.
If item information exists in step S710 (S720), the wireless terminal 100 converts only the character information of the item corresponding to the item information from the character information into voice information through a specific application or conversion unit 245. The conversion process (S730).
If the item information does not exist in the process (S710) (S740), the wireless terminal 100 converts the entire text information into voice information through a specific application or conversion unit 245 (S750). .
Although not shown separately on the drawing, the specific application or converter 245 of the wireless terminal 100 is '^^', '~~', ');' In the case of a character containing emotions expressed through various special characters, such as 'ha ha', which is voice information corresponding to the emotions, the character may be converted. The size of the voice may vary depending on the size of the voice.
Thereafter, the wireless terminal 100 processes the converted voice information through a specific application or processing unit 250 to be output through the vehicle communication device 130 connected to the wireless terminal 100 through a short range communication channel. (S760).
8 is a diagram illustrating a process of converting a user's reply voice information into text information and transmitting the same according to an embodiment of the present invention.
8 illustrates a process for a case in which the user of the wireless terminal 100 sends a reply to the text information transmitting / receiving terminal 120 that has transmitted the text information corresponding to the other party after the output of the voice information.
First, the wireless terminal 100 receives voice information corresponding to a reply of a user transmitted through a vehicle communication device 130 connected to the wireless terminal 100 through a short range communication channel through a specific application or input unit 260. Receive (S810).
Thereafter, the wireless terminal 100 converts the received voice information into text information through a specific application or conversion unit 245 (S820).
Of course, when the conversion unit 245 includes 'haha' corresponding to an onomatopoeic word among the received voice information, the emotion is converted into a special character '^^' or the like when converting the onomatopoeia speech information into text information. The converted character information may be converted and the size of the converted character may be changed according to the size of the voice which is an onomatopoeia.
Thereafter, the wireless terminal 100 confirms an application for transmitting / receiving the text information to transmit the converted text information through a specific application or the processing unit 250, and transmits the converted text information to the confirmed text information transmitting / receiving application or the The text information transmission and reception application is controlled to process the converted text information to be transmitted to the text information transmission and reception terminal 120 (S830).
Here, the text information transmission and reception application for transmitting the converted text information may be a text information transmission and reception application for receiving text information transmitted to the user wireless terminal 100 before the user's reply.
9 is an embodiment diagram according to an embodiment of the present invention.
As shown in FIG. 9, when a friend named 'Young-seok Lee' transmits text information through a smartphone, the wireless terminal 100 of the user who is driving the vehicle converts the text information into voice information through a short-range communication channel. Output through the connected vehicle communication device 130, and converts the reply voice information received through the vehicle communication device 130 to the text information and transmits to the smartphone of the friend 'Yeokyi Lee'.
100: user wireless terminal 110: text information relay server
120: text information transmission and reception terminal 130: vehicle communication device
200: control unit 210: interface unit
215: Register 220: Memory
225: judgment unit 230: communication unit
235 verification unit 240 extraction unit
245 conversion unit 250 processing unit
255: output unit 260: input unit
265: Near field communication unit
Claims (14)
An interface unit for providing an interface for selecting one or more of the first to Nth applications (where N is a natural number of two or more) for text information transmission and reception provided in the wireless terminal;
A registration unit for storing and registering one or more application information for transmitting and receiving text information selected by a wireless terminal user through the interface unit in a memory;
A short range communication unit connecting a short distance communication channel with a communication device or a hands-free device provided in a vehicle;
A confirmation unit for confirming whether the wireless terminal is located in the vehicle based on the local area communication channel connection information through the local area communication unit;
When character information is received through one of the first to Nth applications for text information transmission and reception while the confirmation unit confirms that the wireless terminal is located in the vehicle, the one text information transmission / reception application is stored in the memory. In the case of text information transmission / reception application corresponding to the application information for transmission and reception of registered text information, the text information received by the one text information transmission / reception application is extracted or transmitted through the one text information transmission / reception application. Receiving extractor;
A conversion unit (1) for converting the text information extracted by the extraction unit into voice information;
A processor configured to transmit the converted voice information to the hands free device or the vehicle communication device through a short range communication channel connected through the short range communication unit so that the converted voice information is output from the hands free device or the vehicle communication device;
An input unit for receiving voice information of a wireless terminal user input to the hands-free device or a vehicle communication device through the short range communication channel;
And a conversion unit 2 for converting the input voice information into text information.
The processing unit,
Processing the converted text information to be transmitted through the text information transmission / reception application;
Wireless terminal.
Receiving short-range communication connection information between the wireless terminal and the communication device from a communication device provided in the vehicle;
When the wireless terminal logs in to the communication device, receives a signal corresponding to the login information of the wireless terminal from the communication device;
The Bluetooth device provided in the vehicle and the Bluetooth device provided in the wireless terminal are paired or
Directly receiving vehicle boarding information of the wireless terminal from the communication device;
Receiving the information connected to the wireless terminal charging terminal on the power supply provided in the vehicle through the communication device, to determine whether the wireless terminal is located in the vehicle,
Wireless terminal.
The interface unit,
Receiving further selection or input of the item information to be converted to voice information among the text information received by the selected application other than the application selection function of the first to Nth (N is a natural number of two or more) applications for transmitting and receiving the text information; Features,
The registration unit,
And storing and registering the item information selected or input through the interface with application information for transmitting and receiving the selected text information in a memory.
The extraction unit,
After checking the item information connected to the application information for transmitting and receiving the character information on the memory, and extracts or receives only the character information corresponding to the item information from the character information,
The item information,
If the application is a message transmission and reception application including one or more Short Message Service (SMS), Long Message Service (LMS), Multimedia Message Services (MMS), includes at least one item information of the message sender information and the message content information,
When the application is an e-mail transmission and reception application, and includes one or more item information of the email sender information, email subject information and email body content information,
When the application is an instant messenger application, the application includes one or more item information of messenger user information and messenger content information.
Wireless terminal.
Characterized in that the same configuration or separate configuration,
Wireless terminal.
Message sending and receiving applications including one or more short message service (SMS), long message service (LMS), multimedia message services (MMS),
Email sending and receiving applications,
Instant messenger applications,
Containing one or more Social Networking Service (SNS) applications,
Wireless terminal.
When the application is a message transmission and reception application including one or more Short Message Service (SMS), Long Message Service (LMS), Multimedia Message Services (MMS), and includes one or more message sender information and message content information,
When the application is an e-mail transmission and reception application, the application includes one or more email sender information, email subject information, and email body content information.
When the application is an instant messenger application, the application includes one or more messenger user information and messenger content information.
Wireless terminal.
An interface providing step of providing an interface for selecting at least one of first to Nth applications (where N is a natural number of two or more) for text information transmission and reception provided in the wireless terminal;
A registration step of storing and registering one or more application information for transmitting and receiving text information selected by a user of a wireless terminal through the interface in a memory;
Connecting a short distance communication channel with a communication device or a hands-free device provided in the vehicle;
A confirmation step of confirming whether the wireless terminal is located in the vehicle based on the short range communication channel connection information through the short range communication unit;
When text information is received through one of the first to Nth applications for text information transmission and reception while confirming that the wireless terminal is located in a vehicle, the one text information transmission / reception application is previously stored in the memory. Extracting text information received by the one text information transmitting / receiving application or being transmitted through the one text information transmitting / receiving application if the text information transmitting / receiving application corresponds to the registered text information transmitting / receiving application information;
A voice information conversion step of converting the text information extracted in the extraction step into voice information;
Confirming whether a hands-free device or a vehicle communication device connected to the wireless terminal through short-range communication is activated, and if it is deactivated, activating it;
A voice information processing step of transmitting the converted voice information to the activated hands-free device or the vehicle communication device through a short range communication channel so as to output the converted voice information through the hands-free device or the vehicle communication device;
An input step of receiving voice information of a wireless terminal user input through the hands-free device or the vehicle communication device through a short range communication channel;
A text information converting step of converting the input voice information into text information;
And a text information processing step of processing the converted text information to be transmitted through the text information transmission and reception application.
Information processing method in a wireless terminal.
Receiving short-range communication connection information between the wireless terminal and the communication device from a communication device provided in the vehicle;
When the wireless terminal logs in to the communication device, receives a signal corresponding to the login information of the wireless terminal from the communication device;
The Bluetooth device provided in the vehicle and the Bluetooth device provided in the wireless terminal are paired or
Directly receiving vehicle boarding information of the wireless terminal from the communication device;
Receiving the information connected to the wireless terminal charging terminal on the power supply provided in the vehicle through the communication device, to determine whether the wireless terminal is located in the vehicle,
Information processing method in a wireless terminal.
The interface providing step,
Characteristic information of the first to Nth applications for transmitting and receiving the text information (where N is a natural number of two or more) is selected or received, further selecting or inputting item information to be converted into voice information among text information received by the selected application. ,
The registration step,
And storing and registering the item information selected or input through the interface with application information for transmitting and receiving the selected text information in a memory.
The extraction step,
After checking the item information connected to the application information for transmitting and receiving the character information on the memory, characterized in that only the character information corresponding to the item information of the character information to extract or receive,
Information processing method in a wireless terminal.
Priority Applications (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| KR1020110136908A KR101979605B1 (en) | 2011-12-18 | 2011-12-18 | Wireless Terminal and Information Processing Method |
| PCT/KR2012/011113 WO2013094987A1 (en) | 2011-12-18 | 2012-12-18 | Wireless terminal, and information-processing method for the wireless terminal |
| US14/366,415 US20140316781A1 (en) | 2011-12-18 | 2012-12-18 | Wireless terminal and information processing method of the wireless terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| KR1020110136908A KR101979605B1 (en) | 2011-12-18 | 2011-12-18 | Wireless Terminal and Information Processing Method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| KR20130070678A KR20130070678A (en) | 2013-06-28 |
| KR101979605B1 true KR101979605B1 (en) | 2019-08-29 |
Family
ID=48865345
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| KR1020110136908A KR101979605B1 (en) | 2011-12-18 | 2011-12-18 | Wireless Terminal and Information Processing Method |
Country Status (1)
| Country | Link |
|---|---|
| KR (1) | KR101979605B1 (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR100701519B1 (en) * | 2005-10-19 | 2007-03-29 | 삼성전자주식회사 | Mobile Communication Terminal and Method capable of outputting selectively converting Short Message Service Message to voice sound according to a movement speed |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20050077989A (en) * | 2004-01-30 | 2005-08-04 | 에스케이텔레텍주식회사 | Method for executing mobile phone's specific function in driving mode |
| KR101079692B1 (en) * | 2005-06-13 | 2011-11-04 | 엘지전자 주식회사 | Mobile Communication Terminal and Method for Morse Signal and Analysis and conversion |
| KR20070108613A (en) * | 2006-05-08 | 2007-11-13 | 주식회사 현대오토넷 | Mobile communication terminal control system for vehicle and method thereof |
| KR101513633B1 (en) * | 2008-11-19 | 2015-04-20 | 엘지전자 주식회사 | Call control method for telematics terminal and apparatus thereof |
-
2011
- 2011-12-18 KR KR1020110136908A patent/KR101979605B1/en active IP Right Grant
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR100701519B1 (en) * | 2005-10-19 | 2007-03-29 | 삼성전자주식회사 | Mobile Communication Terminal and Method capable of outputting selectively converting Short Message Service Message to voice sound according to a movement speed |
Also Published As
| Publication number | Publication date |
|---|---|
| KR20130070678A (en) | 2013-06-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP2747389B1 (en) | Mobile terminal having auto answering function and auto answering method for use in the mobile terminal | |
| CN104008749A (en) | Method and apparatus for voice control of a mobile device | |
| KR101932097B1 (en) | Method and apparatus for providing message service using voice of user | |
| US20140316781A1 (en) | Wireless terminal and information processing method of the wireless terminal | |
| EP3355559B1 (en) | Method of augmenting a voice call with supplemental audio | |
| US20150012272A1 (en) | Wireless terminal and information processing method of the wireless terminal | |
| KR101919453B1 (en) | Wireless Terminal and Information Processing Method | |
| KR20190056355A (en) | Wireless Terminal | |
| KR101992358B1 (en) | Wireless Terminal | |
| US20110183725A1 (en) | Hands-Free Text Messaging | |
| US20140329563A1 (en) | Information processing method and system, and recording medium | |
| KR101979605B1 (en) | Wireless Terminal and Information Processing Method | |
| KR101992357B1 (en) | Information Processing Method For Wireless Terminal | |
| US9578478B1 (en) | System for transmitting text message automatically while driving and method for the same | |
| KR101147689B1 (en) | Method for Transmitting/Receiving Contents Message, System, Server, Mobile Communication Terminal And Computer-Readable Recording Medium with Program therefor | |
| KR101983799B1 (en) | Wireless Terminal and Information Processing Method | |
| KR20130070676A (en) | Wireless terminal and information processing method | |
| KR20130074812A (en) | Wireless terminal and information processing method | |
| KR101531828B1 (en) | terminal having function of real time text transmission/reception and mail | |
| EP3089160B1 (en) | Method and apparatus for voice control of a mobile device | |
| KR20130070781A (en) | Recording medium, method and system for information processing | |
| KR20130070677A (en) | Wireless terminal and information processing method | |
| KR20130074811A (en) | Wireless terminal and information processing method | |
| KR20130070675A (en) | Wireless terminal and information processing method | |
| KR20130066468A (en) | Wireless terminal and information processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| A201 | Request for examination | ||
| E902 | Notification of reason for refusal | ||
| E90F | Notification of reason for final refusal | ||
| E701 | Decision to grant or registration of patent right | ||
| GRNT | Written decision to grant |