JP4661074B2 - 情報処理システム、情報処理方法、並びにロボット装置 - Google Patents

情報処理システム、情報処理方法、並びにロボット装置 Download PDF

Info

Publication number
JP4661074B2
JP4661074B2 JP2004113589A JP2004113589A JP4661074B2 JP 4661074 B2 JP4661074 B2 JP 4661074B2 JP 2004113589 A JP2004113589 A JP 2004113589A JP 2004113589 A JP2004113589 A JP 2004113589A JP 4661074 B2 JP4661074 B2 JP 4661074B2
Authority
JP
Japan
Prior art keywords
speech synthesis
phoneme sequence
learning
function
speech
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
JP2004113589A
Other languages
English (en)
Japanese (ja)
Other versions
JP2005300692A5 (https=
JP2005300692A (ja
Inventor
雅博 藤田
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Corp
Original Assignee
Sony Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Corp filed Critical Sony Corp
Priority to JP2004113589A priority Critical patent/JP4661074B2/ja
Priority to US11/101,257 priority patent/US8145492B2/en
Publication of JP2005300692A publication Critical patent/JP2005300692A/ja
Publication of JP2005300692A5 publication Critical patent/JP2005300692A5/ja
Application granted granted Critical
Publication of JP4661074B2 publication Critical patent/JP4661074B2/ja
Anticipated expiration legal-status Critical
Expired - Fee Related legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/08—Speech classification or search
    • G10L15/16—Speech classification or search using artificial neural networks
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L13/00—Speech synthesis; Text to speech systems
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/02—Feature extraction for speech recognition; Selection of recognition unit
    • G10L2015/025—Phonemes, fenemes or fenones being the recognition units

Landscapes

  • Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Manipulator (AREA)
  • Toys (AREA)
JP2004113589A 2004-04-07 2004-04-07 情報処理システム、情報処理方法、並びにロボット装置 Expired - Fee Related JP4661074B2 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2004113589A JP4661074B2 (ja) 2004-04-07 2004-04-07 情報処理システム、情報処理方法、並びにロボット装置
US11/101,257 US8145492B2 (en) 2004-04-07 2005-04-06 Robot behavior control system and method, and robot apparatus

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2004113589A JP4661074B2 (ja) 2004-04-07 2004-04-07 情報処理システム、情報処理方法、並びにロボット装置

Publications (3)

Publication Number Publication Date
JP2005300692A JP2005300692A (ja) 2005-10-27
JP2005300692A5 JP2005300692A5 (https=) 2007-05-24
JP4661074B2 true JP4661074B2 (ja) 2011-03-30

Family

ID=35137593

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2004113589A Expired - Fee Related JP4661074B2 (ja) 2004-04-07 2004-04-07 情報処理システム、情報処理方法、並びにロボット装置

Country Status (2)

Country Link
US (1) US8145492B2 (https=)
JP (1) JP4661074B2 (https=)

Families Citing this family (75)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2000015396A1 (en) * 1998-09-10 2000-03-23 Sony Corporation Robot apparatus, method of controlling robot apparatus, method of display, and medium
US20040073690A1 (en) 2002-09-30 2004-04-15 Neil Hepworth Voice over IP endpoint call admission
US7359979B2 (en) 2002-09-30 2008-04-15 Avaya Technology Corp. Packet prioritization and associated bandwidth and buffer management techniques for audio over IP
US7978827B1 (en) 2004-06-30 2011-07-12 Avaya Inc. Automatic configuration of call handling based on end-user needs and characteristics
US7921066B2 (en) * 2005-10-12 2011-04-05 Henry Van Dyke Parunak Characterizing and predicting agents via multi-agent evolution
US8073681B2 (en) * 2006-10-16 2011-12-06 Voicebox Technologies, Inc. System and method for a cooperative conversational voice user interface
US7818176B2 (en) 2007-02-06 2010-10-19 Voicebox Technologies, Inc. System and method for selecting and presenting advertisements based on natural language processing of voice-based input
JP4682217B2 (ja) * 2007-03-07 2011-05-11 パナソニック株式会社 行動制御装置、方法、プログラム
JP5537432B2 (ja) * 2007-10-22 2014-07-02 本田技研工業株式会社 分散人型ロボットアーキテクチャにおける通信ミドルウェアの設計および評価
US8140188B2 (en) * 2008-02-18 2012-03-20 Toyota Motor Engineering & Manufacturing North America, Inc. Robotic system and method for observing, learning, and supporting human activities
US8218751B2 (en) 2008-09-29 2012-07-10 Avaya Inc. Method and apparatus for identifying and eliminating the source of background noise in multi-party teleconferences
KR20100081587A (ko) * 2009-01-06 2010-07-15 삼성전자주식회사 로봇의 소리 인식 장치 및 그 제어 방법
US8260460B2 (en) * 2009-09-22 2012-09-04 GM Global Technology Operations LLC Interactive robot control system and method of use
US8706497B2 (en) * 2009-12-28 2014-04-22 Mitsubishi Electric Corporation Speech signal restoration device and speech signal restoration method
TWI399563B (zh) * 2010-03-16 2013-06-21 Univ Nat Cheng Kung 馬達轉矩和轉速之估測系統與估測方法
FR2962048A1 (fr) * 2010-07-02 2012-01-06 Aldebaran Robotics S A Robot humanoide joueur, methode et systeme d'utilisation dudit robot
US9906838B2 (en) 2010-07-12 2018-02-27 Time Warner Cable Enterprises Llc Apparatus and methods for content delivery and message exchange across multiple content delivery networks
US9015093B1 (en) 2010-10-26 2015-04-21 Michael Lamport Commons Intelligent control with hierarchical stacked neural networks
US8775341B1 (en) 2010-10-26 2014-07-08 Michael Lamport Commons Intelligent control with hierarchical stacked neural networks
KR20120047577A (ko) * 2010-11-04 2012-05-14 주식회사 케이티 대화형 행동모델을 이용한 로봇 인터랙션 서비스 제공 장치 및 방법
FR2969026B1 (fr) 2010-12-17 2013-02-01 Aldebaran Robotics Robot humanoide dote d'un gestionnaire de ses ressources physiques et virtuelles, procedes d'utilisation et de programmation
CN102200787B (zh) * 2011-04-18 2013-04-17 重庆大学 机器人行为多层次集成学习方法及系统
US8594845B1 (en) * 2011-05-06 2013-11-26 Google Inc. Methods and systems for robotic proactive informational retrieval from ambient context
US9186793B1 (en) 2012-08-31 2015-11-17 Brain Corporation Apparatus and methods for controlling attention of a robot
JP2014170295A (ja) * 2013-03-01 2014-09-18 Honda Motor Co Ltd 物体認識システム及び物体認識方法
JP2014240884A (ja) * 2013-06-11 2014-12-25 株式会社東芝 コンテンツ作成支援装置、方法およびプログラム
KR102222122B1 (ko) * 2014-01-21 2021-03-03 엘지전자 주식회사 감성음성 합성장치, 감성음성 합성장치의 동작방법, 및 이를 포함하는 이동 단말기
US9533413B2 (en) * 2014-03-13 2017-01-03 Brain Corporation Trainable modular robotic apparatus and methods
US9364950B2 (en) 2014-03-13 2016-06-14 Brain Corporation Trainable modular robotic methods
US9987743B2 (en) 2014-03-13 2018-06-05 Brain Corporation Trainable modular robotic apparatus and methods
EP2933067B1 (en) * 2014-04-17 2019-09-18 Softbank Robotics Europe Method of performing multi-modal dialogue between a humanoid robot and user, computer program product and humanoid robot for implementing said method
US9375841B1 (en) 2014-05-28 2016-06-28 Google Inc. Robotic devices with ambient indications of joint status
WO2016044290A1 (en) 2014-09-16 2016-03-24 Kennewick Michael R Voice commerce
US9426946B2 (en) 2014-12-02 2016-08-30 Brain Corporation Computerized learning landscaping apparatus and methods
US9840003B2 (en) 2015-06-24 2017-12-12 Brain Corporation Apparatus and methods for safe navigation of robotic devices
US11112781B2 (en) * 2015-07-31 2021-09-07 Heinz Hemken Training an autonomous robot using previously captured data
US9443192B1 (en) 2015-08-30 2016-09-13 Jasmin Cosic Universal artificial intelligence engine for autonomous computing devices and software applications
KR102235166B1 (ko) 2015-09-21 2021-04-02 주식회사 레인보우로보틱스 실시간 로봇 시스템, 로봇 시스템 제어 장치 및 로봇 시스템 제어 방법
WO2017052060A1 (ko) * 2015-09-21 2017-03-30 주식회사 레인보우 계층적 아키텍처를 갖는 실시간 디바이스 제어 시스템 및 이를 이용한 실시간 로봇 제어 시스템
KR20170044849A (ko) * 2015-10-16 2017-04-26 삼성전자주식회사 전자 장치 및 다국어/다화자의 공통 음향 데이터 셋을 활용하는 tts 변환 방법
US10241514B2 (en) 2016-05-11 2019-03-26 Brain Corporation Systems and methods for initializing a robot to autonomously travel a trained route
US9987752B2 (en) 2016-06-10 2018-06-05 Brain Corporation Systems and methods for automatic detection of spills
US10282849B2 (en) 2016-06-17 2019-05-07 Brain Corporation Systems and methods for predictive/reconstructive visual object tracker
US10016896B2 (en) 2016-06-30 2018-07-10 Brain Corporation Systems and methods for robotic behavior around moving bodies
KR20180018211A (ko) * 2016-08-12 2018-02-21 엘지전자 주식회사 자가 학습 로봇
US9864933B1 (en) 2016-08-23 2018-01-09 Jasmin Cosic Artificially intelligent systems, devices, and methods for learning and/or using visual surrounding for autonomous object operation
US10650307B2 (en) * 2016-09-13 2020-05-12 International Business Machines Corporation Neuromorphic architecture for unsupervised pattern detection and feature learning
CN109844771B (zh) * 2016-09-15 2024-04-23 谷歌有限责任公司 用于机器人代理的控制策略
US10183590B2 (en) * 2016-09-23 2019-01-22 Faraday&Future Inc. Electric vehicle battery monitoring system
US20180158458A1 (en) * 2016-10-21 2018-06-07 Shenetics, Inc. Conversational voice interface of connected devices, including toys, cars, avionics, mobile, iot and home appliances
US10274325B2 (en) 2016-11-01 2019-04-30 Brain Corporation Systems and methods for robotic mapping
US10001780B2 (en) 2016-11-02 2018-06-19 Brain Corporation Systems and methods for dynamic route planning in autonomous navigation
US10452974B1 (en) 2016-11-02 2019-10-22 Jasmin Cosic Artificially intelligent systems, devices, and methods for learning and/or using a device's circumstances for autonomous device operation
US10723018B2 (en) 2016-11-28 2020-07-28 Brain Corporation Systems and methods for remote operating and/or monitoring of a robot
US10607134B1 (en) 2016-12-19 2020-03-31 Jasmin Cosic Artificially intelligent systems, devices, and methods for learning and/or using an avatar's circumstances for autonomous avatar operation
WO2018112833A1 (en) * 2016-12-22 2018-06-28 Intel Corporation Efficient transferring of human experiences to robots and other autonomous machines
US10377040B2 (en) 2017-02-02 2019-08-13 Brain Corporation Systems and methods for assisting a robotic apparatus
US10852730B2 (en) 2017-02-08 2020-12-01 Brain Corporation Systems and methods for robotic mobile platforms
US10293485B2 (en) 2017-03-30 2019-05-21 Brain Corporation Systems and methods for robotic path planning
EP3611690A4 (en) * 2017-04-10 2020-10-28 Fujitsu Limited RECOGNITION DEVICE, PROCESS AND PROGRAM
US20190019497A1 (en) * 2017-07-12 2019-01-17 I AM PLUS Electronics Inc. Expressive control of text-to-speech content
US10102449B1 (en) 2017-11-21 2018-10-16 Jasmin Cosic Devices, systems, and methods for use in automation
US10474934B1 (en) 2017-11-26 2019-11-12 Jasmin Cosic Machine learning for computing enabled systems and/or devices
US10402731B1 (en) 2017-12-15 2019-09-03 Jasmin Cosic Machine learning for computer generated objects and/or applications
DE102018207513A1 (de) * 2018-05-15 2019-11-21 Siemens Aktiengesellschaft Verfahren zum rechnergestützten Lernen eines Roboters über einen Sprachdialog
CN112601592A (zh) * 2018-08-30 2021-04-02 Groove X 株式会社 机器人及声音生成程序
KR102228866B1 (ko) * 2018-10-18 2021-03-17 엘지전자 주식회사 로봇 및 그의 제어 방법
JP7495125B2 (ja) * 2018-12-03 2024-06-04 Ｇｒｏｏｖｅ Ｘ株式会社 ロボット、音声合成プログラム、及び音声出力方法
JP7309371B2 (ja) 2019-01-25 2023-07-18 株式会社ソニー・インタラクティブエンタテインメント ロボット制御システム
JP7190919B2 (ja) 2019-01-25 2022-12-16 株式会社ソニー・インタラクティブエンタテインメント 画像解析システム
JP7417356B2 (ja) * 2019-01-25 2024-01-18 株式会社ソニー・インタラクティブエンタテインメント ロボット制御システム
CN110900598B (zh) * 2019-10-15 2022-09-23 合肥工业大学 机器人三维运动空间动作模仿学习方法和系统
CN111459452B (zh) * 2020-03-31 2023-07-18 北京市商汤科技开发有限公司 交互对象的驱动方法、装置、设备以及存储介质
US11049497B1 (en) * 2020-10-26 2021-06-29 Cresta Intelligence Inc. Contemporaneous machine-learning analysis of audio streams
US12406194B1 (en) 2021-03-10 2025-09-02 Jasmin Cosic Devices, systems, and methods for machine consciousness

Family Cites Families (39)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0673080B2 (ja) * 1987-11-25 1994-09-14 日本電気株式会社 連続音声認識方式
US4979188A (en) * 1988-04-29 1990-12-18 Motorola, Inc. Spectrally efficient method for communicating an information signal
US5359696A (en) * 1988-06-28 1994-10-25 Motorola Inc. Digital speech coder having improved sub-sample resolution long-term predictor
US5095443A (en) * 1988-10-07 1992-03-10 Ricoh Company, Ltd. Plural neural network system having a successive approximation learning method
CA2078927C (en) * 1991-09-25 1997-01-28 Katsushi Seza Code-book driven vocoder device with voice source generator
JPH0612098A (ja) * 1992-03-16 1994-01-21 Sanyo Electric Co Ltd 音声符号化装置
JPH05335967A (ja) * 1992-05-29 1993-12-17 Takeo Miyazawa 音情報圧縮方法及び音情報再生装置
US5377302A (en) * 1992-09-01 1994-12-27 Monowave Corporation L.P. System for recognizing speech
ATE191107T1 (de) * 1994-12-20 2000-04-15 Dolby Lab Licensing Corp Verfahren und gerät zum anwenden von wellenformprädiktion auf teilbänder in einem perzeptiven kodiersystem
US5682463A (en) * 1995-02-06 1997-10-28 Lucent Technologies Inc. Perceptual audio compression based on loudness uncertainty
US6070140A (en) * 1995-06-05 2000-05-30 Tran; Bao Q. Speech recognizer
JP3002136B2 (ja) * 1996-08-06 2000-01-24 株式会社エイ・ティ・アール知能映像通信研究所 感情変換装置
JPH1097267A (ja) * 1996-09-24 1998-04-14 Hitachi Ltd 声質変換方法および装置
JPH10260692A (ja) * 1997-03-18 1998-09-29 Toshiba Corp 音声の認識合成符号化／復号化方法及び音声符号化／復号化システム
US6134528A (en) * 1997-06-13 2000-10-17 Motorola, Inc. Method device and article of manufacture for neural-network based generation of postlexical pronunciations from lexical pronunciations
JPH1185194A (ja) * 1997-09-04 1999-03-30 Atr Onsei Honyaku Tsushin Kenkyusho:Kk 声質変換音声合成装置
WO1999053476A1 (en) * 1998-04-15 1999-10-21 Fujitsu Limited Active noise controller
US6081780A (en) * 1998-04-28 2000-06-27 International Business Machines Corporation TTS and prosody based authoring system
JPH11311999A (ja) * 1998-04-30 1999-11-09 Seiichiro Hanya 音声認識方法
US6249758B1 (en) * 1998-06-30 2001-06-19 Nortel Networks Limited Apparatus and method for coding speech signals by making use of voice/unvoiced characteristics of the speech signals
US6480820B1 (en) * 1999-09-20 2002-11-12 Advanced Cochlear Systems, Inc. Method of processing auditory data
US6587846B1 (en) * 1999-10-01 2003-07-01 Lamuth John E. Inductive inference affective language analyzer simulating artificial intelligence
JP4465768B2 (ja) * 1999-12-28 2010-05-19 ソニー株式会社 音声合成装置および方法、並びに記録媒体
JP4032273B2 (ja) * 1999-12-28 2008-01-16 ソニー株式会社 同期制御装置および方法、並びに記録媒体
JP2001188555A (ja) * 1999-12-28 2001-07-10 Sony Corp 情報処理装置および方法、並びに記録媒体
JP2001188784A (ja) * 1999-12-28 2001-07-10 Sony Corp 会話処理装置および方法、並びに記録媒体
JP3487250B2 (ja) * 2000-02-28 2004-01-13 日本電気株式会社 符号化音声信号形式変換装置
JP2002049385A (ja) * 2000-08-07 2002-02-15 Yamaha Motor Co Ltd 音声合成装置、疑似感情表現装置及び音声合成方法
US7283961B2 (en) * 2000-08-09 2007-10-16 Sony Corporation High-quality speech synthesis device and method by classification and prediction processing of synthesized sound
JP4296714B2 (ja) * 2000-10-11 2009-07-15 ソニー株式会社 ロボット制御装置およびロボット制御方法、記録媒体、並びにプログラム
JP2002239952A (ja) * 2001-02-21 2002-08-28 Sony Corp ロボット装置、ロボット装置の行動制御方法、プログラム及び記録媒体
JP2002268656A (ja) * 2001-03-09 2002-09-20 Nippon Telegr & Teleph Corp <Ntt> 音声分析合成方法および、この方法を実施する装置、プログラム、プログラムを記録した記録媒体
JP2002264057A (ja) * 2001-03-12 2002-09-18 Sony Corp ロボット装置、ロボット装置の行動制御方法、プログラム及び記録媒体
JP2002337075A (ja) 2001-05-15 2002-11-26 Sony Corp 脚式移動ロボット及びその学習方法
US20030093280A1 (en) * 2001-07-13 2003-05-15 Pierre-Yves Oudeyer Method and apparatus for synthesising an emotion conveyed on a sound
DE60215296T2 (de) * 2002-03-15 2007-04-05 Sony France S.A. Verfahren und Vorrichtung zum Sprachsyntheseprogramm, Aufzeichnungsmedium, Verfahren und Vorrichtung zur Erzeugung einer Zwangsinformation und Robotereinrichtung
JP3558222B2 (ja) 2002-03-15 2004-08-25 ソニー株式会社 ロボットの行動制御システム及び行動制御方法、並びにロボット装置
US7197460B1 (en) * 2002-04-23 2007-03-27 At&T Corp. System for handling frequently asked questions in a natural language dialog service
US7319959B1 (en) * 2002-05-14 2008-01-15 Audience, Inc. Multi-source phoneme classification for noise-robust automatic speech recognition

Also Published As

Publication number Publication date
US8145492B2 (en) 2012-03-27
JP2005300692A (ja) 2005-10-27
US20050240412A1 (en) 2005-10-27

Similar Documents

Publication Publication Date Title
JP4661074B2 (ja) 情報処理システム、情報処理方法、並びにロボット装置
CN100509308C (zh) 用于机器人的行为控制系统和行为控制方法及机器人装置
EP1602457B1 (en) Robot device, behavior control method thereof, and program
US7024276B2 (en) Legged mobile robot and its motion teaching method, and storage medium
US7515992B2 (en) Robot apparatus and emotion representing method therefor
Kawamura et al. Implementation of cognitive control for a humanoid robot
JP4179230B2 (ja) ロボット装置及びその動作制御方法
JP3558222B2 (ja) ロボットの行動制御システム及び行動制御方法、並びにロボット装置
JP4296736B2 (ja) ロボット装置
JP2006110707A (ja) ロボット装置
JP2002059384A (ja) ロボットのための学習システム及び学習方法
JP2007125631A (ja) ロボット装置及びその行動制御方法
JP2002205289A (ja) ロボット装置の動作制御方法、プログラム、記録媒体及びロボット装置
JP4552465B2 (ja) 情報処理装置、ロボット装置の行動制御方法、ロボット装置及びコンピュータ・プログラム
Fujita How to make an autonomous robot as a partner with humans: design approach versus emergent approach
JP2007125629A (ja) ロボット装置及びその行動制御方法
JP4147960B2 (ja) ロボット装置、及びロボット装置の動作制御方法
JP2005321954A (ja) ロボット装置、情報処理システム及び情報処理方法、並びにコンピュータ・プログラム
JP2005193330A (ja) ロボット装置及びその情動表出方法
JP2002337075A (ja) 脚式移動ロボット及びその学習方法
JP4552490B2 (ja) ロボット装置及びその情動制御方法
Kim et al. Intelligence technology for ubiquitous robots
JP2003200369A (ja) ロボット装置及びその制御方法、人工エージェント、記憶制御装置及び記憶制御方法
JP2007125630A (ja) ロボット装置及びその行動制御方法
Lee et al. Building neural network-based behaviour systems for emotion-based pet robots

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20070404

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20070404

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20100225

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20100309

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20100506

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20100601

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20100722

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20100831

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20101019

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20101207

A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20101220

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20140114

Year of fee payment: 3

LAPS Cancellation because of no payment of annual fees