JP6574939B2 - 表示制御装置、表示制御方法、表示制御システムおよび頭部装着ディスプレイ - Google Patents

表示制御装置、表示制御方法、表示制御システムおよび頭部装着ディスプレイ Download PDF

Info

Publication number
JP6574939B2
JP6574939B2 JP2014187583A JP2014187583A JP6574939B2 JP 6574939 B2 JP6574939 B2 JP 6574939B2 JP 2014187583 A JP2014187583 A JP 2014187583A JP 2014187583 A JP2014187583 A JP 2014187583A JP 6574939 B2 JP6574939 B2 JP 6574939B2
Authority
JP
Japan
Prior art keywords
display
head
unit
information
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
JP2014187583A
Other languages
English (en)
Japanese (ja)
Other versions
JP2016061827A5 (enExample
JP2016061827A (ja
Inventor
小田　恭一郎
恭一郎 小田
孝仁 分林
孝仁 分林
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Corp
Original Assignee
Sony Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Corp filed Critical Sony Corp
Priority to JP2014187583A priority Critical patent/JP6574939B2/ja
Priority to EP15757573.9A priority patent/EP3178232A1/en
Priority to US15/325,754 priority patent/US20170151034A1/en
Priority to CN201580048359.9A priority patent/CN106687065A/zh
Priority to PCT/JP2015/004199 priority patent/WO2016042705A1/en
Publication of JP2016061827A publication Critical patent/JP2016061827A/ja
Publication of JP2016061827A5 publication Critical patent/JP2016061827A5/ja
Application granted granted Critical
Publication of JP6574939B2 publication Critical patent/JP6574939B2/ja
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/36—Image-producing devices or illumination devices not otherwise provided for
    • A61B90/37—Surgical systems with images on a monitor during operation
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00043—Operational features of endoscopes provided with output arrangements
    • A61B1/00045—Display arrangement
    • A61B1/00048—Constructional features of the display
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/36—Image-producing devices or illumination devices not otherwise provided for
    • A61B90/361—Image-producing devices, e.g. surgical cameras
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/25—User interfaces for surgical systems
    • A61B2034/258—User interfaces for surgical systems providing specific settings for specific users
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/36—Image-producing devices or illumination devices not otherwise provided for
    • A61B90/361—Image-producing devices, e.g. surgical cameras
    • A61B2090/3616—Magnifying glass
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/36—Image-producing devices or illumination devices not otherwise provided for
    • A61B2090/364—Correlation of different images or relation of image positions in respect to the body
    • A61B2090/365—Correlation of different images or relation of image positions in respect to the body augmented reality, i.e. correlating a live optical image with another image
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/36—Image-producing devices or illumination devices not otherwise provided for
    • A61B90/37—Surgical systems with images on a monitor during operation
    • A61B2090/372—Details of monitor hardware
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/50—Supports for surgical instruments, e.g. articulated arms
    • A61B2090/502—Headgear, e.g. helmet, spectacles
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/20—Surgical microscopes characterised by non-optical aspects
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/90—Identification means for patients or instruments, e.g. tags
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/90—Identification means for patients or instruments, e.g. tags
    • A61B90/98—Identification means for patients or instruments, e.g. tags using electromagnetic means, e.g. transponders
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Surgery (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Veterinary Medicine (AREA)
  • Public Health (AREA)
  • Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Pathology (AREA)
  • Radiology & Medical Imaging (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Physics & Mathematics (AREA)
  • Biophysics (AREA)
  • Optics & Photonics (AREA)
  • Gynecology & Obstetrics (AREA)
  • Endoscopes (AREA)
  • Controls And Circuits For Display Device (AREA)
JP2014187583A 2014-09-16 2014-09-16 表示制御装置、表示制御方法、表示制御システムおよび頭部装着ディスプレイ Expired - Fee Related JP6574939B2 (ja)

Priority Applications (5)

Application Number Priority Date Filing Date Title
JP2014187583A JP6574939B2 (ja) 2014-09-16 2014-09-16 表示制御装置、表示制御方法、表示制御システムおよび頭部装着ディスプレイ
EP15757573.9A EP3178232A1 (en) 2014-09-16 2015-08-21 Display control device, display control method, display control system, and head-mounted display
US15/325,754 US20170151034A1 (en) 2014-09-16 2015-08-21 Display control device, display control method, display control system, and head-mounted display
CN201580048359.9A CN106687065A (zh) 2014-09-16 2015-08-21 显示控制设备、显示控制方法、显示控制系统和头戴式显示器
PCT/JP2015/004199 WO2016042705A1 (en) 2014-09-16 2015-08-21 Display control device, display control method, display control system, and head-mounted display

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2014187583A JP6574939B2 (ja) 2014-09-16 2014-09-16 表示制御装置、表示制御方法、表示制御システムおよび頭部装着ディスプレイ

Publications (3)

Publication Number Publication Date
JP2016061827A JP2016061827A (ja) 2016-04-25
JP2016061827A5 JP2016061827A5 (enExample) 2017-03-02
JP6574939B2 true JP6574939B2 (ja) 2019-09-18

Family

ID=54056238

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2014187583A Expired - Fee Related JP6574939B2 (ja) 2014-09-16 2014-09-16 表示制御装置、表示制御方法、表示制御システムおよび頭部装着ディスプレイ

Country Status (5)

Country Link
US (1) US20170151034A1 (enExample)
EP (1) EP3178232A1 (enExample)
JP (1) JP6574939B2 (enExample)
CN (1) CN106687065A (enExample)
WO (1) WO2016042705A1 (enExample)

Families Citing this family (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016119544A (ja) * 2014-12-19 2016-06-30 セイコーエプソン株式会社 頭部装着型表示装置、頭部装着型表示装置を制御する方法、コンピュータープログラム
US10013808B2 (en) 2015-02-03 2018-07-03 Globus Medical, Inc. Surgeon head-mounted display apparatuses
WO2018134642A1 (en) * 2017-01-19 2018-07-26 Novartis Ag System and method for managing patient data during ophthalmic surgery
JP6896458B2 (ja) * 2017-03-07 2021-06-30 ソニー・オリンパスメディカルソリューションズ株式会社 医療用画像表示装置、および表示制御方法
WO2018195319A1 (en) 2017-04-20 2018-10-25 Intuitive Surgical Operations, Inc. Systems and methods for constraining a virtual reality surgical system
CN107197342B (zh) * 2017-06-16 2019-12-13 深圳创维数字技术有限公司 一种数据处理方法、智能终端及存储介质
JP7017385B2 (ja) * 2017-12-05 2022-02-08 オリンパス株式会社 頭部装着型表示装置、表示システム及び表示方法
JP2019105885A (ja) * 2017-12-08 2019-06-27 セイコーエプソン株式会社 頭部装着型表示装置、情報処理装置、情報処理システム、及び頭部装着型表示装置の制御方法
JP2019129398A (ja) * 2018-01-24 2019-08-01 セイコーエプソン株式会社 頭部装着型表示装置、及び頭部装着型表示装置の制御方法
US11114199B2 (en) 2018-01-25 2021-09-07 Mako Surgical Corp. Workflow systems and methods for enhancing collaboration between participants in a surgical procedure
US20190254753A1 (en) 2018-02-19 2019-08-22 Globus Medical, Inc. Augmented reality navigation systems for use with robotic surgical systems and methods of their use
JP2022523670A (ja) * 2019-01-24 2022-04-26 カオ グループ、インク． 電子ルーペ
CN109889739A (zh) * 2019-03-18 2019-06-14 天津安怀信科技有限公司 医用智能眼镜影像显示系统
EP3744285B1 (en) 2019-05-27 2024-11-27 Leica Instruments (Singapore) Pte. Ltd. Microscope system and method for controlling a surgical microcope
US12133772B2 (en) 2019-12-10 2024-11-05 Globus Medical, Inc. Augmented reality headset for navigated robotic surgery
US12220176B2 (en) 2019-12-10 2025-02-11 Globus Medical, Inc. Extended reality instrument interaction zone for navigated robotic
US11992373B2 (en) 2019-12-10 2024-05-28 Globus Medical, Inc Augmented reality headset with varied opacity for navigated robotic surgery
US11464581B2 (en) 2020-01-28 2022-10-11 Globus Medical, Inc. Pose measurement chaining for extended reality surgical navigation in visible and near infrared spectrums
US11382699B2 (en) 2020-02-10 2022-07-12 Globus Medical Inc. Extended reality visualization of optical tool tracking volume for computer assisted navigation in surgery
US11207150B2 (en) 2020-02-19 2021-12-28 Globus Medical, Inc. Displaying a virtual model of a planned instrument attachment to ensure correct selection of physical instrument attachment
JP7501044B2 (ja) * 2020-03-30 2024-06-18 セイコーエプソン株式会社 表示システム、情報処理装置及び表示システムの表示制御方法
US11607277B2 (en) 2020-04-29 2023-03-21 Globus Medical, Inc. Registration of surgical tool with reference array tracked by cameras of an extended reality headset for assisted navigation during surgery
US11153555B1 (en) 2020-05-08 2021-10-19 Globus Medical Inc. Extended reality headset camera system for computer assisted navigation in surgery
US11922581B2 (en) * 2020-05-08 2024-03-05 Coviden Lp Systems and methods of controlling an operating room display using an augmented reality headset
US11510750B2 (en) 2020-05-08 2022-11-29 Globus Medical, Inc. Leveraging two-dimensional digital imaging and communication in medicine imagery in three-dimensional extended reality applications
US11382700B2 (en) 2020-05-08 2022-07-12 Globus Medical Inc. Extended reality headset tool tracking and control
US11737831B2 (en) 2020-09-02 2023-08-29 Globus Medical Inc. Surgical object tracking template generation for computer assisted navigation during surgical procedure
EP4030219B1 (en) * 2021-01-13 2025-07-30 BHS Technologies GmbH Medical imaging system and method of controlling such imaging system
EP4283373A1 (en) 2022-05-27 2023-11-29 Leica Instruments (Singapore) Pte Ltd Medical imaging control apparatus, medical imaging system and method of operating a medical imaging system
JP2024007643A (ja) * 2022-07-06 2024-01-19 セイコーエプソン株式会社 表示システム、制御装置及び表示システムの表示方法
JP2024056484A (ja) * 2022-10-11 2024-04-23 株式会社メディカロイド 外科手術システム
DE102023115877A1 (de) * 2023-06-16 2024-12-19 Leica Instruments (Singapore) Pte. Ltd. Am Kopf getragene Anzeigevorrichtung, wissenschaftliches oder chirurgisches Bildgebungssystem und Verfahren

Family Cites Families (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3362898B2 (ja) * 1993-03-03 2003-01-07 オリンパス光学工業株式会社 人工現実感システム
JP3680373B2 (ja) * 1995-09-28 2005-08-10 ソニー株式会社 光学視覚装置及び光学視覚装置の制御方法
US6774869B2 (en) * 2000-12-22 2004-08-10 Board Of Trustees Operating Michigan State University Teleportal face-to-face system
JP3766598B2 (ja) * 2001-02-13 2006-04-12 オリンパス株式会社 観察システム
KR100538328B1 (ko) * 2003-06-20 2005-12-22 엘지.필립스 엘시디 주식회사 액정표시장치 및 그 제조방법
JP2005107758A (ja) * 2003-09-30 2005-04-21 Hitachi Zosen Corp 保全システムおよび情報共有システム
JP3683575B2 (ja) * 2003-10-28 2005-08-17 オリンパス株式会社 頭部装着型ディスプレイコントローラー
JP2006309534A (ja) * 2005-04-28 2006-11-09 Konica Minolta Photo Imaging Inc 内部情報記録媒体視認システム
JP2007320715A (ja) * 2006-05-31 2007-12-13 Seikatsu Kyodo Kumiai Coop Sapporo 作業関連情報提供システム、及び作業関連情報提供方法
JP2008124885A (ja) * 2006-11-14 2008-05-29 Sony Corp 撮像システム、撮像方法
JP2008198028A (ja) * 2007-02-14 2008-08-28 Sony Corp ウェアラブル装置、認証方法、およびプログラム
JP2009279193A (ja) * 2008-05-22 2009-12-03 Fujifilm Corp 医療機器管理システム
JP2010141446A (ja) * 2008-12-10 2010-06-24 Brother Ind Ltd ヘッドマウントディスプレイ及びヘッドマウントディスプレイにおける画像提示方法
JP5670079B2 (ja) * 2009-09-30 2015-02-18 富士フイルム株式会社 医用画像表示装置および方法、並びにプログラム
EP3263058A1 (en) * 2010-06-28 2018-01-03 Brainlab AG Generating images for at least two displays in image-guided surgery
JP2012170747A (ja) * 2011-02-23 2012-09-10 Toshiba Corp 超音波診断装置、及び超音波診断プログラム
JP6144681B2 (ja) * 2011-08-30 2017-06-07 マイクロソフト テクノロジー ライセンシング，エルエルシー 虹彩スキャン・プロファイリング機能を有する頭部装着ディスプレイ
JP6028357B2 (ja) * 2012-03-22 2016-11-16 ソニー株式会社 ヘッドマウントディスプレイ及び手術システム
JP6004699B2 (ja) * 2012-03-29 2016-10-12 キヤノン株式会社 印刷装置、画像処理装置、印刷装置の制御方法、画像処理装置の制御方法及びプログラム
US20130317848A1 (en) * 2012-05-22 2013-11-28 Andrew Savin Electronic Medical Record Process
JP2014092940A (ja) * 2012-11-02 2014-05-19 Sony Corp 画像表示装置及び画像表示方法、並びにコンピューター・プログラム
JP5784245B2 (ja) * 2012-11-30 2015-09-24 日立マクセル株式会社 映像表示装置、及びその設定変更方法、設定変更プログラム
CN104583982B (zh) * 2012-12-11 2017-03-22 威尔森信息通信株式会社 医疗支援系统及其方法
CN103190883B (zh) * 2012-12-20 2015-06-24 苏州触达信息技术有限公司 一种头戴式显示装置和图像调节方法
JP2016032485A (ja) * 2012-12-27 2016-03-10 国立大学法人 東京医科歯科大学 内視鏡下手術支援システム及び画像制御方法
EP3022672A2 (en) * 2013-07-16 2016-05-25 Seiko Epson Corporation Information processing apparatus, information processing method, and information processing system
US10194860B2 (en) * 2013-09-11 2019-02-05 Industrial Technology Research Institute Virtual image display system
EP3146715B1 (en) * 2014-05-20 2022-03-23 University Of Washington Through Its Center For Commercialization Systems and methods for mediated-reality surgical visualization

Also Published As

Publication number Publication date
EP3178232A1 (en) 2017-06-14
JP2016061827A (ja) 2016-04-25
CN106687065A (zh) 2017-05-17
US20170151034A1 (en) 2017-06-01
WO2016042705A1 (en) 2016-03-24

Similar Documents

Publication Publication Date Title
JP6574939B2 (ja) 表示制御装置、表示制御方法、表示制御システムおよび頭部装着ディスプレイ
JP6264087B2 (ja) 表示制御装置、表示装置および表示制御システム
JP6693507B2 (ja) 情報処理装置、情報処理方法及び情報処理システム
EP3204816B1 (en) Head-mounted displaying of magnified images locked on an object of interest
EP3388921A1 (en) Control device of head mounted display; operation method and operation program thereof; and image display system
US11729493B2 (en) Image capture apparatus and image capture method
US11094283B2 (en) Head-wearable presentation apparatus, method for operating the same, and medical-optical observation system
JP2016032485A (ja) 内視鏡下手術支援システム及び画像制御方法
JP2016189120A (ja) 情報処理装置、情報処理システム、及び頭部装着ディスプレイ
JP2023095970A (ja) 医療用観察システム
JP6589855B2 (ja) 頭部装着ディスプレイ、制御装置および制御方法
US11224329B2 (en) Medical observation apparatus
JP6617766B2 (ja) 医療用観察システム、表示制御システムおよび表示制御装置
JP2019154886A (ja) 医療用表示制御装置、および表示制御方法
JP7017385B2 (ja) 頭部装着型表示装置、表示システム及び表示方法
JP2019154734A (ja) 医療用画像処理装置、医療用観察装置、および画像処理方法

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20170124

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20170124

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20171025

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20171107

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20171226

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20180529

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20180725

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20190108

RD04 Notification of resignation of power of attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7424

Effective date: 20190208

RD03 Notification of appointment of power of attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7423

Effective date: 20190214

RD04 Notification of resignation of power of attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7424

Effective date: 20190222

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20190304

TRDD Decision of grant or rejection written
RD02 Notification of acceptance of power of attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7422

Effective date: 20190515

A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20190521

RD04 Notification of resignation of power of attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7424

Effective date: 20190522

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20190603

R151 Written notification of patent or utility model registration

Ref document number: 6574939

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R151

LAPS Cancellation because of no payment of annual fees