JPH08227341A - User interface - Google Patents
User interfaceInfo
- Publication number
- JPH08227341A JPH08227341A JP3323295A JP3323295A JPH08227341A JP H08227341 A JPH08227341 A JP H08227341A JP 3323295 A JP3323295 A JP 3323295A JP 3323295 A JP3323295 A JP 3323295A JP H08227341 A JPH08227341 A JP H08227341A
- Authority
- JP
- Japan
- Prior art keywords
- image
- key
- unit
- display
- display unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Abstract
Description
【０００１】[0001]
【産業上の利用分野】本発明は、空調機器、照明機器、
家電機器などにおける各種機器の操作器のユーザインタ
ーフェースに関するものである。BACKGROUND OF THE INVENTION The present invention relates to an air conditioner, a lighting device,
The present invention relates to a user interface of an operation device of various devices such as home electric appliances.
【０００２】[0002]
【従来の技術】昨今の空調機器、照明機器、家電機器な
どにおける操作器のユーザインターフェースは、タッチ
スクリーンによるダイレクトタッチ方式のものが主流と
なりつつある。タッチスクリーンを使用した入力方式の
ユーザインターフェースは、画面に現れる操作ボタンや
操作キー等の画像に直接触れることによって操作が行え
るため、快適な操作が行える。2. Description of the Related Art Recently, as a user interface of an operation device in an air conditioner, a lighting device, a home electric appliance, etc., a direct touch type using a touch screen is becoming mainstream. An input-type user interface using a touch screen can be operated by directly touching an image of an operation button or an operation key appearing on the screen, so that a comfortable operation can be performed.
【０００３】このような操作器のユーザインターフェー
スにおける高齢者や目の不自由な方のための施策として
は、タッチスクリーンに表示される表示文字を大きくし
たり、操作ボタンや操作キー等を大きいものにすること
が考えられる。また、ユーザの操作の補助機能に関して
は、主に個々の操作の方法に対応する説明、あるいは操
作手順に対応する支援などのヘルプ機能が考えられる。As measures for the elderly and visually impaired persons in the user interface of such an operating device, the display characters displayed on the touch screen are enlarged, and the operation buttons and keys are large. It is possible to Further, regarding the auxiliary function of the operation of the user, a help function such as a description mainly corresponding to each operation method or a support corresponding to the operation procedure can be considered.
【０００４】そのため、例えば特開平４−２４８１９号
公報に示すように、操作器の操作キーを使用頻度に基づ
いて拡大表示させるようにしたタッチスクリーン式デー
タ入力装置が開示されている。Therefore, as disclosed in, for example, Japanese Patent Laid-Open No. 4-24819, a touch screen type data input device is disclosed in which the operation keys of the operation device are enlarged and displayed based on the frequency of use.
【０００５】特開平４−２４８１９号公報に開示された
タッチスクリーン式データ入力装置は、複数のキー画像
を表示する表示ユニット表面の指示されたポイントの座
標値を検出し、ポイントの座標値と各キー画像の表示領
域データとから指示されたキー画像を識別してキーデー
タを入力するタッチスクリーン式データ入力装置におい
て、各キーの使用頻度を演算する使用頻度演算部と、各
キーの使用頻度に基づいて各キー各キー画像の大きさを
演算する演算部と、演算された大きさのキー画像を発生
して表示ユニットに表示するキー画像発生部と、各キー
画像の表示領域を演算するキー画像領域演算部とを有す
るものである。The touch screen type data input device disclosed in Japanese Unexamined Patent Publication No. 4-24819 detects the coordinate value of a designated point on the surface of a display unit for displaying a plurality of key images, and detects the coordinate value of the point and each coordinate value. In a touch screen type data input device that identifies a key image indicated by the display area data of the key image and inputs the key data, a frequency calculation unit that calculates the frequency of use of each key and a frequency of use of each key Based on each key, a calculation unit that calculates the size of each key image, a key image generation unit that generates a key image of the calculated size and displays it on the display unit, and a key that calculates the display area of each key image And an image area calculation unit.
【０００６】そして、タッチスクリーン式データ入力装
置における各キーの使用頻度を演算し、各キーの使用頻
度に基づいて各キー画像の大きさを求め、大きさでキー
画像を発生して表示ユニットに表示するとともに、各キ
ー画像の表示領域を演算し、以後表示ユニットに表示さ
れている所定のキー画像上のポイントが指示された時、
指示されたポイントの座標値を検出し、ポイントの座標
値と各キー画像の表示領域データとから指示されたキー
画像を識別してキーデータを出力する。Then, the frequency of use of each key in the touch screen data input device is calculated, the size of each key image is obtained based on the frequency of use of each key, and the key image is generated in the size to be displayed on the display unit. While displaying, the display area of each key image is calculated, and when a point on a predetermined key image displayed on the display unit is instructed thereafter,
The coordinate value of the designated point is detected, the designated key image is identified from the coordinate value of the point and the display area data of each key image, and the key data is output.
【０００７】[0007]
【発明が解決しようとする課題】上記のような従来のユ
ーザインターフェースは、グラフィック化されたもので
あるので、画面上の操作キー等を視覚的に捉えて操作す
ることが前提となっている。また、平滑なタッチスクリ
ーン上で手探りでボタンの位置を突き止めるのは極めて
困難である。これらのことが高齢者や目の不自由な方に
とっては、かえって操作を難しくしていた。また、特開
平４−２４８１９号公報に開示されたタッチスクリーン
式データ入力装置においては、キー画像を使用頻度に基
づいて拡大させているが、上記のような問題を解決する
には至らなかった。Since the conventional user interface as described above is a graphic one, it is premised that the operation keys and the like on the screen are visually grasped and operated. In addition, it is extremely difficult to locate the position of the button by groping on a smooth touch screen. These things made the operation rather difficult for the elderly and the visually impaired. Further, in the touch screen type data input device disclosed in Japanese Unexamined Patent Publication No. 4-24819, the key image is enlarged based on the frequency of use, but the above problems cannot be solved.
【０００８】さらに、従来のユーザインターフェースは
タッチスクリーンを使用した入力方式のものなので、タ
ッチスクリーンの画面上に描かれた操作ボタンや操作キ
ーの画像に直接触れるという操作が行われるが、この操
作は現実世界の操作ボタンや操作キーの操作とはかけ離
れているため、「ボタンを押した」あるいは「キーに触
れた」といった操作感覚が得難く、わかり難いものであ
った。また、ユーザの操作補助となるヘルプ機能におい
ても、ユーザが自分の意図することを行うために必要な
情報を得ることが難しかったり、逆に必要以上に過剰な
情報がユーザに与えられてしまってかえって操作の妨げ
になることがあった。Further, since the conventional user interface is an input system using a touch screen, the operation of directly touching the image of the operation button or the operation key drawn on the screen of the touch screen is performed. Since it is far from the operation of the operation buttons and operation keys in the real world, it is difficult to understand the operation feeling of “pressing the button” or “touching the key”, which is difficult to understand. In addition, even in the help function that assists the user, it is difficult for the user to obtain the information necessary to do what he or she intends, or conversely, excessive information is given to the user. On the contrary, it sometimes hindered the operation.
【０００９】本発明は、上記のような課題を解決するた
めになされたもので、高齢者や目の不自由な方などにも
簡単に操作を行うことのできる操作性が向上されたユー
ザインターフェースを提供することを目的としたもので
ある。The present invention has been made in order to solve the above problems, and a user interface with improved operability that allows even the elderly and the visually impaired to easily operate. The purpose is to provide.
【００１０】[0010]
【課題を解決するための手段】本発明に係るユーザイン
ターフェースは、複数の操作ボタンおよび操作キーを表
示する表示ユニットと、表示ユニットの表面に配置さ
れ、指示された位置の座標を検出するタッチスクリーン
と、タッチスクリーンの出力信号に基づいて操作ボタン
および操作キーに対応した制御を行う制御部とを備え、
制御部は、タッチスクリーンの出力信号に基づいて表示
ユニットに表示された操作ボタンおよび操作キーのう
ち、指示されたものの画像を識別する画像識別手段と、
画像識別手段によって識別された画像を他の表示画像と
識別可能にするための特殊表示をさせるかどうかを判断
する特殊効果識別手段と、特殊効果識別手段の判断結果
に基づいて表示ユニットの指示された操作ボタンまたは
操作キーに対して特殊表示をさせる第１の制御手段とを
有するものである。A user interface according to the present invention comprises a display unit for displaying a plurality of operation buttons and operation keys, and a touch screen arranged on the surface of the display unit for detecting the coordinates of a designated position. And a control unit that performs control corresponding to the operation buttons and operation keys based on the output signal of the touch screen,
The control unit is an image identification unit that identifies an image of the instructed one of the operation buttons and the operation keys displayed on the display unit based on the output signal of the touch screen,
Special effect identification means for determining whether to perform a special display for distinguishing the image identified by the image identification means from other display images, and an instruction of the display unit based on the determination result of the special effect identification means. And a first control means for displaying a special display on the operation button or the operation key.
【００１１】また、特殊表示は指示された操作ボタンま
たは操作キーの周囲にダイナミックな動作が付加された
アニメーション画像であるものである。The special display is an animation image in which a dynamic operation is added around the designated operation button or operation key.
【００１２】さらに、第１の制御手段は、特殊表示のア
ニメーション画像である波紋画像のデータが格納されて
いる波紋画像データ記憶部と、波紋画像データ記憶部の
波紋画像データを順次取り出し波紋画像データの所在、
数および表示順序を決定する波紋画像パターン決定部
と、波紋画像パターン決定部からの決定情報、波紋画像
座標演算部からの表示座標情報および波紋画像大きさ演
算部からの表示大きさ情報に基づいて波紋画像を発生さ
せる波紋画像発生部とを有するものである。Further, the first control means sequentially takes out the ripple image data storage unit in which the data of the ripple image which is the animation image of the special display is stored, and the ripple image data in the ripple image data storage unit is sequentially taken out. Whereabouts of
Based on the ripple image pattern determination unit that determines the number and display order, the determination information from the ripple image pattern determination unit, the display coordinate information from the ripple image coordinate calculation unit, and the display size information from the ripple image size calculation unit. And a ripple image generation unit for generating a ripple image.
【００１３】本発明に係るユーザインターフェースの制
御部は、タッチスクリーンの出力信号に基づいて表示ユ
ニットに表示された操作ボタンおよび操作キーのうち、
指示されたものの画像を識別する画像識別手段と、画像
識別手段によって識別された画像に対して他の表示画像
と識別可能にするための効果音を発生させるかどうかを
判断する特殊効果識別手段と、特殊効果識別手段の判断
結果に基づいて表示ユニットの指示された操作ボタンま
たは操作キーに対応した効果音をスピーカから発生させ
る第２の制御手段とを有するものである。The control unit of the user interface according to the present invention, among the operation buttons and the operation keys displayed on the display unit based on the output signal of the touch screen,
Image identifying means for identifying the image of the instructed object, and special effect identifying means for determining whether to generate a sound effect for distinguishing the image identified by the image identifying means from other display images. Second control means for causing the speaker to generate a sound effect corresponding to the operation button or operation key instructed on the display unit based on the determination result of the special effect identification means.
【００１４】また、第２の制御手段は、タッチスクリー
ンの押圧力を検出する圧力検出手段を有し、押圧力が所
定圧力以上の時には指定された操作ボタンまたは操作キ
ーに対応した効果音を発生させるとともにその処理を行
い、押圧力が所定圧力より小さい時には効果音のみを発
生させるものである。The second control means has a pressure detecting means for detecting the pressing force of the touch screen, and when the pressing force is equal to or higher than a predetermined pressure, a sound effect corresponding to the designated operation button or operation key is generated. When the pressing force is smaller than the predetermined pressure, only the sound effect is generated.
【００１５】さらに、第２の制御手段は、操作ボタンお
よび操作キーに対応した効果音データが格納されている
効果音データ記憶部と、指定された操作ボタンまたは操
作キーに対応した効果音データを効果音データ記憶部か
ら取り出しその効果音データの所在を決定する効果音パ
ターン決定部と、効果音パターン決定部からの決定情報
に基づいて効果音をスピーカより発生させる効果音発生
部とを有するものである。Further, the second control means stores a sound effect data storage section in which sound effect data corresponding to the operation button and the operation key is stored, and sound effect data corresponding to the specified operation button or operation key. A sound effect data storage unit having a sound effect pattern determination unit for determining the location of the sound effect data and a sound effect generation unit for generating a sound effect from a speaker based on the determination information from the sound effect pattern determination unit. Is.
【００１６】本発明に係るユーザインターフェースは、
複数の操作ボタンおよび拡大表示キーを含む操作キーを
表示する表示ユニットと、表示ユニットの表面に配置さ
れ、指示された位置の座標を検出するタッチスクリーン
と、タッチスクリーンの出力信号に基づいて操作ボタン
および操作キーに対応した制御を行う制御部とを備え、
制御部は、タッチスクリーンの出力信号に基づいて表示
ユニットに表示された操作ボタンおよび操作キーのう
ち、指示されたものの画像が拡大表示キーの画像である
かどうかを識別する画像識別手段と、画像識別手段にお
いて拡大表示キーが識別された時には必要最小限の操作
ボタンと操作キーとの組合わせからなるシュリンク画面
を表示ユニットに表示させる第３の制御手段とを有する
ものである。The user interface according to the present invention is
A display unit for displaying operation keys including a plurality of operation buttons and an enlarged display key, a touch screen arranged on the surface of the display unit for detecting coordinates of a designated position, and an operation button based on an output signal of the touch screen. And a control unit that performs control corresponding to the operation keys,
The control unit includes an image identification unit that identifies whether or not the image of the operation button and the operation key displayed on the display unit on the basis of the output signal of the touch screen is the image of the enlarged display key, and the image identification unit. When the enlarged display key is identified by the identification means, the third control means is provided to display a shrink screen consisting of a combination of the minimum necessary operation buttons and operation keys on the display unit.
【００１７】また、第３の制御手段は、シュリンク画面
のデータが複数格納されているシュリンク画面データ記
憶部と、シュリンク画面データ記憶部に格納されている
シュリンク画面データをもとに操作ボタンまたは操作キ
ーの画像の大きさあるいは画像の数量を決定する画像大
きさ決定部および画像数量決定部と、画像大きさ決定部
および画像数量決定部の決定情報に基づいて画像の大き
さ演算および座標演算をそれぞれ行う演算部と、これら
の演算結果に基づいてシュリンク画面を発生させる画像
発生部とを有するものである。The third control means includes a shrink screen data storage section in which a plurality of pieces of shrink screen data are stored, and an operation button or an operation based on the shrink screen data stored in the shrink screen data storage section. An image size determination unit and an image quantity determination unit that determine the image size or the number of images of the key, and image size calculation and coordinate calculation based on the determination information of the image size determination unit and the image number determination unit. It has a calculation unit that performs each and an image generation unit that generates a shrink screen based on these calculation results.
【００１８】本発明に係るユーザインターフェースの制
御部は、タッチスクリーンの出力信号に基づいて表示ユ
ニットに表示された操作ボタンおよび操作キーのうち、
指示されたものの画像が拡大表示キーの画像であるかど
うかを識別する画像識別手段と、画像識別手段において
拡大表示キーが識別され次に画面の任意の箇所が押され
て指定されると、指示された箇所の座標を中心に予め定
められた半径内の画像を予め定められた倍率で表示ユニ
ットに拡大表示させる第４の制御手段を有するものであ
る。The control unit of the user interface according to the present invention includes the operation buttons and the operation keys displayed on the display unit based on the output signal of the touch screen.
An image identifying means for identifying whether or not the image of the instructed object is an image of the enlargement display key, and the enlargement display key is identified by the image identification means, and next, an arbitrary portion of the screen is pressed and designated. It has a fourth control means for enlarging and displaying on the display unit an image within a predetermined radius centered on the coordinates of the selected location at a predetermined magnification.
【００１９】また、第４の制御手段は、指示された箇所
の座標が拡大される画面の中心基準となることを決定す
る画面拡大位置決定部と、画面拡大位置決定部によって
決定された基準座標を中心に予め定められた半径内の画
像が予め定められた倍率になるように演算する画像座標
演算部および画像大きさ演算部と、演算結果に基づいて
拡大された拡大画面を発生させる画像発生部とを有する
ものである。Further, the fourth control means is a screen enlargement position determining section for determining that the coordinates of the instructed portion will be the center reference of the enlarged screen, and the reference coordinates determined by the screen enlargement position determining section. An image coordinate calculation unit and an image size calculation unit that perform calculation so that an image within a predetermined radius centering on the image has a predetermined magnification, and image generation that generates an enlarged screen enlarged based on the calculation result. And a part.
【００２０】本発明に係るユーザインターフェースは、
複数の操作ボタンおよび複数のヘルプキー含む操作キー
を表示する表示ユニットと、表示ユニットの表面に配置
され、指示された位置の座標を検出するタッチスクリー
ンと、タッチスクリーンの出力信号に基づいて操作ボタ
ンおよび操作キーに対応した制御を行う制御部とを備
え、制御部は、タッチスクリーンの出力信号に基づいて
表示ユニットに表示された操作ボタンおよび操作キーの
うち、指示されたものの画像がヘルプキーの画像である
かどうかを識別する画像識別手段と、画像識別手段にお
いていずれかのヘルプキーが識別された時にはそのヘル
プキーに対応したヘルプ処理を行わせる第５の制御手段
を有するものである。The user interface according to the present invention is
A display unit for displaying a plurality of operation buttons and operation keys including a plurality of help keys, a touch screen arranged on the surface of the display unit for detecting coordinates of a designated position, and an operation button based on an output signal of the touch screen. And a control unit that performs control corresponding to the operation key, and the control unit is configured to display an image of the operation button and the operation key displayed on the display unit based on the output signal of the touch screen as an image of the help key. It has an image identifying means for identifying whether it is an image and a fifth control means for performing a help process corresponding to the help key when any help key is identified by the image identifying means.
【００２１】また、第５の制御手段は、操作される操作
ボタンまたは操作キーの履歴が格納される操作履歴記憶
部と、操作ボタンおよび操作キーの操作方法データが格
納された操作方法データ記憶部と、表示ユニットに表示
される各画面の操作手順データが格納されている操作手
順データ記憶部と、画像識別手段において操作方法ヘル
プキーが識別された時に指定された操作ボタンまたは操
作キーの操作方法データを表示ユニットに表示させる操
作方法発生部と、画像識別手段において操作手順ヘルプ
キーが識別された時に表示ユニットに表示されている画
面の操作手順データを表示ユニットに表示させる操作手
順発生部と、表示ユニットに表示されている画面の操作
終了後に画像識別手段において操作サマリヘルプキーが
識別された時に操作履歴記憶部に格納された操作キーの
履歴に基づいて操作サマリを表示ユニットに表示させる
操作サマリ提示部とを有するものである。The fifth control means includes an operation history storage section for storing a history of operation buttons or operation keys to be operated, and an operation method data storage section for storing operation method data of the operation buttons and the operation keys. And an operation procedure data storage unit that stores operation procedure data of each screen displayed on the display unit, and an operation button or an operation key specified when the operation method help key is identified by the image identification means. An operation method generating section for displaying data on the display unit, an operation procedure generating section for displaying on the display unit the operation procedure data of the screen displayed on the display unit when the operation procedure help key is identified by the image identification means, After the operation of the screen displayed on the display unit is completed, the operation is performed when the operation summary help key is identified by the image identification means. Those having a based on the history of the operation key stored in the history storage unit displaying an operation summary display unit operation summary presentation unit.
【００２２】本発明に係るユーザインターフェースは、
タッチスクリーンの周囲およびその表面に突起を複数設
けたものである。The user interface according to the present invention is
A plurality of protrusions are provided around the touch screen and on the surface thereof.
【００２３】[0023]
【作用】本発明に係るユーザインターフェースは、表示
ユニットに表示された操作ボタンおよび操作キーのう
ち、指示されたものの画像を画像識別手段によって識別
し、特殊効果識別手段によってその画像において特殊表
示をさせるかどうかを判断し、特殊表示をさせる場合は
第１の制御手段によって指示された操作ボタンまたは操
作キーに対して特殊表示を行い、他の表示画像と識別可
能にする。In the user interface according to the present invention, the image of the instructed one of the operation buttons and the operation keys displayed on the display unit is identified by the image identifying means, and the special effect identifying means makes the special display on the image. If it is determined whether or not to display the special display, the special display is performed on the operation button or the operation key instructed by the first control unit so that it can be distinguished from other display images.
【００２４】また、指示された操作ボタンまたは操作キ
ーの周囲に特殊表示としてダイナミックな動作が付加さ
れたアニメーション画像を表示し、容易に他の表示画像
と識別可能にする。Further, an animation image to which a dynamic action is added as a special display is displayed around the designated operation button or operation key so that it can be easily distinguished from other display images.
【００２５】さらに、第１の制御手段によって指示され
た操作ボタンまたは操作キーの周囲に波紋を表示させ、
画面操作の実感の不足を補う。Further, a ripple is displayed around the operation button or operation key designated by the first control means,
Make up for the lack of experience in screen operations.
【００２６】本発明に係るユーザインターフェースは、
画像識別手段によって識別された画像において効果音を
発生させるかどうかを特殊効果識別手段によって判断
し、効果音を発生させる場合は第２の制御手段によって
指示された操作ボタンまたは操作キーに対応した効果音
をスピーカから発生させて、他の表示画像と識別可能に
する。The user interface according to the present invention is
Whether or not to generate a sound effect in the image identified by the image identifying means is determined by the special effect identifying means, and when the sound effect is produced, the effect corresponding to the operation button or the operation key instructed by the second control means. Sound is generated from the speaker so that it can be distinguished from other displayed images.
【００２７】また、圧力検出手段によってタッチスクリ
ーンの押圧力を検出し、押圧力が所定圧力以上の時には
指定された操作ボタンまたは操作キーに対応した効果音
を発生させるとともにその処理を行い、視覚だけでなく
聴覚からも画面操作の実感の不足を補う。一方、押圧力
が所定圧力より小さい時には効果音のみを発生させ、そ
の効果音を頼りに手探りで画面上のボタンの位置が分か
るようにし、目の不自由な方にも目的とする操作を行う
ためのボタンの位置が容易に分かるようにする。ここ
で、押圧力が所定圧力より小さい時には発生する効果音
と、所定圧力以上の時に発生する効果音とでは、必ずし
も一致する必要はなく、例えば所定圧力より小さい時に
発生する効果音を音声ガイダンスとすることにより、一
層効果的に操作ガイドができる。Further, the pressing force of the touch screen is detected by the pressure detecting means, and when the pressing force is equal to or higher than a predetermined pressure, a sound effect corresponding to the designated operation button or operation key is generated and its processing is performed, and only the visual sense is performed. In addition to hearing, make up for the lack of sense of screen operation. On the other hand, when the pressing force is less than the predetermined pressure, only a sound effect is generated, and by relying on the sound effect, the position of the button on the screen can be found by groping, and the intended operation is performed even for the visually impaired. Make it easy to know the position of the button for. Here, the sound effect generated when the pressing force is lower than the predetermined pressure and the sound effect generated when the pressure is equal to or higher than the predetermined pressure do not necessarily have to match. For example, the sound effect generated when the pressure is lower than the predetermined pressure is used as the voice guidance. By doing so, the operation guide can be performed more effectively.
【００２８】本発明に係るユーザインターフェースは、
画像識別手段によって識別された画像が拡大表示キーの
画像である時、第３の制御手段によって操作画面に対応
したシュリンク画面を表示させ、必要最小限の操作ボタ
ンと操作キーで画面操作ができるようにする。The user interface according to the present invention is
When the image identified by the image identifying means is the image of the enlarged display key, the third control means causes the shrink screen corresponding to the operation screen to be displayed so that the screen operation can be performed with the minimum necessary operation buttons and operation keys. To
【００２９】また、画像識別手段によって識別された画
像が拡大表示キーの画像である時、操作画面に表示され
た文字および操作キー等が見易くなり、高齢者等におい
てもキー操作を簡単かつ確実に行えるように、第４の制
御手段によって次に画面の任意に指示された箇所の座標
を中心に予め定められた半径内の画像を予め定められた
倍率で表示ユニットに拡大表示させる。Further, when the image identified by the image identifying means is the image of the enlarged display key, the characters and the operation keys displayed on the operation screen become easy to see, and even the elderly and the like can easily and surely perform the key operation. As is possible, the fourth control means causes the display unit to magnify and display the image within the predetermined radius centered on the coordinates of the arbitrarily designated point on the screen at a predetermined magnification.
【００３０】本発明に係るユーザインターフェースは、
画像識別手段によって識別された画像がヘルプキーの画
像である時、画面操作を円滑に行えるように第５の制御
手段によってそのヘルプキーに対応したヘルプ処理を行
う。The user interface according to the present invention is
When the image identified by the image identifying means is the image of the help key, the fifth control means performs the help process corresponding to the help key so that the screen operation can be smoothly performed.
【００３１】また、第５の制御部はヘルプキーのうち、
第１のヘルプキーが指示されると次に指定された操作ボ
タンまたは操作キーの操作方法データを表示ユニットに
表示させ、第２のヘルプキーが指示されると表示ユニッ
トに表示されている画面の操作手順データを表示ユニッ
トに表示させ、第３のヘルプキーが指示されると操作サ
マリを表示ユニットに表示させ、ユーザが意図する操作
を達成できるようにする。The fifth control unit is one of the help keys.
When the first help key is instructed, the operation method data of the next designated operation button or operation key is displayed on the display unit, and when the second help key is instructed, the screen displayed on the display unit is displayed. The operation procedure data is displayed on the display unit, and when the third help key is instructed, the operation summary is displayed on the display unit so that the operation intended by the user can be achieved.
【００３２】本発明に係るユーザインターフェースは、
タッチスクリーンの周囲およびその表面に突起を複数設
けることにより、目の不自由な方にもタッチスクリーン
のいずれの辺りを触っているかを容易に知ることができ
る。The user interface according to the present invention is
By providing a plurality of protrusions around the touch screen and on the surface thereof, it is possible for the visually impaired person to easily know which part of the touch screen is touching.
【００３３】[0033]
実施例１．図１は本発明の第１〜第４の実施例の構成を
示すブロック図、図２は本発明の第１の実施例に係る表
示ユニットに表示された操作画面の要部の拡大図であ
る。Example 1. FIG. 1 is a block diagram showing a configuration of first to fourth embodiments of the present invention, and FIG. 2 is an enlarged view of a main part of an operation screen displayed on a display unit according to the first embodiment of the present invention. .
【００３４】この実施例１においては、まず、図２に示
すように、表示ユニット６に表示されている操作画面の
操作ボタン４１の画像がユーザによって押圧されると、
位置座標検出部を有するタッチスクリーン５によって押
圧された操作ボタン４１の座標が検出され、画像識別部
１８およびタッチ圧力検出部２０に検出座標情報が出力
される。そして、画像識別部１８はタッチスクリーン５
からの検出座標情報に基づいてどの操作ボタンまたは操
作キーの画像が操作されたのかを識別する。なお、ここ
では図２（ａ）に示す操作ボタン４１の画像が識別され
る。ついで、特殊効果識別部１９は、画像識別部１８に
よって識別された画像に基づいて識別された画像を他の
表示画像と識別可能にするための特殊表示、例えば識別
された操作ボタンまたは操作キーの周囲にダイナミック
な動作が付加されたアニメーション画像である波紋画像
を表示させるかどうかを判断し、特殊表示をさせる場合
は、波紋画像パターン決定部１２に信号を送る。In the first embodiment, first, as shown in FIG. 2, when the image of the operation button 41 of the operation screen displayed on the display unit 6 is pressed by the user,
The coordinates of the operation button 41 pressed by the touch screen 5 having the position coordinate detection unit are detected, and the detected coordinate information is output to the image identification unit 18 and the touch pressure detection unit 20. Then, the image identification unit 18 uses the touch screen 5
It identifies which operation button or operation key image was operated based on the detected coordinate information. Here, the image of the operation button 41 shown in FIG. 2A is identified. Next, the special effect identification unit 19 displays a special display for distinguishing the image identified based on the image identified by the image identification unit 18 from other display images, for example, identified operation buttons or operation keys. It is determined whether or not to display a ripple image, which is an animation image to which a dynamic motion is added to the surroundings, and when special display is performed, a signal is sent to the ripple image pattern determination unit 12.
【００３５】特殊効果識別部１９から波紋画像パターン
決定部１２に信号が送られると、波紋画像パターン決定
部１２は波紋画像発生部８を介して波紋画像データ記憶
部９から波紋画像データを順次取り出すとともに、波紋
画像データの所在、数、および表示順序を指示情報を決
定し、この指示情報と、波紋画像座標演算部１０によっ
て演算された表示座標情報と、波紋画像大きさ演算部１
１によって演算された表示大きさ情報とに基づいて波紋
画像発生部８は波紋画像を発生させ、画像メモリ７に書
き込む。そして、表示ユニット６ではユーザによって押
圧された操作ボタン４１に対し、図２（ｂ）に示すよう
に、その操作ボタン４１の周囲に波紋４２が広がるアニ
メーション画像が波紋画像パターン決定部１２が指定し
た回数だけ繰り返し表示される。When a signal is sent from the special effect identification unit 19 to the ripple image pattern determination unit 12, the ripple image pattern determination unit 12 sequentially retrieves the ripple image data from the ripple image data storage unit 9 via the ripple image generation unit 8. At the same time, the instruction information about the location, the number, and the display order of the ripple image data is determined, and the instruction information, the display coordinate information calculated by the ripple image coordinate calculation unit 10, and the ripple image size calculation unit 1
The ripple image generation unit 8 generates a ripple image based on the display size information calculated by 1 and writes it in the image memory 7. Then, in the display unit 6, for the operation button 41 pressed by the user, as shown in FIG. 2B, the ripple image pattern determination unit 12 specifies an animation image in which a ripple 42 spreads around the operation button 41. It is displayed repeatedly for the number of times.
【００３６】このように、平坦でかつ押圧しても変化の
ない表示ユニット６（タッチスクリーン）上で操作ボタ
ンまたは操作キーの操作を行うと、押圧された操作ボタ
ンまたは操作キーの周囲に波紋４２が広がるアニメーシ
ョン画像などの特殊表示が付加されるので、画面操作の
実感の不足を補うことができる。また、タッチスクリー
ン上では操作中において操作ボタン等が操作している指
によって隠されて見えなくなってしまうことがあるが、
波紋画像が表示されることによって押圧されたことを容
易に確認することができ、操作性の向上を計ることがで
きる。As described above, when the operation button or the operation key is operated on the display unit 6 (touch screen) which is flat and does not change even when pressed, the ripples 42 are formed around the pressed operation button or the operation key. Since a special display such as an animated image that spreads out is added, it is possible to compensate for the lack of actual feeling of screen operation. Also, on the touch screen, the operating buttons and the like may be hidden by the operating finger during operation, but may not be visible.
Since the ripple image is displayed, it can be easily confirmed that the button is pressed, and the operability can be improved.
【００３７】なお、上述の実施例では特殊表示としてア
ニメーション画像である波紋を表示させる場合を例示し
て説明したが、例えば指示された操作ボタンまたは操作
キー全体やその周囲を変色させるようにしてもよく、ま
た、点滅させるようにしてもよい。In the above embodiment, the case where the ripples, which are the animation images, are displayed as the special display has been described as an example. However, for example, the designated operation button or the entire operation key or its surroundings may be discolored. Well, it may blink.
【００３８】実施例２．図３は本発明の第２の実施例に
係る表示ユニットに表示された操作画面の作用説明図で
ある。以下、この実施例の作用を図４のフローチャート
を用いて説明する。Example 2. FIG. 3 is an operation explanatory view of an operation screen displayed on the display unit according to the second embodiment of the present invention. The operation of this embodiment will be described below with reference to the flowchart of FIG.
【００３９】まず、表示ユニット６に表示されている操
作画面の操作ボタンまたは操作キーの画像がユーザによ
って押圧されると、実施例１で説明した場合と同様に押
圧された操作ボタンまたは操作キーの座標がタッチスク
リーン５によって検出され、その座標に基づいてどの操
作ボタンまたは操作キーの画像が操作されたのかを画像
識別部１８によって識別される。ついで、画像識別部１
８によって識別された画像に基づいて識別された画像に
対して他の表示画像と識別可能にするための効果音、例
えば識別された操作ボタンまたは操作キーにおける操作
内容の説明音を発生させるかどうかを特殊効果識別部１
９によって判断し（ステップ４５）、効果音を発生させ
る場合は効果音パターン決定部４に信号を送る。First, when the image of the operation button or operation key on the operation screen displayed on the display unit 6 is pressed by the user, the operation button or operation key pressed in the same manner as described in the first embodiment. The coordinates are detected by the touch screen 5, and the image identification unit 18 identifies which operation button or operation key image is operated based on the coordinates. Next, the image identification section 1
Whether to generate a sound effect for distinguishing an image identified based on the image identified by 8 from other display images, for example, an explanation sound of the operation content of the identified operation button or operation key Special effect identification unit 1
9 (step 45), and when a sound effect is to be generated, a signal is sent to the sound effect pattern determination section 4.
【００４０】特殊効果識別部１９から効果音パターン決
定部４に信号が送られると、効果音パターン決定部４は
効果音発生部２を介して効果音データ記憶部３に格納さ
れている個々の操作ボタンまたは操作キーの画像に対応
した効果音データ（操作方法説明データ）を取り出すと
ともに、押圧された操作ボタンまたは操作キーの画像に
対する効果音データの所在を効果音発生部２に指示す
る。そして、図３（ａ），（ｂ）に示すように、押圧中
または押圧後において操作ボタン４３または操作キー４
４に対応した効果音Ａまたは効果音Ｂが効果音発生部２
によってスピーカ１から発生される（ステップ４６）。When a signal is sent from the special effect identifying unit 19 to the sound effect pattern determining unit 4, the sound effect pattern determining unit 4 causes the sound effect data storing unit 3 to store the individual sound effect data storing units 3 via the sound effect generating unit 2. The sound effect data (operation method explanation data) corresponding to the image of the operation button or the operation key is taken out, and the location of the sound effect data for the image of the pressed operation button or operation key is instructed to the sound effect generation unit 2. Then, as shown in FIGS. 3A and 3B, the operation button 43 or the operation key 4 is pressed during or after the pressing.
The sound effect A or the sound effect B corresponding to 4 is the sound effect generation unit 2
Is generated from the speaker 1 (step 46).
【００４１】この時、タッチ圧力検出部２０はタッチス
クリーン５から送られてくる押圧した操作ボタンまたは
操作キーの圧力を検出するとともに、その検出圧力が予
め定められたしきい値と比較させており（ステップ４
７）、しきい値より強く操作ボタン等が押圧された場合
は実際に操作ボタン等が押圧されたと判断されて、押圧
された操作ボタンまたは操作キーに対応した効果音を発
生させるとともに、その操作ボタンまたは操作キーに対
応した処理が行われる（ステップ４８）。また、しきい
値より弱く押圧された場合は操作内容の確認のために押
圧されたと判断されて、押圧された操作ボタンまたは操
作キーに対応した操作内容の効果音のみが発生され、処
理は行われない。At this time, the touch pressure detector 20 detects the pressure of the pressed operation button or operation key sent from the touch screen 5, and compares the detected pressure with a predetermined threshold value. (Step 4
7) If the operation button or the like is pressed more strongly than the threshold value, it is determined that the operation button or the like is actually pressed, and a sound effect corresponding to the pressed operation button or operation key is generated and the operation is performed. A process corresponding to the button or the operation key is performed (step 48). If the button is pressed weaker than the threshold value, it is determined that the button was pressed to confirm the operation content, and only the sound effect of the operation content corresponding to the pressed operation button or operation key is generated, and the process is performed. I don't know.
【００４２】このように、表示ユニット６に表示された
操作画面の操作ボタンまたは操作キーが予め定められた
強さよりも強く押された時は、押圧された操作ボタンま
たは操作キーに対応した効果音が発生されるとともに、
その操作ボタンまたは操作キーに対応した処理が行わ
れ、予め定められた強さよりも弱く押された時は、その
操作ボタンまたは操作キーに対応した操作内容の効果音
のみが発生されるので、ユーザはその効果音から操作画
面を見ることなく自ら行った操作の内容の確認が行える
とともに、これから行おうとする操作の操作ボタン等の
位置を簡単かつ確実に認識することができ便利である。As described above, when the operation button or the operation key of the operation screen displayed on the display unit 6 is pressed more strongly than the predetermined strength, the effect sound corresponding to the pressed operation button or the operation key is pressed. Is generated,
When the processing corresponding to the operation button or the operation key is performed, and when the operation button or the operation key is pressed weaker than a predetermined strength, only the sound effect of the operation content corresponding to the operation button or the operation key is generated. It is convenient that the user can check the contents of the operation performed by himself without looking at the operation screen from the sound effect, and can easily and surely recognize the position of the operation button or the like of the operation to be performed.
【００４３】実施例３．図５は本発明の第３の実施例に
係る表示ユニットに表示される操作画面の説明図であ
る。以下、この実施例の作用を図６のフローチャートを
用いて説明する。Example 3. FIG. 5 is an explanatory diagram of an operation screen displayed on the display unit according to the third embodiment of the present invention. The operation of this embodiment will be described below with reference to the flowchart of FIG.
【００４４】まず、図５（ａ）に示すように、表示ユニ
ット６に操作画面６１が表示されてユーザが任意の操作
ボタンまたは操作キーの画像を押圧すると、実施例１で
説明した場合と同様にタッチスクリーン５および画像識
別部１８等によってどの操作ボタンまたは操作キーの画
像が押圧されたかが識別される。この時、操作画面６１
に表示されている拡大表示キーであるルーペアイコンキ
ー５１が押圧されてそのキー５１の画像が識別されると
（ステップ７１）、そのキー５１が２回連続して押圧さ
れていない時は、画像識別部１８は現在表示ユニット６
に表示されている操作画面６１が既に拡大画面であるか
否かを判断し（ステップ７２，７３）、拡大画面である
場合は操作画面６１を拡大前の元の画面に戻す（ステッ
プ７７）。First, as shown in FIG. 5A, when the operation screen 61 is displayed on the display unit 6 and the user presses an image of any operation button or operation key, the same as in the case described in the first embodiment. Then, the touch screen 5 and the image identifying portion 18 identify which operation button or operation key image is pressed. At this time, the operation screen 61
When the magnifier icon key 51, which is the enlarged display key displayed in, is pressed to identify the image of the key 51 (step 71), when the key 51 is not pressed twice in succession, the image is displayed. The identification unit 18 is the current display unit 6
It is judged whether or not the operation screen 61 displayed on the screen is already an enlarged screen (steps 72 and 73), and if it is an enlarged screen, the operation screen 61 is returned to the original screen before enlargement (step 77).
【００４５】また、拡大画面でない場合は、表示ユニッ
ト６に表示される画面を必要最小限の操作ボタンと操作
キーとの組合わせからなるシュリンク画面である拡大画
面にするという指示信号を、画像識別部１８は画像大き
さ決定部２２と画像数量決定部２３に送るとともに、複
数のシュリンク画面が格納されているシュリンク画面デ
ータ記憶部２１から表示ユニット６に表示されていた操
作画面６１に対応した拡大画面６１ａ（図５（ｂ））の
データを取り出す。ついで、画像大きさ決定部２２およ
び画像数量決定部２３はこの拡大画面６１ａのデータを
もとに、操作ボタンおよび操作キーの画像の大きさおよ
び数量を決定する。つまり、図５（ａ）の操作画面６１
に表示されている操作キー５２，５３，５４，５５およ
び操作ボタン５６，５７，５８，５９，６０の画像のう
ち、必要最小限の画像例えば操作キー５２，５３および
操作ボタン５６，５８，６０の画像のみを選択して画像
の数を決定するとともに、その決定した画像の大きさを
通常の場合の例えば２倍の大きさになるように決定す
る。この時、選択されなかった画像の大きさは画面上に
表示されないように通常の場合の０倍の大きさとする。If the screen is not the enlarged screen, an instruction signal for changing the screen displayed on the display unit 6 to the enlarged screen, which is a shrink screen including a combination of the minimum necessary operation buttons and operation keys, is used to identify the image. The unit 18 sends the image size determining unit 22 and the image quantity determining unit 23, and enlarges the shrink screen data storage unit 21 storing a plurality of shrink screens corresponding to the operation screen 61 displayed on the display unit 6. The data on the screen 61a (FIG. 5B) is taken out. Next, the image size determination unit 22 and the image quantity determination unit 23 determine the size and quantity of the image of the operation button and the operation key based on the data of the enlarged screen 61a. That is, the operation screen 61 of FIG.
Of the images of the operation keys 52, 53, 54, 55 and the operation buttons 56, 57, 58, 59, 60 displayed on the screen, the minimum necessary image, for example, the operation keys 52, 53 and the operation buttons 56, 58, 60. Only the images are selected to determine the number of images, and the size of the determined images is determined to be, for example, twice the size of the normal case. At this time, the size of the unselected image is set to 0 times the normal size so as not to be displayed on the screen.
【００４６】そして、決定した大きさおよび数の情報が
画像座標演算部１５および画像大きさ演算部１６に送ら
れると、画像座標演算部１５および画像大きさ演算部１
６は操作ボタンおよび操作キーの座標および大きさを変
換して拡大させ、拡大画面６１ａを作成する。この拡大
画面６１ａは画像発生部１３および画像メモリ７を介し
て表示ユニット６に送られ、図５（ｂ）に示すように、
表示ユニット６に拡大画面６１ａが表示される（ステッ
プ７４）。この時、拡大された操作キー５２ａ等は操作
することができ、操作されるとその操作キー等に対応し
た処理が実行される。また、再度ルーペアイコンキー５
１が押圧されると、表示ユニット６には拡大前の元の操
作画面６１が表示される。When the information of the determined size and number is sent to the image coordinate calculation unit 15 and the image size calculation unit 16, the image coordinate calculation unit 15 and the image size calculation unit 1
Reference numeral 6 converts the coordinates and size of the operation button and the operation key and enlarges them to create an enlarged screen 61a. This enlarged screen 61a is sent to the display unit 6 via the image generation unit 13 and the image memory 7, and as shown in FIG.
An enlarged screen 61a is displayed on the display unit 6 (step 74). At this time, the enlarged operation key 52a or the like can be operated, and when operated, the processing corresponding to the operation key or the like is executed. In addition, again loupe icon key 5
When 1 is pressed, the original operation screen 61 before enlargement is displayed on the display unit 6.
【００４７】一方、ルーペアイコンキー５１が２回連続
して押圧された時は、タッチスクリーン５によって検出
されるユーザが次に押圧する操作画面６１の任意の箇所
（例えば操作キー５２付近）の座標が画面拡大位置決定
部２４に送られ、画面拡大位置決定部２４によりこの座
標を基準座標に決定する（ステップ７５）。ついで、決
定した基準座標情報を画像座標演算部１５および画像大
きさ演算部１６に送って基準座標を中心に予め定められ
た半径内の画像が予め定められた倍率になるように座標
の変換を行い、拡大画面を作成する。そして、この拡大
画面が画像発生部１３および画像メモリ７を介して表示
ユニット６に送られ、図５（ｃ）に示すように、表示ユ
ニット６に拡大画面６１ｂが表示される（ステップ７
６）。この時、操作キーボタン５２，５３は操作キー５
２ｂ，５３ｂと拡大されるとともに、これらに対する操
作も行うことができる。また、再度ルーペアイコンキー
５１が押されると、拡大前の元の操作画面６１が表示ユ
ニット６に表示される。On the other hand, when the magnifying glass icon key 51 is pressed twice in succession, the coordinates of an arbitrary position (for example, near the operation key 52) on the operation screen 61, which is detected by the touch screen 5 and is next pressed by the user. Is sent to the screen enlargement position determination unit 24, and the screen enlargement position determination unit 24 determines these coordinates as reference coordinates (step 75). Then, the determined reference coordinate information is sent to the image coordinate calculation unit 15 and the image size calculation unit 16 to convert the coordinates so that the image within a predetermined radius centered on the reference coordinates has a predetermined magnification. Do and create an enlarged screen. Then, this enlarged screen is sent to the display unit 6 via the image generator 13 and the image memory 7, and the enlarged screen 61b is displayed on the display unit 6 as shown in FIG. 5C (step 7).
6). At this time, the operation key buttons 52 and 53 are the operation key 5
2b and 53b are enlarged, and operations for these can be performed. When the loupe icon key 51 is pressed again, the original operation screen 61 before the enlargement is displayed on the display unit 6.
【００４８】このように、例えば操作画面の表示文字が
小さくて見えない、操作ボタンまたは操作キー小さくて
押し難い、操作ボタン等が多すぎて煩わしい時など、ル
ーペアイコンキー５１を１回押すことによって必要最小
限の操作キー等の画像のみが表示される拡大画面（シュ
リンク画面）６１ａを表示することができるので、操作
画面が単純化されて見易くなり、高齢者等においてもキ
ー操作を簡単かつ確実に行うことができる。また、操作
したい操作キー等の表面に表示されている文字が小さく
て見え難い時や操作しにくい時など、ルーペアイコンキ
ー５１を２回連続して押し、さらに任意の箇所を押して
指示することによって、指示した任意の箇所およびその
周囲が所定の範囲で拡大された拡大画面６１ｂが表示さ
れるので、操作キーの表面に表示されている文字が拡大
されて見えやすくなり、高齢者等においてもキー操作を
確実に行うことができ、操作性を高めることができる。In this way, by pressing the magnifier icon key 51 once, for example, when the characters displayed on the operation screen are too small to be seen, the operation buttons or operation keys are too small to be pressed, and there are too many operation buttons, etc. Since the enlarged screen (shrink screen) 61a in which only the images of the minimum necessary operation keys and the like are displayed can be displayed, the operation screen is simplified and easy to see, and even elderly people can easily and reliably perform the key operation. Can be done. In addition, when the characters displayed on the surface of the operation key you want to operate are small and difficult to see, or when it is difficult to operate, by pressing the magnifying glass icon key 2 times in succession and then pressing any point to give an instruction. Since the enlarged screen 61b in which the designated arbitrary portion and the periphery thereof are enlarged within a predetermined range is displayed, the characters displayed on the surface of the operation keys are enlarged and are easy to see, and even the elderly or the like can use the keys. The operation can be performed reliably and the operability can be improved.
【００４９】実施例４．この実施例は、表示ユニット６
に表示される操作キーにユーザの操作を補助する操作ボ
タンおよび操作キーの操作方法、表示ユニット６に表示
される各画面の操作手順および操作のサマリなどの各ヘ
ルプ機能に対応したヘルプキーを複数設け、いずれかの
ヘルプキーが押圧された場合においてそのヘルプキーに
対応したヘルプ処理が行われるようにしたものである。
以下、この実施例の作用を図７のフローチャートを用い
て説明する。Example 4. In this embodiment, the display unit 6
A plurality of help keys corresponding to each help function such as operation buttons and operation keys for assisting the user's operation, operation procedures on each screen displayed on the display unit 6, and operation summaries are displayed on the operation keys displayed on the screen. It is provided such that when any help key is pressed, the help process corresponding to the help key is performed.
The operation of this embodiment will be described below with reference to the flowchart of FIG.
【００５０】例えば空調機器において室内の温度設定を
行う場合、まず、表示ユニット６には温度設定用操作画
面が表示され、その画面の操作ボタンまたは操作キーの
画像がユーザによって押圧されると（ステップ８１，８
２）、実施例１で説明した場合と同様に押圧された操作
ボタンまたは操作キーの座標がタッチスクリーン５によ
って検出され、その座標に基づいてどの操作ボタンまた
は操作キーの画像が操作されたのかを画像識別部１８に
よって識別される。For example, in the case of setting the room temperature in an air conditioner, first, a temperature setting operation screen is displayed on the display unit 6, and the image of the operation button or operation key on the screen is pressed by the user (step 81, 8
2) Similarly to the case described in the first embodiment, the touch screen 5 detects the coordinates of the pressed operation button or operation key, and which operation button or operation key image was operated based on the detected coordinates. The image is identified by the image identification unit 18.
【００５１】この時、識別された画像が操作画面に表示
されているヘルプキー画像のうち、操作方法ヘルプキー
の画像であった場合（ステップ８３）、次に押圧された
操作ボタンまたは操作キーに対応する操作方法データが
操作方法データ記憶部２８から取り出され、その操作方
法データが操作方法ヘルプ発生部３１および画像メモリ
７を介して表示ユニット６に表示される（ステップ８
４）。また、操作手順ヘルプキーの画像が識別された場
合は（ステップ８５）、表示ユニット６に表示されてい
る操作画面に対応する操作手順データが操作手順データ
記憶部２７から取り出され、その操作手順データが操作
手順ヘルプ発生部３０および画像メモリ７を介して表示
ユニット６に表示される（ステップ８６）。いずれの場
合もヘルプデータが表示ユニット６に表示された後は元
の操作画面に戻る。また、ユーザの押圧した操作ボタン
または操作キーの画像がいずれのヘルプキーの画像でな
かった場合は、押圧された操作キー等に対応した処理が
行われ（ステップ８７）、さらに次の操作が存在すれば
次の操作画面が表示ユニットに表示される（ステップ８
８，８９）。At this time, if the identified image is the image of the operation method help key among the help key images displayed on the operation screen (step 83), the operation button or the operation key pressed next is selected. The corresponding operation method data is retrieved from the operation method data storage unit 28, and the operation method data is displayed on the display unit 6 via the operation method help generating unit 31 and the image memory 7 (step 8).
4). When the image of the operation procedure help key is identified (step 85), the operation procedure data corresponding to the operation screen displayed on the display unit 6 is retrieved from the operation procedure data storage unit 27 and the operation procedure data is retrieved. Is displayed on the display unit 6 via the operation procedure help generating unit 30 and the image memory 7 (step 86). In any case, after the help data is displayed on the display unit 6, the original operation screen is returned to. If the image of the operation button or the operation key pressed by the user is not the image of any help key, the process corresponding to the pressed operation key or the like is performed (step 87), and the next operation is performed. Then, the next operation screen is displayed on the display unit (step 8).
8, 89).
【００５２】一方、次の操作が存在せず目的の操作が終
了した後に、操作サマリヘルプキーの画像が押圧されこ
れが識別されると（ステップ９０）、今行われた操作に
よってどのようなことが行われたかという操作のサマリ
が操作サマリ提示部２９および画像メモリ７を介して表
示ユニット６に表示される（ステップ９１）。ユーザは
この表示を見ることによってユーザの意図した操作が行
われたか否かを確認する。そして、意図した操作が行わ
れ操作ボタンのうちのＯＫボタンが押された場合は（ス
テップ９２）、室内の温度設定操作は完了してその処理
が行われる。また、意図した操作が行われず操作ボタン
のうちのＮＧボタンが押された場合は、再び温度設定操
作が行えるように温度設定用の初期の操作場面が表示ユ
ニット６に表示される（ステップ９３，８１…）。On the other hand, when the image of the operation summary help key is pressed and identified after the desired operation is completed because there is no next operation (step 90), what happens depending on the operation performed now. A summary of the operation that has been performed is displayed on the display unit 6 via the operation summary presentation unit 29 and the image memory 7 (step 91). The user confirms whether or not the operation intended by the user is performed by looking at this display. When the intended operation is performed and the OK button among the operation buttons is pressed (step 92), the indoor temperature setting operation is completed and the processing is performed. If the NG button among the operation buttons is pressed without performing the intended operation, the initial operation scene for temperature setting is displayed on the display unit 6 so that the temperature setting operation can be performed again (step 93, 81 ...).
【００５３】このように、操作キー等の操作方法、操作
画面の操作手順および操作のサマリなど、ユーザの操作
補助となる各ヘルプ機能に対応したヘルプキーを設け、
画面操作中、あるいは操作終了後にいずれかのヘルプキ
ーを操作することによってユーザが要所々所で操作方
法、操作手順または操作のサマリを知るまたは確認する
ことができるので、ユーザは自らが意図する操作を容易
かつ確実に達成することができ、操作性を向上させるこ
とができる。As described above, a help key corresponding to each help function to assist the operation of the user such as the operation method of the operation key, the operation procedure of the operation screen, and the summary of the operation is provided,
The user can know or confirm the operating method, operating procedure, or summary of the operation in various places by operating the help key during the screen operation or after the operation is completed. Can be achieved easily and reliably, and operability can be improved.
【００５４】なお、上述した実施例１〜実施例４におい
てそれぞれ別の実施例として説明したが、図１に示すよ
うに、実施例１〜実施例４のすべてが実施できるユーザ
インターフェースとしてもよく、また、各実施例を適宜
組合わせたユーザインターフェースとしてもよい。Although the above-described first to fourth embodiments have been described as different embodiments, as shown in FIG. 1, a user interface capable of implementing all of the first to fourth embodiments may be used. Further, a user interface in which the respective embodiments are appropriately combined may be used.
【００５５】実施例５．図８は本発明の第５の実施例の
要部の拡大図で、この実施例は、タッチスクリーン５の
内壁５ａおよびその表面５ｂに、タッチスクリーン５お
よび操作ボタンまたは操作キーの位置が手探りでも判別
できるように突起６２を複数個設けたものである。ま
た、この実施例においては、操作ボタンまたは操作キー
の画像を押圧すると、その操作ボタンまたは操作キーに
対応した効果音、例えば操作内容が発生されるようにな
っている。Example 5. FIG. 8 is an enlarged view of an essential part of the fifth embodiment of the present invention. In this embodiment, the positions of the touch screen 5 and the operation buttons or the operation keys on the inner wall 5a of the touch screen 5 and the surface 5b thereof can be felt. A plurality of protrusions 62 are provided so that they can be distinguished. Further, in this embodiment, when an image of the operation button or the operation key is pressed, a sound effect corresponding to the operation button or the operation key, for example, operation content is generated.
【００５６】このように構成したことにより、タッチス
クリーン５の突起６２によって目の不自由な方でもタッ
チスクリーン５のだいたいどの辺りを触っているかを知
ることができる。また、手探りによって検知した操作ボ
タン等を押圧するとその操作ボタン等に対応した効果音
が発生されるので、どのようなことを行うものであるか
を容易に把握することができ、確実に操作ボタン等の操
作を行うことができる。With such a configuration, even the visually impaired person can know which part of the touch screen 5 is touched by the projection 62 of the touch screen 5. In addition, when an operation button or the like detected by fumbling is pressed, a sound effect corresponding to the operation button or the like is generated, so that it is possible to easily grasp what is to be done, and the operation button can be reliably operated. Etc. can be performed.
【００５７】なお、上述の実施例では操作ボタンおよび
操作キーの画像が表示されるタッチスクリーン５上およ
びその周辺に突起６２を設け、この突起６２の付近の操
作ボタンまたは操作キーの画像を押圧した時に、実施例
２において説明した発明とほぼ同様の発明が実施された
場合を説明したが、この発明にさらに実施例１、実施例
３および実施例４の発明を実施してもよい。In the above-described embodiment, the projection 62 is provided on and around the touch screen 5 on which the images of the operation buttons and the operation keys are displayed, and the image of the operation button or the operation key near the projection 62 is pressed. At times, a case has been described in which the invention substantially similar to the invention described in the second embodiment is implemented, but the inventions of the first, third, and fourth embodiments may be further implemented in the present invention.
【００５８】[0058]
【発明の効果】以上のように本発明に係るユーザインタ
ーフェースは、複数の操作ボタンおよび操作キーを表示
する表示ユニットと、表示ユニットの表面に配置され、
指示された位置の座標を検出するタッチスクリーンと、
タッチスクリーンの出力信号に基づいて操作ボタンおよ
び操作キーに対応した制御を行う制御部とを備え、制御
部は、タッチスクリーンの出力信号に基づいて表示ユニ
ットに表示された操作ボタンおよび操作キーのうち、指
示されたものの画像を識別する画像識別手段と、画像識
別手段によって識別された画像を他の表示画像と識別可
能にするための特殊表示をさせるかどうかを判断する特
殊効果識別手段と、特殊効果識別手段の判断結果に基づ
いて表示ユニットの指示された操作ボタンまたは操作キ
ーに対して特殊表示をさせる第１の制御手段とを有する
ので、いずれの操作ボタンまたは操作キーが指示された
のかを容易に識別でき、操作性の向上を計ることができ
る。As described above, the user interface according to the present invention is provided with a display unit for displaying a plurality of operation buttons and operation keys, and arranged on the surface of the display unit.
A touch screen that detects the coordinates of the indicated position,
A control unit that performs control corresponding to the operation buttons and the operation keys based on the output signals of the touch screen, and the control unit includes the operation buttons and the operation keys displayed on the display unit based on the output signals of the touch screen. An image identifying means for identifying the image of the instructed object, a special effect identifying means for determining whether or not to perform a special display for distinguishing the image identified by the image identifying means from other display images, and a special effect Since it has the first control means for displaying a special display on the operation button or operation key instructed on the display unit based on the judgment result of the effect identification means, it is possible to determine which operation button or operation key has been instructed. It can be easily identified and the operability can be improved.
【００５９】また、特殊表示は指示された操作ボタンま
たは操作キーの周囲にダイナミックな動作が付加された
アニメーション画像であるので、いずれの操作ボタンま
たは操作キーが指示されたことを視覚的に早く捉えて確
認することができ、操作性の向上を一層計ることができ
る。Further, since the special display is an animation image in which a dynamic action is added around the designated operation button or operation key, it is possible to visually grasp quickly which operation button or operation key is designated. It can be confirmed by checking, and the operability can be further improved.
【００６０】さらに、第１の制御手段は、特殊表示のア
ニメーション画像である波紋画像のデータが格納されて
いる波紋画像データ記憶部と、波紋画像データ記憶部の
波紋画像データを順次取り出し波紋画像データの所在、
数および表示順序を決定する波紋画像パターン決定部
と、波紋画像パターン決定部からの決定情報、波紋画像
座標演算部からの表示座標情報および波紋画像大きさ演
算部からの表示大きさ情報に基づいて波紋画像を発生さ
せる波紋画像発生部とを有するので、操作中などにおい
て操作ボタン等が操作している指によって隠されて見え
なくなってしまっても指示されたことが容易に確認する
ことができ、画面操作の実感の不足を補うことができ
る。Further, the first control means sequentially extracts the ripple image data storage unit in which the data of the ripple image which is the animation image of the special display is stored, and the ripple image data in the ripple image data storage unit in order. Whereabouts of
Based on the ripple image pattern determination unit that determines the number and display order, the determination information from the ripple image pattern determination unit, the display coordinate information from the ripple image coordinate calculation unit, and the display size information from the ripple image size calculation unit. Since it has a ripple image generation unit for generating a ripple image, it is possible to easily confirm that the instruction has been given even if the operation button or the like is hidden by the operating finger during operation and disappears. It is possible to make up for the lack of sense of screen operation.
【００６１】本発明に係るユーザインターフェースの制
御部は、タッチスクリーンの出力信号に基づいて表示ユ
ニットに表示された操作ボタンおよび操作キーのうち、
指示されたものの画像を識別する画像識別手段と、画像
識別手段によって識別された画像に対して他の表示画像
と識別可能にするための効果音を発生させるかどうかを
判断する特殊効果識別手段と、特殊効果識別手段の判断
結果に基づいて表示ユニットの指示された操作ボタンま
たは操作キーに対応した効果音をスピーカから発生させ
る第２の制御手段とを有するので、視覚だけでなく聴覚
からもいずれの操作ボタンまたは操作キーが指示された
のかを識別することができ、画面操作の実感の不足を補
うことができる。The control unit of the user interface according to the present invention, among the operation buttons and the operation keys displayed on the display unit based on the output signal of the touch screen,
Image identifying means for identifying the image of the instructed object, and special effect identifying means for determining whether to generate a sound effect for distinguishing the image identified by the image identifying means from other display images. Since it has the second control means for causing the speaker to generate the sound effect corresponding to the operation button or the operation key instructed on the display unit based on the judgment result of the special effect identification means, it is not only visually but also auditorily. It is possible to identify whether the operation button or the operation key has been instructed, and it is possible to compensate for the lack of actual feeling of screen operation.
【００６２】また、第２の制御手段は、操作ボタンおよ
び操作キーに対応した効果音データが格納されている効
果音データ記憶部と、指定された操作ボタンまたは操作
キーに対応した効果音データを効果音データ記憶部から
取り出しその効果音データの所在を決定する効果音パタ
ーン決定部と、効果音パターン決定部からの決定情報に
基づいて効果音をスピーカより発生させる効果音発生部
とを有し、また、タッチスクリーンの押圧力を検出する
圧力検出手段を有し、押圧力が所定圧力以上の時には指
定された操作ボタンまたは操作キーに対応した効果音を
発生させるとともにその処理を行い、押圧力が所定圧力
より小さい時には効果音のみを発生させるので、表示ユ
ニットに表示された操作ボタンおよび操作キーのうち、
指示された操作ボタンまたは操作キーの識別が一層容易
になり、操作性を向上させることができる。Further, the second control means stores the sound effect data storage section in which the sound effect data corresponding to the operation button and the operation key is stored, and the sound effect data corresponding to the specified operation button or operation key. The sound effect data storage unit has a sound effect pattern determination unit that determines the location of the sound effect data, and a sound effect generation unit that causes the speaker to generate sound effect based on the determination information from the sound effect pattern determination unit. Also, it has a pressure detecting means for detecting the pressing force of the touch screen, and when the pressing force is equal to or higher than a predetermined pressure, a sound effect corresponding to the designated operation button or operation key is generated and its processing is performed to perform the pressing force. When is less than the predetermined pressure, only the sound effect is generated, so of the operation buttons and keys displayed on the display unit,
The instructed operation button or operation key can be identified more easily, and operability can be improved.
【００６３】本発明に係るユーザインターフェースは、
複数の操作ボタンおよび拡大表示キーを含む操作キーを
表示する表示ユニットと、タッチスクリーンと、制御部
とを備え、制御部は、タッチスクリーンの出力信号に基
づいて表示ユニットに表示された操作ボタンおよび操作
キーのうち、指示されたものの画像が拡大表示キーの画
像であるかどうかを識別する画像識別手段と、画像識別
手段において拡大表示キーが識別された時には必要最小
限の操作ボタンと操作キーとの組合わせからなるシュリ
ンク画面を表示ユニットに表示させる第３の制御手段と
を有し、第３の制御手段は、シュリンク画面のデータが
複数格納されているシュリンク画面データ記憶部と、シ
ュリンク画面データ記憶部に格納されているシュリンク
画面データをもとに操作ボタンまたは操作キーの画像の
大きさあるいは画像の数量を決定する画像大きさ決定部
および画像数量決定部と、画像大きさ決定部および画像
数量決定部の決定情報に基づいて画像の大きさ演算およ
び座標演算をそれぞれ行う演算部と、これらの演算結果
に基づいてシュリンク画面を発生させる画像発生部とを
有するので、操作ボタン等が多すぎて煩わしい時などに
おいて必要最小限の操作ボタン等のみが表示される見易
く操作しやすい単純化された操作画面を表示することが
でき、高齢者等においてもボタン操作を簡単かつ確実に
行うことができて便利である。The user interface according to the present invention is
The control unit includes a display unit that displays a plurality of operation buttons and operation keys including an enlarged display key, a touch screen, and a control unit, and the control unit includes an operation button and an operation button displayed on the display unit based on an output signal of the touch screen. Of the operation keys, an image identification means for identifying whether or not the image of the instructed one is the image of the enlarged display key, and a minimum necessary operation button and operation key when the enlarged display key is identified by the image identification means. And a third control means for displaying a shrink screen composed of a combination of the above on the display unit, and the third control means is a shrink screen data storage section in which a plurality of data of the shrink screen are stored, and the shrink screen data. Based on the shrink screen data stored in the storage unit, the size or image of the operation button or operation key image An image size determination unit and an image quantity determination unit that determine the quantity of the image, and an operation unit that respectively performs image size calculation and coordinate calculation based on the determination information of the image size determination unit and the image quantity determination unit, and these Since it has an image generation unit that generates a shrink screen based on the calculation result, only the minimum necessary operation buttons etc. are displayed when there are too many operation buttons etc. A screen can be displayed, and it is convenient that elderly people can easily and reliably perform button operations.
【００６４】本発明に係るユーザインターフェースの制
御部は、タッチスクリーンの出力信号に基づいて表示ユ
ニットに表示された操作ボタンおよび操作キーのうち、
指示されたものの画像が拡大表示キーの画像であるかど
うかを識別する画像識別手段と、画像識別手段において
拡大表示キーが識別され次に画面の任意の箇所が押され
て指定されると、指示された箇所の座標を中心に予め定
められた半径内の画像を予め定められた倍率で表示ユニ
ットに拡大表示させる第４の制御手段を有し、第４の制
御手段は、指示された箇所の座標が拡大される画面の中
心基準となることを決定する画面拡大位置決定部と、画
面拡大位置決定部によって決定された基準座標を中心に
予め定められた半径内の画像が予め定められた倍率にな
るように演算する画像座標演算部および画像大きさ演算
部と、演算結果に基づいて拡大された拡大画面を発生さ
せる画像発生部とを有するので、操作キー等の表面に表
示されている文字が小さくて見え難い時や操作しにくい
時などにおいて任意の箇所およびその周囲を容易に拡大
させることができ、高齢者等においてもキー操作を簡単
かつ確実に行え、操作性を高めることができる。The control unit of the user interface according to the present invention, among the operation buttons and operation keys displayed on the display unit based on the output signal of the touch screen,
An image identifying means for identifying whether or not the image of the instructed object is an image of the enlargement display key, and the enlargement display key is identified by the image identification means, and next, an arbitrary portion of the screen is pressed and designated. A fourth control means for enlarging and displaying on the display unit an image within a predetermined radius centered on the coordinates of the designated location at a predetermined magnification, and the fourth control means is for the designated location. A screen enlargement position determining unit that determines that the coordinates serve as the center reference of the screen to be enlarged, and a predetermined magnification of an image within a predetermined radius centered on the reference coordinates determined by the screen enlargement position determining unit Since it has an image coordinate calculation unit and an image size calculation unit that calculate so that the characters are displayed on the surface of operation keys, etc., it has an image generation unit that generates an enlarged screen enlarged based on the calculation result. Small can be easily expanded to any location and surrounding the like when hardly or when operations obscured, even perform key operations easily and reliably in the elderly and the like, it is possible to improve the operability.
【００６５】本発明に係るユーザインターフェースは、
複数の操作ボタンおよび複数のヘルプキー含む操作キー
を表示する表示ユニットと、タッチスクリーンと、制御
部とを備え、制御部は、タッチスクリーンの出力信号に
基づいて表示ユニットに表示された操作ボタンおよび操
作キーのうち、指示されたものの画像がヘルプキーの画
像であるかどうかを識別する画像識別手段と、画像識別
手段においていずれかのヘルプキーが識別された時には
そのヘルプキーに対応したヘルプ処理を行わせる第５の
制御手段を有し、第５の制御手段は、操作される操作ボ
タンまたは操作キーの履歴が格納される操作履歴記憶部
と、操作ボタンおよび操作キーの操作方法データが格納
された操作方法データ記憶部と、表示ユニットに表示さ
れる各画面の操作手順データが格納されている操作手順
データ記憶部と、画像識別手段において操作方法ヘルプ
キーが識別された時に指定された操作ボタンまたは操作
キーの操作方法データを表示ユニットに表示させる操作
方法発生部と、画像識別手段において操作手順ヘルプキ
ーが識別された時に表示ユニットに表示されている画面
の操作手順データを表示ユニットに表示させる操作手順
発生部と、表示ユニットに表示されている画面の操作終
了後に画像識別手段において操作サマリヘルプキーが識
別された時に操作履歴記憶部に格納された操作キーの履
歴に基づいて操作サマリを表示ユニットに表示させる操
作サマリ提示部とを有するので、ユーザはその操作の確
認を取りながら操作キー等の操作を進めることができ、
確実に意図する操作を達成することができる。The user interface according to the present invention is
The control unit includes a display unit that displays a plurality of operation buttons and operation keys including a plurality of help keys, a touch screen, and a control unit, and the control unit includes an operation button and a display unit that are displayed on the display unit based on an output signal of the touch screen. Of the operation keys, the image identification means for identifying whether the image of the instructed one is the image of the help key, and when any of the help keys is identified by the image identification means, the help processing corresponding to the help key is performed. A fifth control means for performing the operation, wherein the fifth control means stores an operation history storage section in which a history of operation buttons or operation keys to be operated is stored, and operation method data of operation buttons and operation keys. An operating method data storage section, an operating procedure data storage section in which operating procedure data for each screen displayed on the display unit is stored, When an operation method help key is identified by the image identification means, an operation method generation section that causes the display unit to display operation method data of the operation button or operation key specified when the operation method help key is identified, and when the operation procedure help key is identified by the image identification means Operate when the operation summary help key is identified by the image identification means after the operation procedure generation part that displays the operation procedure data of the screen displayed on the display unit on the display unit and the operation of the screen displayed on the display unit is completed. Since it has an operation summary presentation unit that displays the operation summary on the display unit based on the history of the operation keys stored in the history storage unit, the user can proceed with the operation of the operation keys while confirming the operation. ,
The intended operation can be reliably achieved.
【００６６】本発明に係るユーザインターフェースは、
タッチスクリーンの周囲およびその表面に突起を複数設
けたので、目の不自由な方にもタッチスクリーンや操作
ボタンおよび操作キーの画像が表示される位置を容易に
検知することができ、操作性を向上させることができ
る。The user interface according to the present invention is
Since multiple protrusions are provided on the periphery of the touch screen and on the surface of the touch screen, even the visually impaired can easily detect the positions where the images of the touch screen, operation buttons and operation keys are displayed, thus improving operability. Can be improved.
【図１】 本発明の第１〜第４の実施例の構成を示すブ
ロック図である。FIG. 1 is a block diagram showing a configuration of first to fourth embodiments of the present invention.
【図２】 本発明の第１の実施例に係る表示ユニットに
表示された操作画面の要部の拡大図である。FIG. 2 is an enlarged view of a main part of an operation screen displayed on the display unit according to the first embodiment of the present invention.
【図３】 本発明の第２の実施例に係る表示ユニットに
表示された操作画面の作用説明図である。FIG. 3 is an operation explanatory view of an operation screen displayed on the display unit according to the second embodiment of the present invention.
【図４】 本発明の第２の実施例の作用を示すフローチ
ャートである。FIG. 4 is a flowchart showing the operation of the second embodiment of the present invention.
【図５】 本発明の第３の実施例に係る表示ユニットに
表示される操作画面の説明図である。FIG. 5 is an explanatory diagram of an operation screen displayed on the display unit according to the third embodiment of the present invention.
【図６】 本発明の第３の実施例の作用を示すフローチ
ャートである。FIG. 6 is a flowchart showing the operation of the third embodiment of the present invention.
【図７】 本発明の第４の実施例の作用を示すフローチ
ャートである。FIG. 7 is a flowchart showing the operation of the fourth embodiment of the present invention.
【図８】 本発明の第５の実施例の要部の拡大図であ
る。FIG. 8 is an enlarged view of a main part of the fifth embodiment of the present invention.
１ スピーカ、２ 効果音発生部、３ 効果音データ記
憶部、４ 効果音パターン決定部、５ タッチスクリー
ン、６ 表示ユニット、８ 波紋画像発生部、９ 波紋
画像データ記憶部、１０ 波紋画像座標演算部、１１
波紋画像大きさ演算部、１２ 波紋画像パターン決定
部、１３ 画像発生部、１５ 画像座標演算部、１６
画像大きさ演算部、１８ 画像識別部、１９ 特殊効果
識別部、２０ タッチ圧力検出部、２１ シュリンク画
面データ記憶部、２２ 画像大きさ決定部、２３ 画像
数量決定部、２４ 画面拡大位置決定部、２５ 操作履
歴記憶部、２７ 操作手順データ記憶部、２８ 操作方
法データ記憶部、２９ 操作サマリ提示部、３０ 操作
手順発生部、３１ 操作方法発生部、４１，４３，５
６，５７，５８，５９，６０ 操作ボタン、４２ 波
紋、４４，５２，５３，５４，５５ 操作キー、５１
ルーペアイコンキー、６２ 突起。1 speaker, 2 sound effect generation unit, 3 sound effect data storage unit, 4 sound effect pattern determination unit, 5 touch screen, 6 display unit, 8 ripple image generation unit, 9 ripple image data storage unit, 10 ripple image coordinate calculation unit , 11
Ripple image size calculator, 12 ripple image pattern determiner, 13 image generator, 15 image coordinate calculator, 16
Image size calculation unit, 18 image identification unit, 19 special effect identification unit, 20 touch pressure detection unit, 21 shrink screen data storage unit, 22 image size determination unit, 23 image quantity determination unit, 24 screen enlarged position determination unit, 25 operation history storage unit, 27 operation procedure data storage unit, 28 operation method data storage unit, 29 operation summary presentation unit, 30 operation procedure generation unit, 31 operation method generation unit, 41, 43, 5
6, 57, 58, 59, 60 operation buttons, 42 ripples, 44, 52, 53, 54, 55 operation keys, 51
Loupe icon key, 62 protrusions.
───────────────────────────────────────────────────── フロントページの続き (72)発明者 上村 一穂 鎌倉市大船二丁目14番40号 三菱電機株式 会社住環境研究開発センター内 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Kazuho Uemura 2-14-40 Ofuna, Kamakura City Mitsubishi Electric Corporation
Claims (13)
する表示ユニットと、該表示ユニットの表面に配置さ
れ、指示された位置の座標を検出するタッチスクリーン
と、該タッチスクリーンの出力信号に基づいて前記操作
ボタンおよび操作キーに対応した制御を行う制御部とを
備え、 前記制御部は、前記タッチスクリーンの出力信号に基づ
いて前記表示ユニットに表示された操作ボタンおよび操
作キーのうち、指示されたものの画像を識別する画像識
別手段と、該画像識別手段によって識別された画像を他
の表示画像と識別可能にするための特殊表示をさせるか
どうかを判断する特殊効果識別手段と、該特殊効果識別
手段の判断結果に基づいて前記表示ユニットの指示され
た操作ボタンまたは操作キーに対して特殊表示をさせる
第１の制御手段とを有することを特徴とするユーザイン
ターフェース。1. A display unit for displaying a plurality of operation buttons and operation keys, a touch screen disposed on the surface of the display unit for detecting coordinates of a designated position, and based on an output signal of the touch screen. A control unit that performs control corresponding to the operation button and the operation key, wherein the control unit is instructed among the operation button and the operation key displayed on the display unit based on the output signal of the touch screen. An image identifying means for identifying an image of an object, a special effect identifying means for determining whether or not to perform a special display for distinguishing the image identified by the image identifying means from other display images, and the special effect identifying A first control hand for displaying a special display on the operation button or operation key instructed on the display unit based on the determination result of the means. User interface and having and.
操作キーの周囲にダイナミックな動作が付加されたアニ
メーション画像であることを特徴とする請求項１記載の
ユーザインターフェース。2. The user interface according to claim 1, wherein the special display is an animation image in which a dynamic operation is added around a designated operation button or operation key.
ション画像である波紋画像のデータが格納されている波
紋画像データ記憶部と、該波紋画像データ記憶部の波紋
画像データを順次取り出し波紋画像データの所在、数お
よび表示順序を決定する波紋画像パターン決定部と、該
波紋画像パターン決定部からの決定情報、波紋画像座標
演算部からの表示座標情報および波紋画像大きさ演算部
からの表示大きさ情報に基づいて波紋画像を発生させる
波紋画像発生部とを有することを特徴とする請求項２記
載のユーザインターフェース。3. The first control means sequentially extracts the ripple image data storage unit in which the data of the ripple image which is the animation image of the special display is stored, and the ripple image data in the ripple image data storage unit is sequentially taken out. A ripple image pattern determining unit that determines the location, number and display order of data, determination information from the ripple image pattern determining unit, display coordinate information from the ripple image coordinate calculating unit and display size from the ripple image size calculating unit. 3. The user interface according to claim 2, further comprising a ripple image generation unit that generates a ripple image based on the height information.
する表示ユニットと、該表示ユニットの表面に配置さ
れ、指示された位置の座標を検出するタッチスクリーン
と、該タッチスクリーンの出力信号に基づいて前記操作
ボタンおよび操作キーに対応した制御を行う制御部とを
備え、 前記制御部は、前記タッチスクリーンの出力信号に基づ
いて前記表示ユニットに表示された操作ボタンおよび操
作キーのうち、指示されたものの画像を識別する画像識
別手段と、該画像識別手段によって識別された画像に対
して他の表示画像と識別可能にするための効果音を発生
させるかどうかを判断する特殊効果識別手段と、該特殊
効果識別手段の判断結果に基づいて前記表示ユニットの
指示された操作ボタンまたは操作キーに対応した効果音
をスピーカから発生させる第２の制御手段とを有するこ
とを特徴とするユーザインターフェース。4. A display unit for displaying a plurality of operation buttons and operation keys, a touch screen arranged on the surface of the display unit for detecting coordinates of a designated position, and based on an output signal of the touch screen. A control unit that performs control corresponding to the operation button and the operation key, wherein the control unit is instructed among the operation button and the operation key displayed on the display unit based on the output signal of the touch screen. Image identifying means for identifying an image of an object, special effect identifying means for determining whether or not to generate a sound effect for distinguishing the image identified by the image identifying means from other display images, A sound effect corresponding to the operation button or operation key instructed on the display unit based on the determination result of the special effect identification means is output from the speaker. User interface and having a second control means for generating.
ンの押圧力を検出する圧力検出手段を有し、押圧力が所
定圧力以上の時には指定された操作ボタンまたは操作キ
ーに対応した効果音を発生させるとともにその処理を行
い、押圧力が所定圧力より小さい時には効果音のみを発
生させることを特徴とする請求項４記載のユーザインタ
ーフェース。5. The second control means has pressure detection means for detecting the pressing force of the touch screen, and when the pressing force is equal to or higher than a predetermined pressure, a sound effect corresponding to a designated operation button or operation key is generated. The user interface according to claim 4, wherein when the pressing force is less than a predetermined pressure, only a sound effect is generated, which is generated and processed.
作キーに対応した効果音データが格納されている効果音
データ記憶部と、指定された操作ボタンまたは操作キー
に対応した効果音データを前記効果音データ記憶部から
取り出しその効果音データの所在を決定する効果音パタ
ーン決定部と、該効果音パターン決定部からの決定情報
に基づいて効果音をスピーカより発生させる効果音発生
部とを有することを特徴とする請求項４または５記載の
ユーザインターフェース。6. The second control means stores a sound effect data storage section in which sound effect data corresponding to operation buttons and operation keys are stored, and sound effect data corresponding to a specified operation button or operation key. A sound effect pattern determination unit that extracts the sound effect data from the sound effect data storage unit and determines the location of the sound effect data, and a sound effect generation unit that causes a speaker to generate sound effect based on the determination information from the sound effect pattern determination unit. The user interface according to claim 4, further comprising:
含む操作キーを表示する表示ユニットと、該表示ユニッ
トの表面に配置され、指示された位置の座標を検出する
タッチスクリーンと、該タッチスクリーンの出力信号に
基づいて前記操作ボタンおよび操作キーに対応した制御
を行う制御部とを備え、 前記制御部は、前記タッチスクリーンの出力信号に基づ
いて前記表示ユニットに表示された操作ボタンおよび操
作キーのうち、指示されたものの画像が前記拡大表示キ
ーの画像であるかどうかを識別する画像識別手段と、該
画像識別手段において拡大表示キーが識別された時には
必要最小限の操作ボタンと操作キーとの組合わせからな
るシュリンク画面を前記表示ユニットに表示させる第３
の制御手段とを有することを特徴とするユーザインター
フェース。7. A display unit for displaying operation keys including a plurality of operation buttons and an enlarged display key, a touch screen arranged on the surface of the display unit for detecting coordinates of a designated position, and a touch screen of the touch screen. And a control unit that performs control corresponding to the operation button and the operation key based on an output signal, the control unit of the operation button and the operation key displayed on the display unit based on the output signal of the touch screen. Of these, an image identification means for identifying whether or not the image of the instructed one is the image of the enlarged display key, and a minimum necessary operation button and operation key when the enlarged display key is identified by the image identification means. Thirdly, a shrink screen composed of a combination is displayed on the display unit.
And a control means of the user interface.
のデータが複数格納されているシュリンク画面データ記
憶部と、該シュリンク画面データ記憶部に格納されてい
るシュリンク画面データをもとに操作ボタンまたは操作
キーの画像の大きさあるいは画像の数量を決定する画像
大きさ決定部および画像数量決定部と、前記画像大きさ
決定部および画像数量決定部の決定情報に基づいて画像
の大きさ演算および座標演算をそれぞれ行う画像大きさ
演算部および画像座標演算部と、これらの演算結果に基
づいてシュリンク画面を発生させる画像発生部とを有す
ることを特徴とする請求項７記載のユーザインターフェ
ース。8. The third control means includes a shrink screen data storage section in which a plurality of the shrink screen data are stored, and an operation button based on the shrink screen data stored in the shrink screen data storage section. Alternatively, an image size determination unit and an image quantity determination unit that determine the image size or the number of images of the operation keys, and an image size calculation and an image size calculation based on the determination information of the image size determination unit and the image number determination unit. The user interface according to claim 7, further comprising: an image size calculation unit and an image coordinate calculation unit that respectively perform coordinate calculation; and an image generation unit that generates a shrink screen based on the calculation results.
含む操作キーを表示する表示ユニットと、該表示ユニッ
トの表面に配置され、指示された位置の座標を検出する
タッチスクリーンと、該タッチスクリーンの出力信号に
基づいて前記操作ボタンおよび操作キーに対応した制御
を行う制御部とを備え、 前記制御部は、前記タッチスクリーンの出力信号に基づ
いて前記表示ユニットに表示された操作ボタンおよび操
作キーのうち、指示されたものの画像が前記拡大表示キ
ーの画像であるかどうかを識別する画像識別手段と、該
画像識別手段において拡大表示キーが識別され次に画面
の任意の箇所が押されて指定されると、指示された箇所
の座標を中心に予め定められた半径内の画像を予め定め
られた倍率で前記表示ユニットに拡大表示させる第４の
制御手段を有することを特徴とするユーザインターフェ
ース。9. A display unit for displaying operation keys including a plurality of operation buttons and an enlarged display key, a touch screen arranged on the surface of the display unit for detecting coordinates of a designated position, and a touch screen of the touch screen. And a control unit that performs control corresponding to the operation button and the operation key based on an output signal, the control unit of the operation button and the operation key displayed on the display unit based on the output signal of the touch screen. Of these, the image identification means for identifying whether the image of the instructed one is the image of the enlargement display key, the enlargement display key is identified by the image identification means, and then any portion of the screen is pressed to designate. Then, an image within a predetermined radius centered on the coordinates of the designated location is enlarged and displayed on the display unit at a predetermined magnification. User interface and having a fourth control means.
所の座標が拡大される画面の中心基準となることを決定
する画面拡大位置決定部と、該画面拡大位置決定部によ
って決定された基準座標を中心に予め定められた半径内
の画像が予め定められた倍率になるように演算する演算
部と、演算結果に基づいて拡大された拡大画面を発生さ
せる画像発生部とを有することを特徴とする請求項９記
載のユーザインターフェース。10. The fourth control means is determined by a screen enlargement position determination unit that determines that the coordinates of the designated point serve as the center reference of the screen to be enlarged, and the screen enlargement position determination unit. And an image generation unit that generates an enlarged screen enlarged on the basis of the calculation result, and a calculation unit that calculates an image within a predetermined radius centered on the reference coordinates to have a predetermined magnification. The user interface according to claim 9, wherein the user interface is provided.
キー含む操作キーを表示する表示ユニットと、該表示ユ
ニットの表面に配置され、指示された位置の座標を検出
するタッチスクリーンと、該タッチスクリーンの出力信
号に基づいて前記操作ボタンおよび操作キーに対応した
制御を行う制御部とを備え、 前記制御部は、前記タッチスクリーンの出力信号に基づ
いて前記表示ユニットに表示された操作ボタンおよび操
作キーのうち、指示されたものの画像が前記ヘルプキー
の画像であるかどうかを識別する画像識別手段と、該画
像識別手段においていずれかのヘルプキーが識別された
時にはそのヘルプキーに対応したヘルプ処理を行わせる
第５の制御手段を有することを特徴とするユーザインタ
ーフェース。11. A display unit for displaying operation keys including a plurality of operation buttons and a plurality of help keys, a touch screen arranged on the surface of the display unit for detecting coordinates of a designated position, and a touch screen of the touch screen. And a control unit that performs control corresponding to the operation button and the operation key based on an output signal, the control unit of the operation button and the operation key displayed on the display unit based on the output signal of the touch screen. Of these, image identification means for identifying whether or not the image of the instructed one is the image of the help key, and when any of the help keys is identified by the image identification means, the help processing corresponding to the help key is performed. A user interface characterized by having a fifth control means.
タンまたは操作キーの履歴が格納される操作履歴記憶部
と、操作ボタンおよび操作キーの操作方法データが格納
された操作方法データ記憶部と、前記表示ユニットに表
示される各画面の操作手順データが格納されている操作
手順データ記憶部と、前記画像識別手段において操作方
法ヘルプキーが識別された時に指定された操作ボタンま
たは操作キーの操作方法データを前記表示ユニットに表
示させる操作方法発生部と、前記画像識別手段において
操作手順ヘルプキーが識別された時に表示ユニットに表
示されている画面の操作手順データを前記表示ユニット
に表示させる操作手順発生部と、前記表示ユニットに表
示されている画面の操作終了後に前記画像識別手段にお
いて操作サマリヘルプキーが識別された時に前記操作履
歴記憶部に格納された操作キーの履歴に基づいて操作サ
マリを前記表示ユニットに表示させる操作サマリ提示部
とを有することを特徴とする請求項１１記載のユーザイ
ンターフェース。12. The fifth control means comprises an operation history storage section for storing a history of operation buttons or operation keys to be operated, and an operation method data storage section for storing operation method data of the operation buttons and operation keys. And an operation procedure data storage section in which operation procedure data of each screen displayed on the display unit is stored, and an operation button or an operation key designated when the operation method help key is identified by the image identification means. An operation method generating section for displaying operation method data on the display unit, and an operation for displaying on the display unit the operation procedure data of the screen displayed on the display unit when the operation procedure help key is identified by the image identification means. After the operation of the procedure generating section and the screen displayed on the display unit is completed, the operation identification 12. The user interface according to claim 11, further comprising an operation summary presenting unit that displays an operation summary on the display unit based on a history of the operation keys stored in the operation history storage unit when a push key is identified. .
面に突起を複数設けたことを特徴とする請求項１，２，
３，４，５，６，７，８，９，１０，１１または１２記
載のユーザインターフェース。13. The touch screen is provided with a plurality of protrusions around and on the surface of the touch screen.
The user interface according to 3, 4, 5, 6, 7, 8, 9, 10, 11 or 12.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP3323295A JPH08227341A (en) | 1995-02-22 | 1995-02-22 | User interface |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP3323295A JPH08227341A (en) | 1995-02-22 | 1995-02-22 | User interface |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| JPH08227341A true JPH08227341A (en) | 1996-09-03 |
Family
ID=12380718
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| JP3323295A Pending JPH08227341A (en) | 1995-02-22 | 1995-02-22 | User interface |
Country Status (1)
| Country | Link |
|---|---|
| JP (1) | JPH08227341A (en) |
Cited By (165)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2000259311A (en) * | 1999-03-04 | 2000-09-22 | Fujitsu Takamisawa Component Ltd | Processor and method for information processing |
| JP2001051759A (en) * | 1999-08-12 | 2001-02-23 | Nec Corp | Operation reaction sound generating device and recording medium where operation reaction sound generating program is recorded |
| WO2005101172A1 (en) * | 2004-03-31 | 2005-10-27 | Sharp Kabushiki Kaisha | Electronic device |
| JP2007060033A (en) * | 2005-08-22 | 2007-03-08 | Ricoh Co Ltd | Multifunction image forming apparatus, screen display mode setting method, program and recording medium |
| JP2007334747A (en) * | 2006-06-16 | 2007-12-27 | Miyake Design Jimusho:Kk | Information processor, portable telephone set, contact input picture forming apparatus, information processing program, and recording medium recording information processing program |
| JP2008211383A (en) * | 2007-02-23 | 2008-09-11 | Matsushita Electric Works Ltd | Remote monitoring system |
| JP2011118595A (en) * | 2009-12-02 | 2011-06-16 | Square Enix Co Ltd | Apparatus, method and program for processing user interface |
| US8032843B2 (en) | 1999-12-20 | 2011-10-04 | Apple Inc. | User interface for providing consolidation and access |
| US8255830B2 (en) | 2009-03-16 | 2012-08-28 | Apple Inc. | Methods and graphical user interfaces for editing on a multifunction device with a touch screen display |
| US8427445B2 (en) | 2004-07-30 | 2013-04-23 | Apple Inc. | Visual expander |
| JP2014096074A (en) * | 2012-11-12 | 2014-05-22 | Sony Corp | Information processing device, communication system, and information processing method |
| US8744852B1 (en) | 2004-10-01 | 2014-06-03 | Apple Inc. | Spoken interfaces |
| US8799813B2 (en) | 2000-01-05 | 2014-08-05 | Apple Inc. | Method and system for providing an embedded application tool bar |
| US8892446B2 (en) | 2010-01-18 | 2014-11-18 | Apple Inc. | Service orchestration for intelligent automated assistant |
| US8977584B2 (en) | 2010-01-25 | 2015-03-10 | Newvaluexchange Global Ai Llp | Apparatuses, methods and systems for a digital conversation management platform |
| US9207855B2 (en) | 2006-10-26 | 2015-12-08 | Apple Inc. | Portable multifunction device, method, and graphical user interface for adjusting an insertion point marker |
| EP2960767A1 (en) * | 2014-06-24 | 2015-12-30 | Google, Inc. | Computerized systems and methods for rendering an animation of an object in response to user input |
| US9262612B2 (en) | 2011-03-21 | 2016-02-16 | Apple Inc. | Device access using voice authentication |
| US9300784B2 (en) | 2013-06-13 | 2016-03-29 | Apple Inc. | System and method for emergency calls initiated by voice command |
| US9330720B2 (en) | 2008-01-03 | 2016-05-03 | Apple Inc. | Methods and apparatus for altering audio output signals |
| US9338493B2 (en) | 2014-06-30 | 2016-05-10 | Apple Inc. | Intelligent automated assistant for TV user interactions |
| US9368114B2 (en) | 2013-03-14 | 2016-06-14 | Apple Inc. | Context-sensitive handling of interruptions |
| US9430463B2 (en) | 2014-05-30 | 2016-08-30 | Apple Inc. | Exemplar-based natural language processing |
| JP2016167298A (en) * | 2016-04-28 | 2016-09-15 | ソニー株式会社 | Information processing device, information processing method and program |
| US9483461B2 (en) | 2012-03-06 | 2016-11-01 | Apple Inc. | Handling speech synthesis of content for multiple languages |
| US9495129B2 (en) | 2012-06-29 | 2016-11-15 | Apple Inc. | Device, method, and user interface for voice-activated navigation and browsing of a document |
| US9502031B2 (en) | 2014-05-27 | 2016-11-22 | Apple Inc. | Method for supporting dynamic grammars in WFST-based ASR |
| US9529524B2 (en) | 2008-03-04 | 2016-12-27 | Apple Inc. | Methods and graphical user interfaces for editing on a portable multifunction device |
| US9535906B2 (en) | 2008-07-31 | 2017-01-03 | Apple Inc. | Mobile device having human language translation capability with positional feedback |
| US9576574B2 (en) | 2012-09-10 | 2017-02-21 | Apple Inc. | Context-sensitive handling of interruptions by intelligent digital assistant |
| US9582608B2 (en) | 2013-06-07 | 2017-02-28 | Apple Inc. | Unified ranking with entropy-weighted information for phrase-based semantic auto-completion |
| US9620105B2 (en) | 2014-05-15 | 2017-04-11 | Apple Inc. | Analyzing audio input for efficient speech and music recognition |
| US9620104B2 (en) | 2013-06-07 | 2017-04-11 | Apple Inc. | System and method for user-specified pronunciation of words for speech synthesis and recognition |
| US9626955B2 (en) | 2008-04-05 | 2017-04-18 | Apple Inc. | Intelligent text-to-speech conversion |
| US9633674B2 (en) | 2013-06-07 | 2017-04-25 | Apple Inc. | System and method for detecting errors in interactions with a voice-based digital assistant |
| US9633660B2 (en) | 2010-02-25 | 2017-04-25 | Apple Inc. | User profiling for voice input processing |
| US9633004B2 (en) | 2014-05-30 | 2017-04-25 | Apple Inc. | Better resolution when referencing to concepts |
| JP2017511947A (en) * | 2015-01-23 | 2017-04-27 | 小米科技有限責任公司Ｘｉａｏｍｉ Ｉｎｃ． | Button interaction method, apparatus, program, and recording medium |
| US9646609B2 (en) | 2014-09-30 | 2017-05-09 | Apple Inc. | Caching apparatus for serving phonetic pronunciations |
| US9646614B2 (en) | 2000-03-16 | 2017-05-09 | Apple Inc. | Fast, language-independent method for user authentication by voice |
| US9668121B2 (en) | 2014-09-30 | 2017-05-30 | Apple Inc. | Social reminders |
| JP2017516163A (en) * | 2015-03-08 | 2017-06-15 | アップル インコーポレイテッド | Device, method and graphical user interface for manipulating user interface objects using visual and / or tactile feedback |
| US9697822B1 (en) | 2013-03-15 | 2017-07-04 | Apple Inc. | System and method for updating an adaptive speech recognition model |
| US9697820B2 (en) | 2015-09-24 | 2017-07-04 | Apple Inc. | Unit-selection text-to-speech synthesis using concatenation-sensitive neural networks |
| US9706127B2 (en) | 2015-06-07 | 2017-07-11 | Apple Inc. | Devices and methods for capturing and interacting with enhanced digital images |
| US9711141B2 (en) | 2014-12-09 | 2017-07-18 | Apple Inc. | Disambiguating heteronyms in speech synthesis |
| US9715875B2 (en) | 2014-05-30 | 2017-07-25 | Apple Inc. | Reducing the need for manual start/end-pointing and trigger phrases |
| US9721566B2 (en) | 2015-03-08 | 2017-08-01 | Apple Inc. | Competing devices responding to voice triggers |
| US9734193B2 (en) | 2014-05-30 | 2017-08-15 | Apple Inc. | Determining domain salience ranking from ambiguous words in natural speech |
| US9753639B2 (en) | 2012-05-09 | 2017-09-05 | Apple Inc. | Device, method, and graphical user interface for displaying content associated with a corresponding affordance |
| US9760559B2 (en) | 2014-05-30 | 2017-09-12 | Apple Inc. | Predictive text input |
| US9778771B2 (en) | 2012-12-29 | 2017-10-03 | Apple Inc. | Device, method, and graphical user interface for transitioning between touch input to display output relationships |
| US9785305B2 (en) | 2015-03-19 | 2017-10-10 | Apple Inc. | Touch input cursor manipulation |
| US9785630B2 (en) | 2014-05-30 | 2017-10-10 | Apple Inc. | Text prediction using combined word N-gram and unigram language models |
| US9798393B2 (en) | 2011-08-29 | 2017-10-24 | Apple Inc. | Text correction processing |
| US9818400B2 (en) | 2014-09-11 | 2017-11-14 | Apple Inc. | Method and apparatus for discovering trending terms in speech requests |
| US9823839B2 (en) | 2012-05-09 | 2017-11-21 | Apple Inc. | Device, method, and graphical user interface for displaying additional information in response to a user contact |
| US9830048B2 (en) | 2015-06-07 | 2017-11-28 | Apple Inc. | Devices and methods for processing touch inputs with instructions in a web page |
| US9842101B2 (en) | 2014-05-30 | 2017-12-12 | Apple Inc. | Predictive conversion of language input |
| US9842105B2 (en) | 2015-04-16 | 2017-12-12 | Apple Inc. | Parsimonious continuous-space phrase representations for natural language processing |
| US9858925B2 (en) | 2009-06-05 | 2018-01-02 | Apple Inc. | Using context information to facilitate processing of commands in a virtual assistant |
| US9865280B2 (en) | 2015-03-06 | 2018-01-09 | Apple Inc. | Structured dictation using intelligent automated assistants |
| US9880735B2 (en) | 2015-08-10 | 2018-01-30 | Apple Inc. | Devices, methods, and graphical user interfaces for manipulating user interface objects with visual and/or haptic feedback |
| US9886184B2 (en) | 2012-05-09 | 2018-02-06 | Apple Inc. | Device, method, and graphical user interface for providing feedback for changing activation states of a user interface object |
| US9886432B2 (en) | 2014-09-30 | 2018-02-06 | Apple Inc. | Parsimonious handling of word inflection via categorical stem + suffix N-gram language models |
| US9886953B2 (en) | 2015-03-08 | 2018-02-06 | Apple Inc. | Virtual assistant activation |
| US9891811B2 (en) | 2015-06-07 | 2018-02-13 | Apple Inc. | Devices and methods for navigating between user interfaces |
| US9899019B2 (en) | 2015-03-18 | 2018-02-20 | Apple Inc. | Systems and methods for structured stem and suffix language models |
| US9922642B2 (en) | 2013-03-15 | 2018-03-20 | Apple Inc. | Training an at least partial voice command system |
| US9934775B2 (en) | 2016-05-26 | 2018-04-03 | Apple Inc. | Unit-selection text-to-speech synthesis based on predicted concatenation parameters |
| US9953088B2 (en) | 2012-05-14 | 2018-04-24 | Apple Inc. | Crowd sourcing information to fulfill user requests |
| US9959870B2 (en) | 2008-12-11 | 2018-05-01 | Apple Inc. | Speech recognition involving a mobile device |
| US9959025B2 (en) | 2012-12-29 | 2018-05-01 | Apple Inc. | Device, method, and graphical user interface for navigating user interface hierarchies |
| US9966068B2 (en) | 2013-06-08 | 2018-05-08 | Apple Inc. | Interpreting and acting upon commands that involve sharing information with remote devices |
| US9966065B2 (en) | 2014-05-30 | 2018-05-08 | Apple Inc. | Multi-command single utterance input method |
| US9971774B2 (en) | 2012-09-19 | 2018-05-15 | Apple Inc. | Voice-based media searching |
| US9972304B2 (en) | 2016-06-03 | 2018-05-15 | Apple Inc. | Privacy preserving distributed evaluation framework for embedded personalized systems |
| US9990121B2 (en) | 2012-05-09 | 2018-06-05 | Apple Inc. | Device, method, and graphical user interface for moving a user interface object based on an intensity of a press input |
| US9990107B2 (en) | 2015-03-08 | 2018-06-05 | Apple Inc. | Devices, methods, and graphical user interfaces for displaying and using menus |
| US9996231B2 (en) | 2012-05-09 | 2018-06-12 | Apple Inc. | Device, method, and graphical user interface for manipulating framed graphical objects |
| US10037138B2 (en) | 2012-12-29 | 2018-07-31 | Apple Inc. | Device, method, and graphical user interface for switching between user interfaces |
| US10042542B2 (en) | 2012-05-09 | 2018-08-07 | Apple Inc. | Device, method, and graphical user interface for moving and dropping a user interface object |
| US10049663B2 (en) | 2016-06-08 | 2018-08-14 | Apple, Inc. | Intelligent automated assistant for media exploration |
| US10049668B2 (en) | 2015-12-02 | 2018-08-14 | Apple Inc. | Applying neural network language models to weighted finite state transducers for automatic speech recognition |
| US10048757B2 (en) | 2015-03-08 | 2018-08-14 | Apple Inc. | Devices and methods for controlling media presentation |
| US10057736B2 (en) | 2011-06-03 | 2018-08-21 | Apple Inc. | Active transport based notifications |
| US10067938B2 (en) | 2016-06-10 | 2018-09-04 | Apple Inc. | Multilingual word prediction |
| US10067653B2 (en) | 2015-04-01 | 2018-09-04 | Apple Inc. | Devices and methods for processing touch inputs based on their intensities |
| US10073615B2 (en) | 2012-05-09 | 2018-09-11 | Apple Inc. | Device, method, and graphical user interface for displaying user interface objects corresponding to an application |
| US10074360B2 (en) | 2014-09-30 | 2018-09-11 | Apple Inc. | Providing an indication of the suitability of speech recognition |
| US10079014B2 (en) | 2012-06-08 | 2018-09-18 | Apple Inc. | Name recognition system |
| US10078442B2 (en) | 2012-12-29 | 2018-09-18 | Apple Inc. | Device, method, and graphical user interface for determining whether to scroll or select content based on an intensity theshold |
| US10078631B2 (en) | 2014-05-30 | 2018-09-18 | Apple Inc. | Entropy-guided text prediction using combined word and character n-gram language models |
| US10083688B2 (en) | 2015-05-27 | 2018-09-25 | Apple Inc. | Device voice control for selecting a displayed affordance |
| US10089072B2 (en) | 2016-06-11 | 2018-10-02 | Apple Inc. | Intelligent device arbitration and control |
| US10095396B2 (en) | 2015-03-08 | 2018-10-09 | Apple Inc. | Devices, methods, and graphical user interfaces for interacting with a control object while dragging another object |
| US10095391B2 (en) | 2012-05-09 | 2018-10-09 | Apple Inc. | Device, method, and graphical user interface for selecting user interface objects |
| US10101822B2 (en) | 2015-06-05 | 2018-10-16 | Apple Inc. | Language input correction |
| US10127220B2 (en) | 2015-06-04 | 2018-11-13 | Apple Inc. | Language identification from short strings |
| US10126930B2 (en) | 2012-05-09 | 2018-11-13 | Apple Inc. | Device, method, and graphical user interface for scrolling nested regions |
| US10127911B2 (en) | 2014-09-30 | 2018-11-13 | Apple Inc. | Speaker identification and unsupervised speaker adaptation techniques |
| US10134385B2 (en) | 2012-03-02 | 2018-11-20 | Apple Inc. | Systems and methods for name pronunciation |
| US10162452B2 (en) | 2015-08-10 | 2018-12-25 | Apple Inc. | Devices and methods for processing touch inputs based on their intensities |
| US10170123B2 (en) | 2014-05-30 | 2019-01-01 | Apple Inc. | Intelligent assistant for home automation |
| US10176167B2 (en) | 2013-06-09 | 2019-01-08 | Apple Inc. | System and method for inferring user intent from speech inputs |
| US10175864B2 (en) | 2012-05-09 | 2019-01-08 | Apple Inc. | Device, method, and graphical user interface for selecting object within a group of objects in accordance with contact intensity |
| US10175757B2 (en) | 2012-05-09 | 2019-01-08 | Apple Inc. | Device, method, and graphical user interface for providing tactile feedback for touch-based operations performed and reversed in a user interface |
| US10186254B2 (en) | 2015-06-07 | 2019-01-22 | Apple Inc. | Context-based endpoint detection |
| US10185542B2 (en) | 2013-06-09 | 2019-01-22 | Apple Inc. | Device, method, and graphical user interface for enabling conversation persistence across two or more instances of a digital assistant |
| US10192552B2 (en) | 2016-06-10 | 2019-01-29 | Apple Inc. | Digital assistant providing whispered speech |
| US10199051B2 (en) | 2013-02-07 | 2019-02-05 | Apple Inc. | Voice trigger for a digital assistant |
| US10200598B2 (en) | 2015-06-07 | 2019-02-05 | Apple Inc. | Devices and methods for capturing and interacting with enhanced digital images |
| US10222980B2 (en) | 2015-03-19 | 2019-03-05 | Apple Inc. | Touch input cursor manipulation |
| US10223066B2 (en) | 2015-12-23 | 2019-03-05 | Apple Inc. | Proactive assistance based on dialog communication between devices |
| US10235035B2 (en) | 2015-08-10 | 2019-03-19 | Apple Inc. | Devices, methods, and graphical user interfaces for content navigation and manipulation |
| US10241752B2 (en) | 2011-09-30 | 2019-03-26 | Apple Inc. | Interface for a virtual digital assistant |
| US10241644B2 (en) | 2011-06-03 | 2019-03-26 | Apple Inc. | Actionable reminder entries |
| US10248308B2 (en) | 2015-08-10 | 2019-04-02 | Apple Inc. | Devices, methods, and graphical user interfaces for manipulating user interfaces with physical gestures |
| US10249300B2 (en) | 2016-06-06 | 2019-04-02 | Apple Inc. | Intelligent list reading |
| US10255907B2 (en) | 2015-06-07 | 2019-04-09 | Apple Inc. | Automatic accent detection using acoustic models |
| US10269345B2 (en) | 2016-06-11 | 2019-04-23 | Apple Inc. | Intelligent task discovery |
| US10275087B1 (en) | 2011-08-05 | 2019-04-30 | P4tents1, LLC | Devices, methods, and graphical user interfaces for manipulating user interface objects with visual and/or haptic feedback |
| US10276170B2 (en) | 2010-01-18 | 2019-04-30 | Apple Inc. | Intelligent automated assistant |
| US10283110B2 (en) | 2009-07-02 | 2019-05-07 | Apple Inc. | Methods and apparatuses for automatic speech recognition |
| US10289433B2 (en) | 2014-05-30 | 2019-05-14 | Apple Inc. | Domain specific language for encoding assistant dialog |
| US10297253B2 (en) | 2016-06-11 | 2019-05-21 | Apple Inc. | Application integration with a digital assistant |
| US10318871B2 (en) | 2005-09-08 | 2019-06-11 | Apple Inc. | Method and apparatus for building an intelligent automated assistant |
| US10346030B2 (en) | 2015-06-07 | 2019-07-09 | Apple Inc. | Devices and methods for navigating between user interfaces |
| US10354011B2 (en) | 2016-06-09 | 2019-07-16 | Apple Inc. | Intelligent automated assistant in a home environment |
| US10366158B2 (en) | 2015-09-29 | 2019-07-30 | Apple Inc. | Efficient word encoding for recurrent neural network language models |
| US10387029B2 (en) | 2015-03-08 | 2019-08-20 | Apple Inc. | Devices, methods, and graphical user interfaces for displaying and using menus |
| US10416800B2 (en) | 2015-08-10 | 2019-09-17 | Apple Inc. | Devices, methods, and graphical user interfaces for adjusting user interface objects |
| US10437333B2 (en) | 2012-12-29 | 2019-10-08 | Apple Inc. | Device, method, and graphical user interface for forgoing generation of tactile output for a multi-contact gesture |
| US10446143B2 (en) | 2016-03-14 | 2019-10-15 | Apple Inc. | Identification of voice inputs providing credentials |
| US10446141B2 (en) | 2014-08-28 | 2019-10-15 | Apple Inc. | Automatic speech recognition based on user feedback |
| US10490187B2 (en) | 2016-06-10 | 2019-11-26 | Apple Inc. | Digital assistant providing automated status report |
| US10496753B2 (en) | 2010-01-18 | 2019-12-03 | Apple Inc. | Automatically adapting user interfaces for hands-free interaction |
| US10496260B2 (en) | 2012-05-09 | 2019-12-03 | Apple Inc. | Device, method, and graphical user interface for pressure-based alteration of controls in a user interface |
| US10509862B2 (en) | 2016-06-10 | 2019-12-17 | Apple Inc. | Dynamic phrase expansion of language input |
| US10521466B2 (en) | 2016-06-11 | 2019-12-31 | Apple Inc. | Data driven natural language event detection and classification |
| US10552013B2 (en) | 2014-12-02 | 2020-02-04 | Apple Inc. | Data detection |
| US10553209B2 (en) | 2010-01-18 | 2020-02-04 | Apple Inc. | Systems and methods for hands-free notification summaries |
| US10567477B2 (en) | 2015-03-08 | 2020-02-18 | Apple Inc. | Virtual assistant continuity |
| US10568032B2 (en) | 2007-04-03 | 2020-02-18 | Apple Inc. | Method and system for operating a multi-function portable electronic device using voice-activation |
| US10592095B2 (en) | 2014-05-23 | 2020-03-17 | Apple Inc. | Instantaneous speaking of content on touch devices |
| US10593346B2 (en) | 2016-12-22 | 2020-03-17 | Apple Inc. | Rank-reduced token representation for automatic speech recognition |
| US10592041B2 (en) | 2012-05-09 | 2020-03-17 | Apple Inc. | Device, method, and graphical user interface for transitioning between display states in response to a gesture |
| US10620781B2 (en) | 2012-12-29 | 2020-04-14 | Apple Inc. | Device, method, and graphical user interface for moving a cursor according to a change in an appearance of a control icon with simulated three-dimensional characteristics |
| US10659851B2 (en) | 2014-06-30 | 2020-05-19 | Apple Inc. | Real-time digital assistant knowledge updates |
| US10664144B2 (en) | 2011-05-31 | 2020-05-26 | Apple Inc. | Devices, methods, and graphical user interfaces for document manipulation |
| US10671428B2 (en) | 2015-09-08 | 2020-06-02 | Apple Inc. | Distributed personal assistant |
| US10679605B2 (en) | 2010-01-18 | 2020-06-09 | Apple Inc. | Hands-free list-reading by intelligent automated assistant |
| US10691473B2 (en) | 2015-11-06 | 2020-06-23 | Apple Inc. | Intelligent automated assistant in a messaging environment |
| US10706373B2 (en) | 2011-06-03 | 2020-07-07 | Apple Inc. | Performing actions associated with task items that represent tasks to perform |
| US10705794B2 (en) | 2010-01-18 | 2020-07-07 | Apple Inc. | Automatically adapting user interfaces for hands-free interaction |
| US10733993B2 (en) | 2016-06-10 | 2020-08-04 | Apple Inc. | Intelligent digital assistant in a multi-tasking environment |
| US10747498B2 (en) | 2015-09-08 | 2020-08-18 | Apple Inc. | Zero latency digital assistant |
| US10762293B2 (en) | 2010-12-22 | 2020-09-01 | Apple Inc. | Using parts-of-speech tagging and named entity recognition for spelling correction |
| US10791216B2 (en) | 2013-08-06 | 2020-09-29 | Apple Inc. | Auto-activating smart responses based on activities from remote devices |
| US10791176B2 (en) | 2017-05-12 | 2020-09-29 | Apple Inc. | Synchronization and task delegation of a digital assistant |
| US10789041B2 (en) | 2014-09-12 | 2020-09-29 | Apple Inc. | Dynamic thresholds for always listening speech trigger |
| US10810274B2 (en) | 2017-05-15 | 2020-10-20 | Apple Inc. | Optimizing dialogue policy decisions for digital assistants using implicit feedback |
| US11010550B2 (en) | 2015-09-29 | 2021-05-18 | Apple Inc. | Unified language modeling framework for word prediction, auto-completion and auto-correction |
| US11025565B2 (en) | 2015-06-07 | 2021-06-01 | Apple Inc. | Personalized prediction of responses for instant messaging |
| US11587559B2 (en) | 2015-09-30 | 2023-02-21 | Apple Inc. | Intelligent device identification |
-
1995
- 1995-02-22 JP JP3323295A patent/JPH08227341A/en active Pending
Cited By (289)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2000259311A (en) * | 1999-03-04 | 2000-09-22 | Fujitsu Takamisawa Component Ltd | Processor and method for information processing |
| JP2001051759A (en) * | 1999-08-12 | 2001-02-23 | Nec Corp | Operation reaction sound generating device and recording medium where operation reaction sound generating program is recorded |
| US9684436B2 (en) | 1999-12-20 | 2017-06-20 | Apple Inc. | User interface for providing consolidation and access |
| US8032843B2 (en) | 1999-12-20 | 2011-10-04 | Apple Inc. | User interface for providing consolidation and access |
| US8799813B2 (en) | 2000-01-05 | 2014-08-05 | Apple Inc. | Method and system for providing an embedded application tool bar |
| US9646614B2 (en) | 2000-03-16 | 2017-05-09 | Apple Inc. | Fast, language-independent method for user authentication by voice |
| WO2005101172A1 (en) * | 2004-03-31 | 2005-10-27 | Sharp Kabushiki Kaisha | Electronic device |
| US8427445B2 (en) | 2004-07-30 | 2013-04-23 | Apple Inc. | Visual expander |
| US8744852B1 (en) | 2004-10-01 | 2014-06-03 | Apple Inc. | Spoken interfaces |
| JP2007060033A (en) * | 2005-08-22 | 2007-03-08 | Ricoh Co Ltd | Multifunction image forming apparatus, screen display mode setting method, program and recording medium |
| US10318871B2 (en) | 2005-09-08 | 2019-06-11 | Apple Inc. | Method and apparatus for building an intelligent automated assistant |
| JP2007334747A (en) * | 2006-06-16 | 2007-12-27 | Miyake Design Jimusho:Kk | Information processor, portable telephone set, contact input picture forming apparatus, information processing program, and recording medium recording information processing program |
| US8930191B2 (en) | 2006-09-08 | 2015-01-06 | Apple Inc. | Paraphrasing of user requests and results by automated digital assistant |
| US8942986B2 (en) | 2006-09-08 | 2015-01-27 | Apple Inc. | Determining user intent based on ontologies of domains |
| US9117447B2 (en) | 2006-09-08 | 2015-08-25 | Apple Inc. | Using event alert text as input to an automated assistant |
| US9632695B2 (en) | 2006-10-26 | 2017-04-25 | Apple Inc. | Portable multifunction device, method, and graphical user interface for adjusting an insertion point marker |
| US9207855B2 (en) | 2006-10-26 | 2015-12-08 | Apple Inc. | Portable multifunction device, method, and graphical user interface for adjusting an insertion point marker |
| JP2008211383A (en) * | 2007-02-23 | 2008-09-11 | Matsushita Electric Works Ltd | Remote monitoring system |
| US10568032B2 (en) | 2007-04-03 | 2020-02-18 | Apple Inc. | Method and system for operating a multi-function portable electronic device using voice-activation |
| US9330720B2 (en) | 2008-01-03 | 2016-05-03 | Apple Inc. | Methods and apparatus for altering audio output signals |
| US10381016B2 (en) | 2008-01-03 | 2019-08-13 | Apple Inc. | Methods and apparatus for altering audio output signals |
| US9529524B2 (en) | 2008-03-04 | 2016-12-27 | Apple Inc. | Methods and graphical user interfaces for editing on a portable multifunction device |
| US9865248B2 (en) | 2008-04-05 | 2018-01-09 | Apple Inc. | Intelligent text-to-speech conversion |
| US9626955B2 (en) | 2008-04-05 | 2017-04-18 | Apple Inc. | Intelligent text-to-speech conversion |
| US9535906B2 (en) | 2008-07-31 | 2017-01-03 | Apple Inc. | Mobile device having human language translation capability with positional feedback |
| US10108612B2 (en) | 2008-07-31 | 2018-10-23 | Apple Inc. | Mobile device having human language translation capability with positional feedback |
| US9959870B2 (en) | 2008-12-11 | 2018-05-01 | Apple Inc. | Speech recognition involving a mobile device |
| US10761716B2 (en) | 2009-03-16 | 2020-09-01 | Apple, Inc. | Methods and graphical user interfaces for editing on a multifunction device with a touch screen display |
| US8370736B2 (en) | 2009-03-16 | 2013-02-05 | Apple Inc. | Methods and graphical user interfaces for editing on a multifunction device with a touch screen display |
| US9846533B2 (en) | 2009-03-16 | 2017-12-19 | Apple Inc. | Methods and graphical user interfaces for editing on a multifunction device with a touch screen display |
| US8255830B2 (en) | 2009-03-16 | 2012-08-28 | Apple Inc. | Methods and graphical user interfaces for editing on a multifunction device with a touch screen display |
| US9875013B2 (en) | 2009-03-16 | 2018-01-23 | Apple Inc. | Methods and graphical user interfaces for editing on a multifunction device with a touch screen display |
| US9858925B2 (en) | 2009-06-05 | 2018-01-02 | Apple Inc. | Using context information to facilitate processing of commands in a virtual assistant |
| US10475446B2 (en) | 2009-06-05 | 2019-11-12 | Apple Inc. | Using context information to facilitate processing of commands in a virtual assistant |
| US10795541B2 (en) | 2009-06-05 | 2020-10-06 | Apple Inc. | Intelligent organization of tasks items |
| US11080012B2 (en) | 2009-06-05 | 2021-08-03 | Apple Inc. | Interface for a virtual digital assistant |
| US10283110B2 (en) | 2009-07-02 | 2019-05-07 | Apple Inc. | Methods and apparatuses for automatic speech recognition |
| JP2011118595A (en) * | 2009-12-02 | 2011-06-16 | Square Enix Co Ltd | Apparatus, method and program for processing user interface |
| US8892446B2 (en) | 2010-01-18 | 2014-11-18 | Apple Inc. | Service orchestration for intelligent automated assistant |
| US9318108B2 (en) | 2010-01-18 | 2016-04-19 | Apple Inc. | Intelligent automated assistant |
| US8903716B2 (en) | 2010-01-18 | 2014-12-02 | Apple Inc. | Personalized vocabulary for digital assistant |
| US10276170B2 (en) | 2010-01-18 | 2019-04-30 | Apple Inc. | Intelligent automated assistant |
| US10553209B2 (en) | 2010-01-18 | 2020-02-04 | Apple Inc. | Systems and methods for hands-free notification summaries |
| US10496753B2 (en) | 2010-01-18 | 2019-12-03 | Apple Inc. | Automatically adapting user interfaces for hands-free interaction |
| US10679605B2 (en) | 2010-01-18 | 2020-06-09 | Apple Inc. | Hands-free list-reading by intelligent automated assistant |
| US10706841B2 (en) | 2010-01-18 | 2020-07-07 | Apple Inc. | Task flow identification based on user intent |
| US10705794B2 (en) | 2010-01-18 | 2020-07-07 | Apple Inc. | Automatically adapting user interfaces for hands-free interaction |
| US11423886B2 (en) | 2010-01-18 | 2022-08-23 | Apple Inc. | Task flow identification based on user intent |
| US9548050B2 (en) | 2010-01-18 | 2017-01-17 | Apple Inc. | Intelligent automated assistant |
| US9424861B2 (en) | 2010-01-25 | 2016-08-23 | Newvaluexchange Ltd | Apparatuses, methods and systems for a digital conversation management platform |
| US11410053B2 (en) | 2010-01-25 | 2022-08-09 | Newvaluexchange Ltd. | Apparatuses, methods and systems for a digital conversation management platform |
| US9424862B2 (en) | 2010-01-25 | 2016-08-23 | Newvaluexchange Ltd | Apparatuses, methods and systems for a digital conversation management platform |
| US9431028B2 (en) | 2010-01-25 | 2016-08-30 | Newvaluexchange Ltd | Apparatuses, methods and systems for a digital conversation management platform |
| US10607140B2 (en) | 2010-01-25 | 2020-03-31 | Newvaluexchange Ltd. | Apparatuses, methods and systems for a digital conversation management platform |
| US8977584B2 (en) | 2010-01-25 | 2015-03-10 | Newvaluexchange Global Ai Llp | Apparatuses, methods and systems for a digital conversation management platform |
| US10607141B2 (en) | 2010-01-25 | 2020-03-31 | Newvaluexchange Ltd. | Apparatuses, methods and systems for a digital conversation management platform |
| US10984327B2 (en) | 2010-01-25 | 2021-04-20 | New Valuexchange Ltd. | Apparatuses, methods and systems for a digital conversation management platform |
| US10984326B2 (en) | 2010-01-25 | 2021-04-20 | Newvaluexchange Ltd. | Apparatuses, methods and systems for a digital conversation management platform |
| US10049675B2 (en) | 2010-02-25 | 2018-08-14 | Apple Inc. | User profiling for voice input processing |
| US9633660B2 (en) | 2010-02-25 | 2017-04-25 | Apple Inc. | User profiling for voice input processing |
| US10762293B2 (en) | 2010-12-22 | 2020-09-01 | Apple Inc. | Using parts-of-speech tagging and named entity recognition for spelling correction |
| US10102359B2 (en) | 2011-03-21 | 2018-10-16 | Apple Inc. | Device access using voice authentication |
| US9262612B2 (en) | 2011-03-21 | 2016-02-16 | Apple Inc. | Device access using voice authentication |
| US10664144B2 (en) | 2011-05-31 | 2020-05-26 | Apple Inc. | Devices, methods, and graphical user interfaces for document manipulation |
| US11256401B2 (en) | 2011-05-31 | 2022-02-22 | Apple Inc. | Devices, methods, and graphical user interfaces for document manipulation |
| US10057736B2 (en) | 2011-06-03 | 2018-08-21 | Apple Inc. | Active transport based notifications |
| US10241644B2 (en) | 2011-06-03 | 2019-03-26 | Apple Inc. | Actionable reminder entries |
| US10706373B2 (en) | 2011-06-03 | 2020-07-07 | Apple Inc. | Performing actions associated with task items that represent tasks to perform |
| US11120372B2 (en) | 2011-06-03 | 2021-09-14 | Apple Inc. | Performing actions associated with task items that represent tasks to perform |
| US10275087B1 (en) | 2011-08-05 | 2019-04-30 | P4tents1, LLC | Devices, methods, and graphical user interfaces for manipulating user interface objects with visual and/or haptic feedback |
| US10386960B1 (en) | 2011-08-05 | 2019-08-20 | P4tents1, LLC | Devices, methods, and graphical user interfaces for manipulating user interface objects with visual and/or haptic feedback |
| US10540039B1 (en) | 2011-08-05 | 2020-01-21 | P4tents1, LLC | Devices and methods for navigating between user interface |
| US10345961B1 (en) | 2011-08-05 | 2019-07-09 | P4tents1, LLC | Devices and methods for navigating between user interfaces |
| US10649571B1 (en) | 2011-08-05 | 2020-05-12 | P4tents1, LLC | Devices, methods, and graphical user interfaces for manipulating user interface objects with visual and/or haptic feedback |
| US10338736B1 (en) | 2011-08-05 | 2019-07-02 | P4tents1, LLC | Devices, methods, and graphical user interfaces for manipulating user interface objects with visual and/or haptic feedback |
| US10656752B1 (en) | 2011-08-05 | 2020-05-19 | P4tents1, LLC | Gesture-equipped touch screen system, method, and computer program product |
| US10664097B1 (en) | 2011-08-05 | 2020-05-26 | P4tents1, LLC | Devices, methods, and graphical user interfaces for manipulating user interface objects with visual and/or haptic feedback |
| US10365758B1 (en) | 2011-08-05 | 2019-07-30 | P4tents1, LLC | Devices, methods, and graphical user interfaces for manipulating user interface objects with visual and/or haptic feedback |
| US9798393B2 (en) | 2011-08-29 | 2017-10-24 | Apple Inc. | Text correction processing |
| US10241752B2 (en) | 2011-09-30 | 2019-03-26 | Apple Inc. | Interface for a virtual digital assistant |
| US10134385B2 (en) | 2012-03-02 | 2018-11-20 | Apple Inc. | Systems and methods for name pronunciation |
| US9483461B2 (en) | 2012-03-06 | 2016-11-01 | Apple Inc. | Handling speech synthesis of content for multiple languages |
| US11314407B2 (en) | 2012-05-09 | 2022-04-26 | Apple Inc. | Device, method, and graphical user interface for providing feedback for changing activation states of a user interface object |
| US10496260B2 (en) | 2012-05-09 | 2019-12-03 | Apple Inc. | Device, method, and graphical user interface for pressure-based alteration of controls in a user interface |
| US10191627B2 (en) | 2012-05-09 | 2019-01-29 | Apple Inc. | Device, method, and graphical user interface for manipulating framed graphical objects |
| US10969945B2 (en) | 2012-05-09 | 2021-04-06 | Apple Inc. | Device, method, and graphical user interface for selecting user interface objects |
| US10942570B2 (en) | 2012-05-09 | 2021-03-09 | Apple Inc. | Device, method, and graphical user interface for providing tactile feedback for operations performed in a user interface |
| US10175757B2 (en) | 2012-05-09 | 2019-01-08 | Apple Inc. | Device, method, and graphical user interface for providing tactile feedback for touch-based operations performed and reversed in a user interface |
| US11068153B2 (en) | 2012-05-09 | 2021-07-20 | Apple Inc. | Device, method, and graphical user interface for displaying user interface objects corresponding to an application |
| US10175864B2 (en) | 2012-05-09 | 2019-01-08 | Apple Inc. | Device, method, and graphical user interface for selecting object within a group of objects in accordance with contact intensity |
| US9753639B2 (en) | 2012-05-09 | 2017-09-05 | Apple Inc. | Device, method, and graphical user interface for displaying content associated with a corresponding affordance |
| US11947724B2 (en) | 2012-05-09 | 2024-04-02 | Apple Inc. | Device, method, and graphical user interface for providing tactile feedback for operations performed in a user interface |
| US10908808B2 (en) | 2012-05-09 | 2021-02-02 | Apple Inc. | Device, method, and graphical user interface for displaying additional information in response to a user contact |
| US10884591B2 (en) | 2012-05-09 | 2021-01-05 | Apple Inc. | Device, method, and graphical user interface for selecting object within a group of objects |
| US10126930B2 (en) | 2012-05-09 | 2018-11-13 | Apple Inc. | Device, method, and graphical user interface for scrolling nested regions |
| US11023116B2 (en) | 2012-05-09 | 2021-06-01 | Apple Inc. | Device, method, and graphical user interface for moving a user interface object based on an intensity of a press input |
| US10996788B2 (en) | 2012-05-09 | 2021-05-04 | Apple Inc. | Device, method, and graphical user interface for transitioning between display states in response to a gesture |
| US10114546B2 (en) | 2012-05-09 | 2018-10-30 | Apple Inc. | Device, method, and graphical user interface for displaying user interface objects corresponding to an application |
| US9971499B2 (en) | 2012-05-09 | 2018-05-15 | Apple Inc. | Device, method, and graphical user interface for displaying content associated with a corresponding affordance |
| US11010027B2 (en) | 2012-05-09 | 2021-05-18 | Apple Inc. | Device, method, and graphical user interface for manipulating framed graphical objects |
| US9990121B2 (en) | 2012-05-09 | 2018-06-05 | Apple Inc. | Device, method, and graphical user interface for moving a user interface object based on an intensity of a press input |
| US11354033B2 (en) | 2012-05-09 | 2022-06-07 | Apple Inc. | Device, method, and graphical user interface for managing icons in a user interface region |
| US9996231B2 (en) | 2012-05-09 | 2018-06-12 | Apple Inc. | Device, method, and graphical user interface for manipulating framed graphical objects |
| US9823839B2 (en) | 2012-05-09 | 2017-11-21 | Apple Inc. | Device, method, and graphical user interface for displaying additional information in response to a user contact |
| US10095391B2 (en) | 2012-05-09 | 2018-10-09 | Apple Inc. | Device, method, and graphical user interface for selecting user interface objects |
| US10042542B2 (en) | 2012-05-09 | 2018-08-07 | Apple Inc. | Device, method, and graphical user interface for moving and dropping a user interface object |
| US11221675B2 (en) | 2012-05-09 | 2022-01-11 | Apple Inc. | Device, method, and graphical user interface for providing tactile feedback for operations performed in a user interface |
| US10782871B2 (en) | 2012-05-09 | 2020-09-22 | Apple Inc. | Device, method, and graphical user interface for providing feedback for changing activation states of a user interface object |
| US10775999B2 (en) | 2012-05-09 | 2020-09-15 | Apple Inc. | Device, method, and graphical user interface for displaying user interface objects corresponding to an application |
| US10592041B2 (en) | 2012-05-09 | 2020-03-17 | Apple Inc. | Device, method, and graphical user interface for transitioning between display states in response to a gesture |
| US10481690B2 (en) | 2012-05-09 | 2019-11-19 | Apple Inc. | Device, method, and graphical user interface for providing tactile feedback for media adjustment operations performed in a user interface |
| US9886184B2 (en) | 2012-05-09 | 2018-02-06 | Apple Inc. | Device, method, and graphical user interface for providing feedback for changing activation states of a user interface object |
| US10073615B2 (en) | 2012-05-09 | 2018-09-11 | Apple Inc. | Device, method, and graphical user interface for displaying user interface objects corresponding to an application |
| US10775994B2 (en) | 2012-05-09 | 2020-09-15 | Apple Inc. | Device, method, and graphical user interface for moving and dropping a user interface object |
| US9953088B2 (en) | 2012-05-14 | 2018-04-24 | Apple Inc. | Crowd sourcing information to fulfill user requests |
| US10079014B2 (en) | 2012-06-08 | 2018-09-18 | Apple Inc. | Name recognition system |
| US9495129B2 (en) | 2012-06-29 | 2016-11-15 | Apple Inc. | Device, method, and user interface for voice-activated navigation and browsing of a document |
| US9576574B2 (en) | 2012-09-10 | 2017-02-21 | Apple Inc. | Context-sensitive handling of interruptions by intelligent digital assistant |
| US9971774B2 (en) | 2012-09-19 | 2018-05-15 | Apple Inc. | Voice-based media searching |
| US11106292B2 (en) | 2012-11-12 | 2021-08-31 | Sony Corporation | Information processing device, communication system, and information processing method |
| US10276132B2 (en) | 2012-11-12 | 2019-04-30 | Sony Corporation | Information processing for recognition of acceptance of user operation |
| US10679586B2 (en) | 2012-11-12 | 2020-06-09 | Sony Corporation | Information processing device, communication system, and information processing method |
| JP2014096074A (en) * | 2012-11-12 | 2014-05-22 | Sony Corp | Information processing device, communication system, and information processing method |
| US9857897B2 (en) | 2012-12-29 | 2018-01-02 | Apple Inc. | Device and method for assigning respective portions of an aggregate intensity to a plurality of contacts |
| US10185491B2 (en) | 2012-12-29 | 2019-01-22 | Apple Inc. | Device, method, and graphical user interface for determining whether to scroll or enlarge content |
| US10101887B2 (en) | 2012-12-29 | 2018-10-16 | Apple Inc. | Device, method, and graphical user interface for navigating user interface hierarchies |
| US9996233B2 (en) | 2012-12-29 | 2018-06-12 | Apple Inc. | Device, method, and graphical user interface for navigating user interface hierarchies |
| US10620781B2 (en) | 2012-12-29 | 2020-04-14 | Apple Inc. | Device, method, and graphical user interface for moving a cursor according to a change in an appearance of a control icon with simulated three-dimensional characteristics |
| US10037138B2 (en) | 2012-12-29 | 2018-07-31 | Apple Inc. | Device, method, and graphical user interface for switching between user interfaces |
| US10437333B2 (en) | 2012-12-29 | 2019-10-08 | Apple Inc. | Device, method, and graphical user interface for forgoing generation of tactile output for a multi-contact gesture |
| US10078442B2 (en) | 2012-12-29 | 2018-09-18 | Apple Inc. | Device, method, and graphical user interface for determining whether to scroll or select content based on an intensity theshold |
| US10175879B2 (en) | 2012-12-29 | 2019-01-08 | Apple Inc. | Device, method, and graphical user interface for zooming a user interface while performing a drag operation |
| US9965074B2 (en) | 2012-12-29 | 2018-05-08 | Apple Inc. | Device, method, and graphical user interface for transitioning between touch input to display output relationships |
| US9778771B2 (en) | 2012-12-29 | 2017-10-03 | Apple Inc. | Device, method, and graphical user interface for transitioning between touch input to display output relationships |
| US9959025B2 (en) | 2012-12-29 | 2018-05-01 | Apple Inc. | Device, method, and graphical user interface for navigating user interface hierarchies |
| US10915243B2 (en) | 2012-12-29 | 2021-02-09 | Apple Inc. | Device, method, and graphical user interface for adjusting content selection |
| US10199051B2 (en) | 2013-02-07 | 2019-02-05 | Apple Inc. | Voice trigger for a digital assistant |
| US10978090B2 (en) | 2013-02-07 | 2021-04-13 | Apple Inc. | Voice trigger for a digital assistant |
| US9368114B2 (en) | 2013-03-14 | 2016-06-14 | Apple Inc. | Context-sensitive handling of interruptions |
| US9922642B2 (en) | 2013-03-15 | 2018-03-20 | Apple Inc. | Training an at least partial voice command system |
| US9697822B1 (en) | 2013-03-15 | 2017-07-04 | Apple Inc. | System and method for updating an adaptive speech recognition model |
| US9966060B2 (en) | 2013-06-07 | 2018-05-08 | Apple Inc. | System and method for user-specified pronunciation of words for speech synthesis and recognition |
| US9620104B2 (en) | 2013-06-07 | 2017-04-11 | Apple Inc. | System and method for user-specified pronunciation of words for speech synthesis and recognition |
| US9633674B2 (en) | 2013-06-07 | 2017-04-25 | Apple Inc. | System and method for detecting errors in interactions with a voice-based digital assistant |
| US9582608B2 (en) | 2013-06-07 | 2017-02-28 | Apple Inc. | Unified ranking with entropy-weighted information for phrase-based semantic auto-completion |
| US9966068B2 (en) | 2013-06-08 | 2018-05-08 | Apple Inc. | Interpreting and acting upon commands that involve sharing information with remote devices |
| US10657961B2 (en) | 2013-06-08 | 2020-05-19 | Apple Inc. | Interpreting and acting upon commands that involve sharing information with remote devices |
| US10185542B2 (en) | 2013-06-09 | 2019-01-22 | Apple Inc. | Device, method, and graphical user interface for enabling conversation persistence across two or more instances of a digital assistant |
| US10176167B2 (en) | 2013-06-09 | 2019-01-08 | Apple Inc. | System and method for inferring user intent from speech inputs |
| US9300784B2 (en) | 2013-06-13 | 2016-03-29 | Apple Inc. | System and method for emergency calls initiated by voice command |
| US10791216B2 (en) | 2013-08-06 | 2020-09-29 | Apple Inc. | Auto-activating smart responses based on activities from remote devices |
| US9620105B2 (en) | 2014-05-15 | 2017-04-11 | Apple Inc. | Analyzing audio input for efficient speech and music recognition |
| US10592095B2 (en) | 2014-05-23 | 2020-03-17 | Apple Inc. | Instantaneous speaking of content on touch devices |
| US9502031B2 (en) | 2014-05-27 | 2016-11-22 | Apple Inc. | Method for supporting dynamic grammars in WFST-based ASR |
| US9430463B2 (en) | 2014-05-30 | 2016-08-30 | Apple Inc. | Exemplar-based natural language processing |
| US10289433B2 (en) | 2014-05-30 | 2019-05-14 | Apple Inc. | Domain specific language for encoding assistant dialog |
| US10078631B2 (en) | 2014-05-30 | 2018-09-18 | Apple Inc. | Entropy-guided text prediction using combined word and character n-gram language models |
| US11133008B2 (en) | 2014-05-30 | 2021-09-28 | Apple Inc. | Reducing the need for manual start/end-pointing and trigger phrases |
| US11257504B2 (en) | 2014-05-30 | 2022-02-22 | Apple Inc. | Intelligent assistant for home automation |
| US9715875B2 (en) | 2014-05-30 | 2017-07-25 | Apple Inc. | Reducing the need for manual start/end-pointing and trigger phrases |
| US9760559B2 (en) | 2014-05-30 | 2017-09-12 | Apple Inc. | Predictive text input |
| US10083690B2 (en) | 2014-05-30 | 2018-09-25 | Apple Inc. | Better resolution when referencing to concepts |
| US10497365B2 (en) | 2014-05-30 | 2019-12-03 | Apple Inc. | Multi-command single utterance input method |
| US9966065B2 (en) | 2014-05-30 | 2018-05-08 | Apple Inc. | Multi-command single utterance input method |
| US10169329B2 (en) | 2014-05-30 | 2019-01-01 | Apple Inc. | Exemplar-based natural language processing |
| US10170123B2 (en) | 2014-05-30 | 2019-01-01 | Apple Inc. | Intelligent assistant for home automation |
| US9633004B2 (en) | 2014-05-30 | 2017-04-25 | Apple Inc. | Better resolution when referencing to concepts |
| US9842101B2 (en) | 2014-05-30 | 2017-12-12 | Apple Inc. | Predictive conversion of language input |
| US9785630B2 (en) | 2014-05-30 | 2017-10-10 | Apple Inc. | Text prediction using combined word N-gram and unigram language models |
| US9734193B2 (en) | 2014-05-30 | 2017-08-15 | Apple Inc. | Determining domain salience ranking from ambiguous words in natural speech |
| EP2960767A1 (en) * | 2014-06-24 | 2015-12-30 | Google, Inc. | Computerized systems and methods for rendering an animation of an object in response to user input |
| US10659851B2 (en) | 2014-06-30 | 2020-05-19 | Apple Inc. | Real-time digital assistant knowledge updates |
| US10904611B2 (en) | 2014-06-30 | 2021-01-26 | Apple Inc. | Intelligent automated assistant for TV user interactions |
| US9338493B2 (en) | 2014-06-30 | 2016-05-10 | Apple Inc. | Intelligent automated assistant for TV user interactions |
| US9668024B2 (en) | 2014-06-30 | 2017-05-30 | Apple Inc. | Intelligent automated assistant for TV user interactions |
| US10446141B2 (en) | 2014-08-28 | 2019-10-15 | Apple Inc. | Automatic speech recognition based on user feedback |
| US10431204B2 (en) | 2014-09-11 | 2019-10-01 | Apple Inc. | Method and apparatus for discovering trending terms in speech requests |
| US9818400B2 (en) | 2014-09-11 | 2017-11-14 | Apple Inc. | Method and apparatus for discovering trending terms in speech requests |
| US10789041B2 (en) | 2014-09-12 | 2020-09-29 | Apple Inc. | Dynamic thresholds for always listening speech trigger |
| US9668121B2 (en) | 2014-09-30 | 2017-05-30 | Apple Inc. | Social reminders |
| US9986419B2 (en) | 2014-09-30 | 2018-05-29 | Apple Inc. | Social reminders |
| US10074360B2 (en) | 2014-09-30 | 2018-09-11 | Apple Inc. | Providing an indication of the suitability of speech recognition |
| US10127911B2 (en) | 2014-09-30 | 2018-11-13 | Apple Inc. | Speaker identification and unsupervised speaker adaptation techniques |
| US9646609B2 (en) | 2014-09-30 | 2017-05-09 | Apple Inc. | Caching apparatus for serving phonetic pronunciations |
| US9886432B2 (en) | 2014-09-30 | 2018-02-06 | Apple Inc. | Parsimonious handling of word inflection via categorical stem + suffix N-gram language models |
| US10552013B2 (en) | 2014-12-02 | 2020-02-04 | Apple Inc. | Data detection |
| US11556230B2 (en) | 2014-12-02 | 2023-01-17 | Apple Inc. | Data detection |
| US9711141B2 (en) | 2014-12-09 | 2017-07-18 | Apple Inc. | Disambiguating heteronyms in speech synthesis |
| US10705676B2 (en) | 2015-01-23 | 2020-07-07 | Xiaomi Inc. | Method and device for interacting with button |
| JP2017511947A (en) * | 2015-01-23 | 2017-04-27 | 小米科技有限責任公司Ｘｉａｏｍｉ Ｉｎｃ． | Button interaction method, apparatus, program, and recording medium |
| US9865280B2 (en) | 2015-03-06 | 2018-01-09 | Apple Inc. | Structured dictation using intelligent automated assistants |
| US10268342B2 (en) | 2015-03-08 | 2019-04-23 | Apple Inc. | Devices, methods, and graphical user interfaces for manipulating user interface objects with visual and/or haptic feedback |
| US10311871B2 (en) | 2015-03-08 | 2019-06-04 | Apple Inc. | Competing devices responding to voice triggers |
| US10387029B2 (en) | 2015-03-08 | 2019-08-20 | Apple Inc. | Devices, methods, and graphical user interfaces for displaying and using menus |
| US10338772B2 (en) | 2015-03-08 | 2019-07-02 | Apple Inc. | Devices, methods, and graphical user interfaces for manipulating user interface objects with visual and/or haptic feedback |
| US9721566B2 (en) | 2015-03-08 | 2017-08-01 | Apple Inc. | Competing devices responding to voice triggers |
| JP2018181355A (en) * | 2015-03-08 | 2018-11-15 | アップル インコーポレイテッドＡｐｐｌｅ Ｉｎｃ． | Devices, methods and graphical user interfaces for manipulating user interface objects with visual and/or haptic feedback |
| JP2017516163A (en) * | 2015-03-08 | 2017-06-15 | アップル インコーポレイテッド | Device, method and graphical user interface for manipulating user interface objects using visual and / or tactile feedback |
| US11087759B2 (en) | 2015-03-08 | 2021-08-10 | Apple Inc. | Virtual assistant activation |
| US9886953B2 (en) | 2015-03-08 | 2018-02-06 | Apple Inc. | Virtual assistant activation |
| US10567477B2 (en) | 2015-03-08 | 2020-02-18 | Apple Inc. | Virtual assistant continuity |
| US10067645B2 (en) | 2015-03-08 | 2018-09-04 | Apple Inc. | Devices, methods, and graphical user interfaces for manipulating user interface objects with visual and/or haptic feedback |
| US10268341B2 (en) | 2015-03-08 | 2019-04-23 | Apple Inc. | Devices, methods, and graphical user interfaces for manipulating user interface objects with visual and/or haptic feedback |
| US10860177B2 (en) | 2015-03-08 | 2020-12-08 | Apple Inc. | Devices, methods, and graphical user interfaces for manipulating user interface objects with visual and/or haptic feedback |
| US10048757B2 (en) | 2015-03-08 | 2018-08-14 | Apple Inc. | Devices and methods for controlling media presentation |
| US10180772B2 (en) | 2015-03-08 | 2019-01-15 | Apple Inc. | Devices, methods, and graphical user interfaces for manipulating user interface objects with visual and/or haptic feedback |
| US10402073B2 (en) | 2015-03-08 | 2019-09-03 | Apple Inc. | Devices, methods, and graphical user interfaces for interacting with a control object while dragging another object |
| US11112957B2 (en) | 2015-03-08 | 2021-09-07 | Apple Inc. | Devices, methods, and graphical user interfaces for interacting with a control object while dragging another object |
| US10613634B2 (en) | 2015-03-08 | 2020-04-07 | Apple Inc. | Devices and methods for controlling media presentation |
| US9990107B2 (en) | 2015-03-08 | 2018-06-05 | Apple Inc. | Devices, methods, and graphical user interfaces for displaying and using menus |
| US10095396B2 (en) | 2015-03-08 | 2018-10-09 | Apple Inc. | Devices, methods, and graphical user interfaces for interacting with a control object while dragging another object |
| US9899019B2 (en) | 2015-03-18 | 2018-02-20 | Apple Inc. | Systems and methods for structured stem and suffix language models |
| US11054990B2 (en) | 2015-03-19 | 2021-07-06 | Apple Inc. | Touch input cursor manipulation |
| US10599331B2 (en) | 2015-03-19 | 2020-03-24 | Apple Inc. | Touch input cursor manipulation |
| US10222980B2 (en) | 2015-03-19 | 2019-03-05 | Apple Inc. | Touch input cursor manipulation |
| US11550471B2 (en) | 2015-03-19 | 2023-01-10 | Apple Inc. | Touch input cursor manipulation |
| US9785305B2 (en) | 2015-03-19 | 2017-10-10 | Apple Inc. | Touch input cursor manipulation |
| US10152208B2 (en) | 2015-04-01 | 2018-12-11 | Apple Inc. | Devices and methods for processing touch inputs based on their intensities |
| US10067653B2 (en) | 2015-04-01 | 2018-09-04 | Apple Inc. | Devices and methods for processing touch inputs based on their intensities |
| US9842105B2 (en) | 2015-04-16 | 2017-12-12 | Apple Inc. | Parsimonious continuous-space phrase representations for natural language processing |
| US10083688B2 (en) | 2015-05-27 | 2018-09-25 | Apple Inc. | Device voice control for selecting a displayed affordance |
| US10127220B2 (en) | 2015-06-04 | 2018-11-13 | Apple Inc. | Language identification from short strings |
| US10101822B2 (en) | 2015-06-05 | 2018-10-16 | Apple Inc. | Language input correction |
| US9891811B2 (en) | 2015-06-07 | 2018-02-13 | Apple Inc. | Devices and methods for navigating between user interfaces |
| US10455146B2 (en) | 2015-06-07 | 2019-10-22 | Apple Inc. | Devices and methods for capturing and interacting with enhanced digital images |
| US10705718B2 (en) | 2015-06-07 | 2020-07-07 | Apple Inc. | Devices and methods for navigating between user interfaces |
| US10200598B2 (en) | 2015-06-07 | 2019-02-05 | Apple Inc. | Devices and methods for capturing and interacting with enhanced digital images |
| US11835985B2 (en) | 2015-06-07 | 2023-12-05 | Apple Inc. | Devices and methods for capturing and interacting with enhanced digital images |
| US11681429B2 (en) | 2015-06-07 | 2023-06-20 | Apple Inc. | Devices and methods for capturing and interacting with enhanced digital images |
| US10186254B2 (en) | 2015-06-07 | 2019-01-22 | Apple Inc. | Context-based endpoint detection |
| US9860451B2 (en) | 2015-06-07 | 2018-01-02 | Apple Inc. | Devices and methods for capturing and interacting with enhanced digital images |
| US11240424B2 (en) | 2015-06-07 | 2022-02-01 | Apple Inc. | Devices and methods for capturing and interacting with enhanced digital images |
| US11231831B2 (en) | 2015-06-07 | 2022-01-25 | Apple Inc. | Devices and methods for content preview based on touch input intensity |
| US9830048B2 (en) | 2015-06-07 | 2017-11-28 | Apple Inc. | Devices and methods for processing touch inputs with instructions in a web page |
| US10255907B2 (en) | 2015-06-07 | 2019-04-09 | Apple Inc. | Automatic accent detection using acoustic models |
| US10303354B2 (en) | 2015-06-07 | 2019-05-28 | Apple Inc. | Devices and methods for navigating between user interfaces |
| US9916080B2 (en) | 2015-06-07 | 2018-03-13 | Apple Inc. | Devices and methods for navigating between user interfaces |
| US10346030B2 (en) | 2015-06-07 | 2019-07-09 | Apple Inc. | Devices and methods for navigating between user interfaces |
| US11025565B2 (en) | 2015-06-07 | 2021-06-01 | Apple Inc. | Personalized prediction of responses for instant messaging |
| US10841484B2 (en) | 2015-06-07 | 2020-11-17 | Apple Inc. | Devices and methods for capturing and interacting with enhanced digital images |
| US9706127B2 (en) | 2015-06-07 | 2017-07-11 | Apple Inc. | Devices and methods for capturing and interacting with enhanced digital images |
| US10248308B2 (en) | 2015-08-10 | 2019-04-02 | Apple Inc. | Devices, methods, and graphical user interfaces for manipulating user interfaces with physical gestures |
| US10884608B2 (en) | 2015-08-10 | 2021-01-05 | Apple Inc. | Devices, methods, and graphical user interfaces for content navigation and manipulation |
| US10416800B2 (en) | 2015-08-10 | 2019-09-17 | Apple Inc. | Devices, methods, and graphical user interfaces for adjusting user interface objects |
| US11740785B2 (en) | 2015-08-10 | 2023-08-29 | Apple Inc. | Devices, methods, and graphical user interfaces for manipulating user interface objects with visual and/or haptic feedback |
| US10754542B2 (en) | 2015-08-10 | 2020-08-25 | Apple Inc. | Devices, methods, and graphical user interfaces for manipulating user interface objects with visual and/or haptic feedback |
| US10203868B2 (en) | 2015-08-10 | 2019-02-12 | Apple Inc. | Devices, methods, and graphical user interfaces for manipulating user interface objects with visual and/or haptic feedback |
| US10963158B2 (en) | 2015-08-10 | 2021-03-30 | Apple Inc. | Devices, methods, and graphical user interfaces for manipulating user interface objects with visual and/or haptic feedback |
| US10209884B2 (en) | 2015-08-10 | 2019-02-19 | Apple Inc. | Devices, Methods, and Graphical User Interfaces for Manipulating User Interface Objects with Visual and/or Haptic Feedback |
| US11182017B2 (en) | 2015-08-10 | 2021-11-23 | Apple Inc. | Devices and methods for processing touch inputs based on their intensities |
| US10162452B2 (en) | 2015-08-10 | 2018-12-25 | Apple Inc. | Devices and methods for processing touch inputs based on their intensities |
| US10235035B2 (en) | 2015-08-10 | 2019-03-19 | Apple Inc. | Devices, methods, and graphical user interfaces for content navigation and manipulation |
| US9880735B2 (en) | 2015-08-10 | 2018-01-30 | Apple Inc. | Devices, methods, and graphical user interfaces for manipulating user interface objects with visual and/or haptic feedback |
| US10698598B2 (en) | 2015-08-10 | 2020-06-30 | Apple Inc. | Devices, methods, and graphical user interfaces for manipulating user interface objects with visual and/or haptic feedback |
| US11327648B2 (en) | 2015-08-10 | 2022-05-10 | Apple Inc. | Devices, methods, and graphical user interfaces for manipulating user interface objects with visual and/or haptic feedback |
| US10671428B2 (en) | 2015-09-08 | 2020-06-02 | Apple Inc. | Distributed personal assistant |
| US11500672B2 (en) | 2015-09-08 | 2022-11-15 | Apple Inc. | Distributed personal assistant |
| US10747498B2 (en) | 2015-09-08 | 2020-08-18 | Apple Inc. | Zero latency digital assistant |
| US9697820B2 (en) | 2015-09-24 | 2017-07-04 | Apple Inc. | Unit-selection text-to-speech synthesis using concatenation-sensitive neural networks |
| US11010550B2 (en) | 2015-09-29 | 2021-05-18 | Apple Inc. | Unified language modeling framework for word prediction, auto-completion and auto-correction |
| US10366158B2 (en) | 2015-09-29 | 2019-07-30 | Apple Inc. | Efficient word encoding for recurrent neural network language models |
| US11587559B2 (en) | 2015-09-30 | 2023-02-21 | Apple Inc. | Intelligent device identification |
| US10691473B2 (en) | 2015-11-06 | 2020-06-23 | Apple Inc. | Intelligent automated assistant in a messaging environment |
| US11526368B2 (en) | 2015-11-06 | 2022-12-13 | Apple Inc. | Intelligent automated assistant in a messaging environment |
| US10049668B2 (en) | 2015-12-02 | 2018-08-14 | Apple Inc. | Applying neural network language models to weighted finite state transducers for automatic speech recognition |
| US10223066B2 (en) | 2015-12-23 | 2019-03-05 | Apple Inc. | Proactive assistance based on dialog communication between devices |
| US10446143B2 (en) | 2016-03-14 | 2019-10-15 | Apple Inc. | Identification of voice inputs providing credentials |
| JP2016167298A (en) * | 2016-04-28 | 2016-09-15 | ソニー株式会社 | Information processing device, information processing method and program |
| US9934775B2 (en) | 2016-05-26 | 2018-04-03 | Apple Inc. | Unit-selection text-to-speech synthesis based on predicted concatenation parameters |
| US9972304B2 (en) | 2016-06-03 | 2018-05-15 | Apple Inc. | Privacy preserving distributed evaluation framework for embedded personalized systems |
| US10249300B2 (en) | 2016-06-06 | 2019-04-02 | Apple Inc. | Intelligent list reading |
| US11069347B2 (en) | 2016-06-08 | 2021-07-20 | Apple Inc. | Intelligent automated assistant for media exploration |
| US10049663B2 (en) | 2016-06-08 | 2018-08-14 | Apple, Inc. | Intelligent automated assistant for media exploration |
| US10354011B2 (en) | 2016-06-09 | 2019-07-16 | Apple Inc. | Intelligent automated assistant in a home environment |
| US10490187B2 (en) | 2016-06-10 | 2019-11-26 | Apple Inc. | Digital assistant providing automated status report |
| US10733993B2 (en) | 2016-06-10 | 2020-08-04 | Apple Inc. | Intelligent digital assistant in a multi-tasking environment |
| US10192552B2 (en) | 2016-06-10 | 2019-01-29 | Apple Inc. | Digital assistant providing whispered speech |
| US10067938B2 (en) | 2016-06-10 | 2018-09-04 | Apple Inc. | Multilingual word prediction |
| US11037565B2 (en) | 2016-06-10 | 2021-06-15 | Apple Inc. | Intelligent digital assistant in a multi-tasking environment |
| US10509862B2 (en) | 2016-06-10 | 2019-12-17 | Apple Inc. | Dynamic phrase expansion of language input |
| US10521466B2 (en) | 2016-06-11 | 2019-12-31 | Apple Inc. | Data driven natural language event detection and classification |
| US10297253B2 (en) | 2016-06-11 | 2019-05-21 | Apple Inc. | Application integration with a digital assistant |
| US10269345B2 (en) | 2016-06-11 | 2019-04-23 | Apple Inc. | Intelligent task discovery |
| US11152002B2 (en) | 2016-06-11 | 2021-10-19 | Apple Inc. | Application integration with a digital assistant |
| US10089072B2 (en) | 2016-06-11 | 2018-10-02 | Apple Inc. | Intelligent device arbitration and control |
| US10593346B2 (en) | 2016-12-22 | 2020-03-17 | Apple Inc. | Rank-reduced token representation for automatic speech recognition |
| US10791176B2 (en) | 2017-05-12 | 2020-09-29 | Apple Inc. | Synchronization and task delegation of a digital assistant |
| US11405466B2 (en) | 2017-05-12 | 2022-08-02 | Apple Inc. | Synchronization and task delegation of a digital assistant |
| US10810274B2 (en) | 2017-05-15 | 2020-10-20 | Apple Inc. | Optimizing dialogue policy decisions for digital assistants using implicit feedback |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JPH08227341A (en) | User interface | |
| CN106997241B (en) | Method for interacting with real world in virtual reality environment and virtual reality system | |
| US7187394B2 (en) | User friendly selection apparatus based on touch screens for visually impaired people | |
| JP6037344B2 (en) | Advanced camera-based input | |
| AU9418701A (en) | Image processing apparatus, image processing method, record medium, computer program, and semiconductor device | |
| JP2004504675A (en) | Pointing direction calibration method in video conferencing and other camera-based system applications | |
| JPH0844490A (en) | Interface device | |
| JP2003067119A (en) | Equipment operating device, program and recording medium | |
| Kajastila et al. | Eyes-free interaction with free-hand gestures and auditory menus | |
| JPH09171569A (en) | Merchandise simulator system and its device | |
| KR20100075281A (en) | Apparatus having function of space projection and space touch and the controlling method thereof | |
| CN108829325A (en) | For dynamically adjusting the equipment, method and graphic user interface of the presentation of audio output | |
| KR20150128386A (en) | display apparatus and method for performing videotelephony using the same | |
| JP2007172303A (en) | Information input system | |
| JP2004164375A (en) | Information processor | |
| JP2002373057A (en) | Method and device for input operation, and program | |
| US10747308B2 (en) | Line-of-sight operation apparatus, method, and medical device | |
| JPH07168691A (en) | Selecting system for display graphic area | |
| CN109144598A (en) | Electronics mask man-machine interaction method and system based on gesture | |
| JP2003233465A (en) | Input device and processing program for detecting button-operation | |
| US20020149563A1 (en) | Joystick control with icon selection function | |
| JPH08194856A (en) | Input device | |
| JP3625549B2 (en) | Multipoint video conferencing system | |
| JPH1165814A (en) | Interactive system and image display method | |
| JP3468371B2 (en) | Window display area control device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| A131 | Notification of reasons for refusal |
Free format text: JAPANESE INTERMEDIATE CODE: A131 Effective date: 20040330 |
|
| A02 | Decision of refusal |
Free format text: JAPANESE INTERMEDIATE CODE: A02 Effective date: 20040727 |