JP4022806B2 - Display device, display method, program, and recording medium - Google Patents

Description

で定義すると、次のような観測方程式が成立する。
【０５７０】
ＸＷ＝Ｙ’・・・（２）
ここで、行列Ｘの成分ｘ_ijは、ｉ件目の生徒データの集合（ｉ件目の教師データｙ_iの予測に用いる生徒データの集合）の中のｊ番目の生徒データを意味し、行列Ｗの成分ｗ_jは、生徒データの集合の中のｊ番目の生徒データとの積が演算されるタップ係数を表す。また、ｙ_iは、ｉ件目の教師データを表し、従って、Ｅ［ｙ_i］は、ｉ件目の教師データの予測値を表す。なお、式（１）の左辺におけるｙは、行列Ｙの成分ｙ_iのサフィックスｉを省略したものであり、また、式（１）の右辺におけるｘ₁，ｘ₂，・・・も、行列Ｘの成分ｘ_ijのサフィックスｉを省略したものである。
【０５７１】
式（２）の観測方程式に最小自乗法を適用して、ＨＤ画素の画素値ｙに近い予測値Ｅ［ｙ］を求めることを考える。この場合、教師データとなるＨＤ画素の真の画素値ｙの集合でなる行列Ｙ、およびＨＤ画素の画素値ｙに対する予測値Ｅ［ｙ］の残差ｅの集合でなる行列Ｅを、
【数２】

で定義すると、式（２）から、次のような残差方程式が成立する。
【０５７２】
ＸＷ＝Ｙ＋Ｅ・・・（３）
【０５７３】
この場合、ＨＤ画素の画素値ｙに近い予測値Ｅ［ｙ］を求めるためのタップ係数ｗ_jは、自乗誤差
【数３】

を最小にすることで求めることができる。
【０５７４】
従って、上述の自乗誤差をタップ係数ｗ_jで微分したものが０になる場合、即ち、次式を満たすタップ係数ｗ_jが、ＨＤ画素の画素値ｙに近い予測値Ｅ［ｙ］を求めるため最適値ということになる。
【０５７５】
【数４】

【０５７６】
そこで、まず、式（３）を、タップ係数ｗ_jで微分することにより、次式が成立する。
【０５７７】
【数５】

【０５７８】
式（４）および（５）より、式（６）が得られる。
【０５７９】
【数６】

【０５８０】
さらに、式（３）の残差方程式における生徒データｘ_ij、タップ係数ｗ_j、教師データｙ_i、および残差ｅ_iの関係を考慮すると、式（６）から、次のような正規方程式を得ることができる。
【０５８１】
【数７】

【０５８２】
なお、式（７）に示した正規方程式は、行列（共分散行列）Ａおよびベクトルｖを、
【数８】

で定義するとともに、ベクトルＷを、数１で示したように定義すると、式
ＡＷ＝ｖ・・・（８）
で表すことができる。
【０５８３】
式（７）における各正規方程式は、生徒データｘ_ijおよび教師データｙ_iのセットを、ある程度の数だけ用意することで、求めるべきタップ係数ｗ_jの数Ｊと同じ数だけたてることができ、従って、式（８）を、ベクトルＷについて解くことで（但し、式（８）を解くには、式（８）における行列Ａが正則である必要がある）、最適なタップ係数ｗ_jを求めることができる。なお、式（８）を解くにあたっては、例えば、掃き出し法（Gauss-Jordanの消去法）などを用いることが可能である。
【０５８４】
以上のように、生徒データと教師データを用いて、生徒データおよびタップ係数から教師データを求めるのに統計的な誤差（ここでは、例えば、自乗誤差）を最小にするタップ係数ｗ_jを求める学習をしておき、さらに、そのタップ係数ｗ_jを用い、式（１）により、教師データｙに近い予測値Ｅ［ｙ］を求めるのが適応処理である。
【０５８５】
なお、適応処理は、ＳＤ画像には含まれていないが、ＨＤ画像に含まれる成分が再現される点で、単なる補間とは異なる。即ち、適応処理では、式（１）だけを見る限りは、いわゆる補間フィルタを用いての単なる補間と同一に見えるが、その補間フィルタのタップ係数に相当するタップ係数ｗが、教師データｙを用いての、いわば学習により求められるため、ＨＤ画像に含まれる成分を再現することができる。このことから、適応処理は、いわば画像の創造（解像度創造）作用がある処理ということができる。
【０５８６】
なお、ここでは、線形一次予測（式（１））による適応処理について説明したが、適応処理では、２次以上の高次の予測演算を用いることも可能である。
【０５８７】
図４８は、以上のような適応処理を行う、図４７の適応処理部４４７の構成例を示している。
【０５８８】
バッファ４５１には、ストレージ４４４に記憶されたＳＤ画像データが、例えば、１フレーム単位で供給されるようになっており、バッファ４５１は、そのＳＤ画像データを一時記憶する。
【０５８９】
予測タップ抽出部４５２は、バッファ４５１に記憶されたＳＤ画像データのフレームのうち、まだ処理していない最も古い（時間的に最も過去のフレーム）フレームを注目フレームとし、さらに、その注目フレームのＳＤ画像データの画質を向上させたＨＤ画像データを構成するＨＤ画素を、順次、注目画素として、その注目画素の画素値を予測するのに用いるＳＤ画素を、バッファ４５１に記憶されたＳＤ画像データから抽出し、後段の積和演算部４５３で用いる予測タップとして出力する。
【０５９０】
即ち、予測タップ抽出部４５２は、注目画素の位置に対応するＳＤ画像データの位置から空間的または時間的に近い位置にある幾つかのＳＤ画素（例えば、水平方向×垂直方向の画素数が５×５のＳＤ画素）を、予測タップとして抽出して出力する。
【０５９１】
積和演算部４５３は、予測タップ抽出部４５２が出力する注目画素についての予測タップと、後述する係数メモリ４５５から供給される注目画素のクラスのタップ係数とを用いて、式（１）の線形一次予測演算を行い、これにより、注目画素の画素値（ＨＤ画素の画素値の予測値）を求めて出力する。
【０５９２】
クラスコード読み出し部４５４は、クラスコードデータベース４４６（図４７）に記憶されたクラスコード画像のうち、注目フレームのＨＤ画像データから生成されたものを、注目クラスコード画像として読み出す。さらに、クラスコード読み出し部４５４は、注目クラスコード画像のクラスコードのうち、注目画素に対応するものを、係数メモリ４５５に対して、アドレスとして与える。
【０５９３】
ここで、図４５の送信処理部４０１で生成されるクラスコード画像には、そのクラスコード画像に対応するＨＤ画像のフレームを識別するフレーム識別情報が付加されるようになっており、クラスコード読み出し部４５４は、注目フレームに対応するクラスコード画像（注目フレームのＨＤ画像データから生成されたクラスコード画像）を、そのフレーム識別情報に基づいて認識する。
【０５９４】
係数メモリ４５５は、学習用のＨＤ画像データを教師データとするとともに、学習用のＳＤ画像データを生徒データとして、ＨＤ画像データのＨＤ画素がクラス分類され得るクラス（以下、適宜、ＨＤクラスという）ごとに、式（８）に示した正規方程式をたてて解くことにより求められたＨＤクラスごとのタップ係数を記憶している。即ち、係数メモリ４５５は、各ＨＤクラスに対応するアドレスに、そのＨＤクラスのタップ係数が記憶されている。そして、係数メモリ４５５は、クラスコード読み出し部４５４から供給される注目画素のクラスに対応するアドレスから、そのクラスのタップ係数を読み出し、積和演算部４５３に供給する。
【０５９５】
なお、クラスごとのタップ係数を求める学習を行う方法については、後述する。
【０５９６】
次に、図４７の受信処理部４０２では、ＰＤＡ１０１から送信されてくる画像データを処理する画像データ受信処理、ＰＤＡ１０１にリクエスト信号を送信するリクエスト信号送信処理、および画像データの画質を向上させる適応処理が行われるようになっている。
【０５９７】
そこで、図４９のフローチャートを参照して、図４７の受信処理部４０２で行われる画像データ受信処理、リクエスト信号送信処理、および適応処理について説明する。
【０５９８】
まず最初に、図４９（Ａ）のフローチャートを参照して、画像データ受信処理について説明する。
【０５９９】
ＰＤＡ１０１から送信されてくる送信データは、アンテナ６４（図２３）で受信され、通信Ｉ／Ｆ２１８を介して、受信データとして、受信制御部４４１に供給される。
【０６００】
受信制御部４４１は、受信データがクラスコード画像である場合、そのクラスコード画像を、クラスコードデータベース４４６に供給して記憶させる。
【０６０１】
また、受信制御部４４１は、受信データがＳＤ画像データである場合、そのＳＤ画像データを受信バッファ４４２に供給して記憶させる。
【０６０２】
受信バッファ４４２におけるＳＤ画像データの記憶が開始されると、画像データ受信処理が開始され、まず最初に、ステップＳ２３１において、登録部４４３は、受信バッファ４４２に記憶されたＳＤ画像データのうち、最も古い（最も時間的に過去の）フレームを、注目フレームとして、その注目フレームのＳＤ画像データを読み出し、その読み出したＳＤ画像データを、受信バッファ４４２から削除する。さらに、ステップＳ２３１において、登録部４４３は、受信バッファ４４２から読み出た注目フレームのＳＤ画像データと同一のフレームのＳＤ画像データが、ストレージ４４４に既に記憶されているかどうかを判定する。
【０６０３】
ここで、図４５の送信処理部４０１では、送信制御部４２２において、送信データとして出力されるＳＤ画像データの各フレームに、そのフレームを識別する識別情報が付加されるようになっており、登録部４４３は、そのフレーム識別情報を参照することで、受信バッファ４４２から読み出したＳＤ画像データと同一フレームのＳＤ画像データが、ストレージ４４４に記憶されているかどうかを判定する。
【０６０４】
なお、このように、受信処理部４０２において、同一フレームのＳＤ画像データが複数存在しうるのは、図４６（Ａ）のフローチャートで説明したように、図４５の送信処理部４０１が同一フレームのＳＤ画像データを送信することがあるからである。
【０６０５】
ステップＳ２３１において、注目フレームと同一フレームのＳＤ画像データが、ストレージ４４４に記憶されていないと判定された場合、ステップＳ２３２に進み、登録部４４３は、注目フレームのＳＤ画像データを、ストレージ４４４に書き込み、ステップＳ２３６に進む。
【０６０６】
一方、ステップＳ２３１において、注目フレームと同一フレームのＳＤ画像データが、ストレージ４４４に記憶されていると判定された場合、ステップＳ２３３に進み、登録部４４３は、ストレージ４４４に記憶されているＳＤ画像データのうち、注目フレームと同一フレームのＳＤ画像データを読み出し、ステップＳ２３４に進む。
【０６０７】
ステップＳ２３４では、登録部４４３は、注目フレームのＳＤ画像データと、ステップＳ２３４で読み出した、注目フレームと同一フレームのＳＤ画像データとを重み付け加算し、その重み付け加算値を画素値とする新たな注目フレームのＳＤ画像データを生成する。
【０６０８】
即ち、登録部４４３は、注目フレームのＳＤ画像データと、ストレージ４４４から読み出した、注目フレームと同一フレームのＳＤ画像データとを位置あわせした上で、同一位置にある画素値どうしを重み付け加算し、その位置における新たな画素値を求める。
【０６０９】
ここで、ステップＳ２３４における重み付け加算においては、その重み付け加算に用いる重みを、例えば、ストレージ４４４から読み出したＳＤ画像データの重み付け加算回数に基づいて決定することができる。
【０６１０】
即ち、ストレージ４４４に、注目フレームと同一のフレームのＳＤ画像データが記憶されている場合、登録部４４３において、その注目フレームのＳＤ画像データと、ストレージ４４４に記憶されているＳＤ画像データとが重み付け加算され、その重み付け加算値を画素値とするＳＤ画像データが、注目フレームのＳＤ画像データとして、新たに、ストレージ４４４に上書きする形で記憶される。
【０６１１】
そこで、ストレージ４４４に記憶された、注目フレームと同一フレームのＳＤ画像データが、Ｎフレーム分のＳＤ画像データの重み付け加算値である場合には、ステップＳ２３４では、注目フレームのＳＤ画像データの重みを１とするとともに、ストレージ４４４に記憶された注目フレームと同一フレームのＳＤ画像データの重みをＮとして、その２つのＳＤ画像データの重み付け加算を行うようにすることができる。
【０６１２】
なお、重みの付け方は、上述の方法に限定されるものではない。
【０６１３】
登録部４４３は、注目フレームについて、受信バッファ４４２から読み出したＳＤ画像データと、ストレージ４４４から読み出したＳＤ画像データとの重み付け加算値としての新たなＳＤ画像データを得ると、ステップＳ２３５に進み、その注目フレームの新たなＳＤ画像データを、ストレージ４４４に供給し、そこに記憶されていた注目フレームと同一フレームのＳＤ画像データに上書きする形で記憶させる。
【０６１４】
そして、ステップＳ２３６に進み、登録部４４３は、受信バッファ４４２に、まだ、ＳＤ画像データが記憶されているかどうかを判定し、記憶されていると判定した場合、ステップＳ２３１に戻り、次のフレーム（次に処理すべきフレーム）を新たに注目フレームとして、以下、同様の処理を繰り返す。
【０６１５】
一方、ステップＳ２３６において、受信バッファ４４２に、ＳＤ画像データが記憶されていないと判定された場合、処理を終了する。
【０６１６】
以上のように、画像データ受信処理では、同一フレームのＳＤ画像データが重み付け加算されるため、画質を向上させたＳＤ画像データを得ることが可能となる。
【０６１７】
即ち、ＰＤＡ１０１から１０３に対して、ＳＤ画像データが送信される場合には、その送信中に、ＳＤ画像データにノイズが重畳したり、また、一部のデータが欠落したりすることがあり、このようなノイズの重畳やデータの欠落等は、ＳＤ画像データの画質を大きく劣化させる。そこで、登録部４４３は、上述のように、同一フレームのＳＤ画像データどうしの重み付け加算を行うことで、その画質の劣化を改善する（画質が劣化した画像を基準とすれば、画質を向上させる）ようになっている。
【０６１８】
次に、図４９（Ｂ）のフローチャートを参照して、リクエスト信号送信処理について説明する。
【０６１９】
リクエスト信号送信処理は、例えば、任意のタイミングで開始され、まず最初に、ステップＳ２４１において、画質判定部４３２が、ストレージ４４４に記憶されている各フレームのＳＤ画像データを読み出し、各フレームのＳＤ画像データの自己相関を演算する。
【０６２０】
そして、ステップＳ２４２に進み、ステップＳ２４１で計算した自己相関に基づいて、各フレームのＳＤ画像データの画質を評価し、ステップＳ２４３に進む。
【０６２１】
ステップＳ２４３では、画質判定部４３２は、ステップＳ２４２での画質の評価結果に基づき、ストレージ４４４に記憶された各フレームのＳＤ画像データが、ある程度画質の良いものとなっているかどうかを判定する。ステップＳ２４３において、ストレージ４４４に記憶された各フレームのＳＤ画像データが、あまり画質の良いものとなっていないと判定された場合、ステップＳ２４４をスキップして、処理を終了する。
【０６２２】
また、ステップＳ２４３において、ストレージ４４４に記憶された各フレームのＳＤ画像データが、ある程度画質の良いものとなっていると判定された場合、ステップＳ２４４に進み、画質判定部４３２は、リクエスト信号送信部４３３にリクエスト信号を出力させ、処理を終了する。
【０６２３】
このリクエスト信号は、上述したように、ＰＤＡ１０１に送信され、リクエスト信号を受信したＰＤＡ１０１の送信処理部４０１（図４５）では、上述したように、そのリクエスト信号をイベントとして、クラスコード画像が送信される。そして、ＰＤＡ１０１から送信されてくるクラスコード画像は、上述したようにに、受信制御部４４１を介して、クラスコードデータベース４４６に供給されて記憶される。
【０６２４】
ここで、ステップＳ２４２では、例えば、ＳＤ画像データの自己相関が所定の閾値以上（より大）であるかどうかや、ＳＤ画像データの自己相関が所定の閾値以上であり、かつその所定の閾値以上の自己相関が、図４９（Ａ）の画像データ受信処理においてＳＤ画像データの重み付け加算を行う前後で、ほとんど変化しないかどうか等を、ＳＤ画像の画質の評価基準として採用することができる。
【０６２５】
また、ステップＳ２４３では、上述の評価基準を満たす場合、即ち、ＳＤ画像データの自己相関が所定の閾値以上である場合や、ＳＤ画像データの自己相関が所定の閾値以上であり、かつその所定の閾値以上の自己相関が、図４９（Ａ）の画像データ受信処理においてＳＤ画像データの重み付け加算を行う前後で、ほとんど変化しない場合に、ストレージ４４４に記憶された各フレームのＳＤ画像データが、ある程度画質の良いものとなっていると判定するようにすることができる。
【０６２６】
なお、ここでは、ＳＤ画像データの画質を評価する評価値として、そのＳＤ画像データの自己相関を用いることとしたが、他の評価値を用いることも可能である。
【０６２７】
即ち、ＳＤ画像データの画質を評価する評価値としては、その他、例えば、ストレージ４４４に記憶されたＳＤ画像データの、図４９（Ａ）の画像データ受信処理で重み付け加算がされる前後のものどうしの相互相関などを採用することも可能である。この場合、画質判定部４３２では、重み付け加算前のＳＤ画像データと、重み付け加算後のＳＤ画像データとが必要となるが、重み付け加算前のＳＤ画像データは、ストレージ４４４から読み出すことで取得し、重み付け加算後のＳＤ画像データは、登録部４４３に要求することで取得する。
【０６２８】
なお、ＳＤ画像データの画質を評価する評価値として、上述のような相互相関を使用する場合も、例えば、ＳＤ画像データの自己相関を使用する場合と同様に、ＳＤ画像データの相互相関が所定の閾値以上であるかどうか等を、ＳＤ画像データの画質の評価基準として採用することが可能である。
【０６２９】
次に、図４９（Ｃ）のフローチャートを参照して、適応処理について説明する。
【０６３０】
適応処理は、例えば、クラスコードデータベース４４６にクラスコード画像が記憶されると開始される。
【０６３１】
即ち、クラスコードデータベース４４６に、クラスコード画像が記憶されると、適応処理部４４７（図４８）は、ストレージ４４４に記憶されたＳＤ画像データを、順次読み出し、バッファ４５１に記憶させる。
【０６３２】
そして、予測タップ抽出部４５２は、バッファ４５１に記憶されたＳＤ画像データのフレームのうち、まだ処理していない最も古い（時間的に最も過去のフレーム）フレームを注目フレームとする。さらに、予測タップ抽出部４５２は、ステップＳ２５１において、注目フレームのＳＤ画像データの画質を向上させたＨＤ画像データ（このＨＤ画像データは、実際には存在しないが、存在するものと仮定される）を構成するＨＤ画素の、まだ予測値を求めていないものの１つを、注目画素として、その注目画素の画素値を予測するのに用いるＳＤ画素を、バッファ４５１に記憶されたＳＤ画像データから抽出し、予測タップとして出力する。
【０６３３】
その後、ステップＳ２５２に進み、クラスコード読み出し部４５４は、クラスコードデータベース４４６（図４７）に記憶されたクラスコード画像のうち、注目フレームのＨＤ画像データから生成されたものを、注目クラスコード画像とし、その注目クラスコード画像のクラスコードのうち、注目画素に対応するものを読み出して、係数メモリ４５５に対して、アドレスとして与える。
【０６３４】
なお、クラスコード読み出し部４５４は、クラスコードデータベース４４６から読み出したクラスコードについては、例えば、その読み出し後、クラスコードデータベース４４６から削除する。
【０６３５】
係数メモリ４５５は、アドレスとしてのクラスコードを受信すると、ステップＳ２５３に進み、そのクラスコードに対応するクラスの予測タップを読み出し、積和演算部４５３に供給して、ステップＳ２５４に進む。
【０６３６】
ステップＳ２５４では、積和演算部４５３が、予測タップ抽出部４５２が出力する注目画素についての予測タップと、係数メモリ４５５から供給される注目画素のクラスのタップ係数とを用いて、式（１）の線形一次予測演算を行い、これにより、注目画素の画素値（ＨＤ画素の画素値の予測値）を求めて出力し、ステップＳ２５５に進む。
【０６３７】
ステップＳ２５５では、クラスコード読み出し部４５４が、クラスコードデータベース４４６に、クラスコードが、まだ記憶されているかどうかを判定し、まだ記憶されていると判定された場合、ステップＳ２５１に戻り、クラスコードデータベース４４６に記憶されているクラスコードのうちのいずれかに対応するＨＤ画素を、新たに注目画素として、以下、同様の処理が繰り返される。
【０６３８】
また、ステップＳ２５５において、クラスコードデータベース４４６にクラスコードが記憶されていないと判定された場合、処理を終了する。
【０６３９】
以上のように、受信処理部４０２では、送信処理部４０１から送信されてくるＳＤ画像データが蓄積され、重み付け加算により画質が改善されていく。
【０６４０】
さらに、ＳＤ画像データの画質が、ある程度改善されると、リクエスト信号が、送信処理部４０１に送信され、これにより、送信処理部４０１から送信されてくるクラスコード画像が、クラスコードデータベース４４６に記憶される。そして、そのクラスコード画像を用いて、ＳＤ画像データがＨＤ画像データに変換される。
【０６４１】
従って、受信処理部４０２では、ＳＤ画像データの画質が、ある程度改善されると、そのことがトリガとなって、クラスコード画像が要求され、そのクラスコード画像に基づき、ＳＤ画像データの画質をさらに改善したＨＤ画像データが得られることになる。即ち、ＳＤ画像データの画質の改善がトリガとなって、そのＳＤ画像データが、より画質の良いＨＤ画像データに変換される。
【０６４２】
従って、ユーザからすれば、画像の画質が、突然改善されることになる。
【０６４３】
なお、ここでは、ＳＤ画像データの画質（Ｓ／Ｎ(Signal/Noise)等）を、重み付け加算により改善し、ＳＤ画像データの画質が、ある程度良好になってから、そのＳＤ画像データをＨＤ画像データに変換するようにしたが、そのようなＳＤ画像データの画質の改善を行わずに、ＨＤ画像データへの変換を行うことも可能である。
【０６４４】
但し、ＳＤ画像からＨＤ画像への変換処理としての適応処理を行う適応処理部４４７（図４８）で用いられる係数メモリ４５５に記憶されたタップ係数は、一般に、学習用のＨＤ画像データを教師データとするとともに、そのＨＤ画像データの解像度を単に劣化させたＳＤ画像データを生徒データとして用いて、式（８）の正規方程式をたてて求められるものであるため、即ち、生徒データとしてのＳＤ画像データは、データの欠落が生じたものや、ノイズが重畳されたものではないため、そのようなデータが欠落していたり、ノイズが重畳された画質が大きく劣化したＳＤ画像データを対象に適応処理を行うと、十分に画質を改善したＨＤ画像データを得ることが困難なことがある。
【０６４５】
このため、適応処理は、上述したように、画質が改善されたＳＤ画像データ、即ち、ここでは、過去に受信したＳＤ画像データを重み付け加算しながら蓄積したものを用いて行うのが望ましい。
【０６４６】
また、上述の場合には、画質判定部４３２において、ＳＤ画像データの自己相関や相互相関に基づいて、ストレージ４４４に記憶されたＳＤ画像データの画質を判定するようにしが、ＳＤ画像の画質の判定は、ユーザに行ってもらうようにすることも可能である。
【０６４７】
即ち、ＬＣＤ３等（図２３）に、ストレージ４４４に記憶されたＳＤ画像データを表示し、ユーザに、そのＳＤ画像データの画質が良くなったと感じたときに、操作部２２４等（図２３）を、画質が良くなったことを表すように操作してもらうようにすることができる。この場合、操作部２２４は、そのような操作がされたことを表す操作信号（以下、適宜、画質向上操作信号という）を、リクエスト信号送信部４３３に出力する。そして、リクエスト信号送信部４３３は、図４７において点線で示すように、画質向上操作信号を受信すると、リクエスト信号を送信する。
【０６４８】
この場合も、送信処理部４０１（図４５）から受信処理部４０２に対しては、クラスコード画像が送信されてくるので、受信処理部４０２では、やはり、ＳＤ画像データの画質をより向上させたＨＤ画像データを得ることができる。即ち、この場合、外部入力としてのユーザの操作をトリガとして、ＳＤ画像がＨＤ画像に変換されることになる。
【０６４９】
なお、その他、リクエスト信号送信部４３３では、例えば、受信装置としてのＰＤＡ１０３が、データの送受信を行ってない待ち受け状態にあるかどうかを判定し、待ち受け状態にあるときに、リクエスト信号を送信するようにすることが可能である。この場合、送信処理部４０１から受信処理部４０２に対しては、ＰＤＡ１０３（受信処理部４０２）が待ち受け状態にあるときに、クラスコードが送信されることになる。
【０６５０】
また、図４５の送信処理部４０１では、イベント検出部４１２において、リクエスト信号を検出する他、送信装置としてのＰＤＡ１０１が、データの送受信を行っていない待ち受け状態にあるかどうかを判定し、待ち受け状態にあるときに、イベントメッセージを出力するようにすることが可能である。この場合、送信処理部４０１から受信処理部４０２に対しては、ＰＤＡ１０１（送信処理部４０１）が待ち受け状態にあるときに、クラスコードが送信されることになる。
【０６５１】
その他、送信処理部４０１から受信処理部４０２に対するクラスコードの送信は、送信装置としてのＰＤＡ１０１と、受信装置としてのＰＤＡ１０３のいずれもが待ち受け状態であることを、所定のイベントとして行うようにすることも可能である。
【０６５２】
次に、図５０は、図４４の送信処理部４０１の第２の構成例を示している。なお、図中、図４５における場合と対応する部分については、同一の符号を付してあり、以下では、その説明は、適宜省略する。即ち、図５０の送信処理部４０１は、ストレージ４１３、学習部４１４、タップ係数バッファ４１５、およびストレージ４１６が新たに設けられている他は、基本的に、図４５における場合と同様に構成されている。
【０６５３】
ストレージ４１３には、符号化部４１１に供給されるのと同一のＨＤ画像データが供給されるようになっており、ストレージ４１３は、そのＨＤ画像データを一時記憶する。
【０６５４】
学習部４１４は、ストレージ４１３に新たに記憶されたＨＤ画像データを、学習用の画像データとして、上述した式（８）の正規方程式をたてて解く学習を行うことにより、適応処理に用いられるタップ係数を取得する。このタップ係数は、タップ係数バッファ４１５に供給される。
【０６５５】
タップ係数バッファ４１５は、学習部４１４が出力するタップ係数を一時記憶する。なお、タップ係数バッファ４１５に記憶されたタップ係数は、例えば、送信制御部４２２において、クラスコードバッファ４２５に記憶されたクラスコードが送信されるのと同じタイミングで、そのクラスコードとともに送信される。
【０６５６】
ストレージ４１６は、学習部４１４がタップ係数の学習を行うにあたって、その学習の途中で得られる情報を一時記憶する。
【０６５７】
次に、図５１は、図５０の学習部４１４の構成例を示している。
【０６５８】
教師データメモリ４６１は、例えば、ストレージ４１３に新たに記憶されたＨＤ画像データを、教師データとして記憶する。
【０６５９】
データ圧縮部４６２は、図５０のデータ圧縮部４２１と同様にして、ＨＤ画像データを圧縮し、ＳＤ画像データを生成して出力する。
【０６６０】
生徒データメモリ４６３は、データ圧縮部４６２が出力するＳＤ画像データを、生徒データとして記憶する。
【０６６１】
予測タップ抽出部４６４は、教師データメモリ４６１に記憶された教師データとしてのＨＤ画像データを構成するＨＤ画素を、順次、注目画素として、その注目画素について、図４８の予測タップ抽出部４５２（後述する図５４の予測タップ抽出部４５２）が構成するのと同一の予測タップを、生徒データメモリ４６３に記憶された生徒データとしてのＳＤ画像データを構成するＳＤ画素から生成し、足し込み部４６７に供給する。
【０６６２】
クラスタップ抽出部４６５は、注目画素について、図５０のクラスタップ抽出部４２３が構成するのと同一のクラスタップを、教師データメモリ４６１に記憶された教師データとしてのＨＤ画像データを構成するＨＤ画素から構成し、クラス分類部４６６に出力する。
【０６６３】
クラス分類部４６６は、クラスタップ抽出部４６５から供給される注目画素についてのクラスタップに基づいて、図５０のクラス分類部４２４における場合と同様にして、注目画素をクラス分類し、そのクラスを表すクラスコードを、足し込み部４６７に出力する。
【０６６４】
足し込み部４６７は、教師データメモリ４６１から、注目画素となっている教師データ（ＨＤ画素）を読み出し、予測タップ生成部４６４からの予測タップを構成する生徒データ、および注目画素としての教師データを対象とした足し込みを、ストレージ４１６（図５０）の記憶内容を用いながら、クラス分類部４６６から供給されるクラスごとに行う。
【０６６５】
即ち、足し込み部４６７は、まず、クラス分類部４６６から供給されるクラスコードに対応するクラスごとに、予測タップ（生徒データ）を用い、式（８）の行列Ａにおける各コンポーネントとなっている、生徒データどうしの乗算（ｘ_inｘ_im）と、サメーション（Σ）に相当する演算を行う。
【０６６６】
さらに、足し込み部４６７は、やはり、クラス分類部４６６から供給されるクラスコードに対応するクラスごとに、予測タップ（生徒データ）および注目画素（教師データ）を用い、式（８）のベクトルｖにおける各コンポーネントとなっている、生徒データと教師データの乗算（ｘ_inｙ_i）と、サメーション（Σ）に相当する演算を行う。
【０６６７】
一方、ストレージ４１６は、足し込み部４６７において前回の学習までに求められた式（８）における行列Ａのコンポーネントと、ベクトルｖのコンポーネントを、クラスごとに記憶している。
【０６６８】
足し込み部４６７は、新たな学習用の画像データを用いて学習を行う場合、ストレージ４１６に記憶された、前回までの学習で求められた式（８）における行列Ａのコンポーネントと、ベクトルｖのコンポーネントを読み出し、その行列Ａまたはベクトルｖのコンポーネントに対して、新たな学習用の画像データから得られる教師データおよび生徒データを用いて計算される、対応するコンポーネントｘ_inｘ_imまたはｘ_inｙ_iをそれぞれ足し込むことにより（行列Ａ、ベクトルｖにおけるサメーションで表される加算を行うことにより）、各クラスについて、式（８）に示した新たな正規方程式をたてる。
【０６６９】
従って、足し込み部４６７では、新たな学習用の画像データだけではなく、過去の学習に用いられた画像データにも基づいて、式（８）の正規方程式がたてられる。即ち、ストレージ４１６は、過去の学習において得られた行列Ａおよびベクトルｖのコンポーネントを蓄積しており、足し込み部４６７では、その蓄積された行列Ａおよびベクトルｖのコンポーネントをも用いて、式（８）の正規方程式がたてられる。
【０６７０】
なお、例えば、学習部４１４で、初めて学習が行われる場合おいては、ストレージ４１６には、前回の学習までに求められた行列Ａとベクトルｖのコンポーネントは記憶されていないため、式（８）の正規方程式は、いまある学習用の画像データだけを用いてたてられることになる。
【０６７１】
この場合、学習用の画像データのサンプル数が十分でないこと等に起因して、タップ係数を求めるのに必要な数の正規方程式が得られないクラスが生じる場合があり得る。
【０６７２】
そこで、ストレージ４１６には、あらかじめ用意された多数のＨＤ画像データを学習用のデータとして学習を行うことにより得られた、クラスごとの行列Ａのコンポーネントと、ベクトルｖのコンポーネントを、初期値として記憶させておくようにすることができる。この場合、タップ係数を求めるのに必要な数の正規方程式が得られないクラスが生じることを防止することができる。
【０６７３】
足し込み部４６７は、新たな学習用の画像データから得られた行列Ａおよびベクトルｖのコンポーネントと、ストレージ４１６に記憶された行列Ａおよびベクトルｖのコンポーネントと用いて、新たに、クラスごとの行列Ａおよびベクトルｖのコンポーネントを求めると、それらのコンポーネントを、ストレージ４１６に供給し、上書きする形で記憶させる。
【０６７４】
さらに、足し込み部４６７は、新たに求めたクラスごとの行列Ａおよびベクトルｖのコンポーネントで構成される式（８）の正規方程式を、タップ係数決定部４６８に供給する。
【０６７５】
タップ係数決定部４６８は、足し込み部４６７から供給されるクラスごとの正規方程式を解くことにより、クラスごとに、タップ係数を求め、このクラスごとのタップ係数を、タップ係数バッファ４１５（図５０）に供給して、上書きする形で記憶させる。
【０６７６】
なお、図５１の学習部４１４では、クラスタップ抽出部４６５において、図５０のクラスタップ抽出部４２３と同様に、ＨＤ画像データを構成するＨＤ画素からクラスタップが生成され、さらに、クラス分類部４６６において、そのＨＤ画素から構成されるクラスタップに基づいて、クラス分類が行われるから、タップ係数決定部４６８で得られるクラスごとのタップ係数は、ＨＤクラスごとのタップ係数である。
【０６７７】
以上のように構成される送信処理部４０１では、図４６で説明した画像データ送信処理、クラスコード生成処理、およびクラスコード送信処理の他、ＨＤクラスごとのタップ係数を求める学習処理、およびそのＨＤクラスごとのタップ係数を送信するタップ係数送信処理が行われるようになっている。
【０６７８】
そこで、図５２のフローチャートを参照して、図５０の送信処理部４０１で行われる学習処理およびタップ係数送信処理について説明する。
【０６７９】
まず、図５２（Ａ）のフローチャートを参照して、図５０の送信処理部４０１が行う学習処理について説明する。
【０６８０】
学習処理は、所定のタイミングで、学習部４１４（図５１）において開始される。
【０６８１】
即ち、学習部４１４は、例えば、周期的に、あるいは、ストレージ４１３（図５０）に、所定数フレーム以上の新たなＨＤ画像データが記憶されると、学習処理を開始する。
【０６８２】
学習処理が開始されると、まず最初に、前回の学習から今回の学習までの間に、ストレージ４１３（図５０）に新たに記憶されたＨＤ画像データが、新たな学習用の画像データとして読み出され、教師データメモリ４６１に供給され、教師データとして記憶される。
【０６８３】
そして、ステップＳ２６１において、足し込み部４６７が、ストレージ４１６から、式（８）における行列Ａのコンポーネントと、ベクトルｖのコンポーネントを読み出し、ステップＳ２６２に進む。
【０６８４】
ステップＳ２６２では、データ圧縮部４６２が、教師データメモリ４６１に記憶された教師データを読み出し、ＳＤ画像データに変換する。このＳＤ画像データは、生徒データメモリ４６３に供給され、生徒データとして記憶される。
【０６８５】
そして、ステップＳ２６３に進み、予測タップ抽出部４６１が、教師データメモリ４１６に記憶された教師データとしてのＨＤ画素のうち、まだ注目画素としていないもののうちの１つを注目画素として、その注目画素について、生徒データメモリ４６３から生徒データとしての幾つかのＳＤ画素を読み出すことにより、予測タップを生成する。
【０６８６】
さらに、ステップＳ２６３では、クラスタップ抽出部４６５が、注目画素について、教師データメモリ４６１から教師データとしての幾つかのＨＤ画素を読み出すことにより、クラスタップを生成する。
【０６８７】
予測タップ抽出部４６４で生成された予測タップは、足し込み部４６７に供給され、クラスタップ抽出部４６５で生成されたクラスタップは、クラス分類部４６６に供給される。
【０６８８】
その後、ステップＳ２６４に進み、クラス分類部４６６が、クラスタップ抽出部４６５から供給されるクラスタップに基づき、注目画素をクラス分類し、その注目画素のクラスを表すクラスコードを、足し込み部４６７に供給する。
【０６８９】
そして、ステップＳ２６５に進み、足し込み部４６７は、教師データメモリ４６１から注目画素を読み出し、その注目画素と、予測タップ抽出部４６４からの予測タップを用いて、行列Ａとベクトルｖのコンポーネントを計算する。さらに、足し込み部４６７は、ステップＳ２６１でストレージ４１６（図５０）から読み出した行列Ａとベクトルｖのコンポーネントのうち、クラス分類部４６６からのクラスコードに対応するものに対して、注目画素と予測タップから求められた行列Ａとベクトルｖのコンポーネントを足し込み、ステップＳ２６６に進む。
【０６９０】
ステップＳ２６６では、予測タップ抽出部４６４が、教師データメモリ４６１に、まだ、注目画素としていない教師データが存在するかどうかを判定し、存在すると判定した場合、ステップＳ２６３に戻り、まだ、注目画素とされていない教師データを、新たに注目画素として、以下、同様の処理が繰り返される。
【０６９１】
また、ステップＳ２６６において、教師データメモリ４６１に、注目画素としていない教師データが存在しないと判定された場合、足し込み部４６７は、それまでに得られているクラスごとの行列Ａおよびベクトルｖのコンポーネントで構成される式（８）の正規方程式を、タップ係数決定部４６８に供給し、ステップＳ２６７に進む。
【０６９２】
ステップＳ２６７では、足し込み部４６７は、タップ係数決定部４６８に供給したクラスごとの行列Ａおよびベクトルｖのコンポーネントを、ストレージ４１６にも供給し、上書きする形で記憶させ、ステップＳ２６８に進む。
【０６９３】
ステップＳ２６８では、タップ係数決定部４６８は、足し込み部４６７から供給される各クラスごとの正規方程式を解くことにより、各クラスごとに、タップ係数を求める。さらに、ステップＳ２６８では、タップ係数決定部４６８は、各クラスごとのタップ係数を、タップ係数バッファ４１５に供給して上書きする形で記憶させ、学習処理を終了する。
【０６９４】
次に、図５２（Ｂ）のフローチャートを参照して、図５０の送信処理部４０１が行うタップ係数送信処理について説明する。
【０６９５】
タップ係数送信処理では、まず最初に、ステップＳ２７１において、送信制御部４０１が、図４６（Ｃ）のステップＳ２２１における場合と同様にして、所定のイベントが発生したかどうかを判定し、所定のイベントが発生していないと判定した場合、ステップＳ２７１に戻り、所定のイベントが発生するまで待つ。
【０６９６】
そして、ステップＳ２７１において、所定のイベントが発生したと判定された場合、ステップＳ２７２に進み、送信制御部４２２は、クラス（ＨＤクラス）ごとのタップ係数が、タップ係数バッファ４１５に記憶されているかどうかを判定する。
【０６９７】
ステップＳ２７２において、ＨＤクラスごとのタップ係数が記憶されていないと判定された場合、ステップＳ２７１に戻り、以下、同様の処理が繰り返される。
【０６９８】
また、ステップＳ２７２において、ＨＤクラスごとのタップ係数が記憶されていると判定された場合、ステップＳ２７３に進み、送信制御部４２２は、タップ係数バッファ４１５に記憶されたクラスコードを読み出し、送信データとして選択する。この送信データは、送信制御部４２２から通信Ｉ／Ｆ２１８（図２３）に供給され、アンテナ６４から送信される。
【０６９９】
送信制御部４２２は、ステップＳ２２３において、タップ係数バッファ４１５に記憶されていたすべてのＨＤクラスごとのタップ係数の送信が終了すると、ステップＳ２７１に戻り、以下、同様の処理を繰り返す。
【０７００】
次に、図５３は、図４４の受信処理部４０２の第２の構成例を示している。即ち、図５３は、送信処理部４０１が図５０に示したように構成される場合の受信処理部４０２の構成例を示している。
【０７０１】
なお、図中、図４７における場合と対応する部分については、同一の符号を付してあり、以下では、その説明は、適宜省略する。即ち、図５３の受信処理部４０２は、適応処理部４４７に代えて適応処理部４４８が設けられている他は、基本的に、図４７における場合と同様に構成されている。
【０７０２】
但し、図５３の実施の形態においては、図５０の送信処理部４０１から、ＳＤ画像データおよびＨＤクラスを表すクラスコードの他、ＨＤクラスごとのタップ係数が送信されてくる場合があり、ＨＤクラスごとのタップ係数が送信されてきた場合には、受信制御部４４１は、そのＨＤクラスごとのタップ係数を、適応処理部４４８に供給するようになっている。
【０７０３】
適応処理部４４８は、受信制御部４４１から供給されるタップ係数を用いて、図４８の適応処理部４４７における場合と同様の処理（適応処理）を行い、これにより、ＨＤ画像データを生成する（予測する）ようになっている。
【０７０４】
即ち、図５４は、図５３の適応処理部４４８の構成例を示している。なお、図中、図４８の適応処理部４４７における場合と対応する部分については、同一の符号を付してあり、以下では、その説明は、適宜省略する。即ち、図５４の適応処理部４４８は、登録部４５６が新たに設けられている他は、図４８の適応処理部４４７と同様に構成されている。
【０７０５】
登録部４５６は、受信制御部４４１（図５３）から供給されるＨＤクラスごとのタップ係数を受信し、係数メモリ４５５に上書きする形で記憶させる。
【０７０６】
従って、図４８の実施の形態では、固定の、ＨＤクラスごとのタップ係数を用いて適応処理が行われるようになっていたが、図５４の実施の形態では、図５０の送信処理部４０１から、新たなＨＤクラスごとのタップ係数が送信されてくると、その新たなＨＤクラスごとのタップ係数によって、係数メモリ４５５の記憶内容が更新され、その新たなＨＤクラスごとのタップ係数を用いて適応処理が行われる。
【０７０７】
図５０の送信処理部４０１では、上述したように、新たなＨＤ画像データを、新たな学習用の画像データとして、タップ係数が更新されるから、そのタップ係数は、ＳＤ画像を、より、ＨＤ画像に近い画像に変換するものとなっていく。従って、図５３の受信処理部４０２では、そのようなタップ係数を用いて、適応処理が行われるので、より画質の良いＨＤ画像を得ることが可能となる。
【０７０８】
なお、図４５および図５０のデータ圧縮部４２１、並びに図５１のデータ圧縮部４６２では、ＨＤ画像データの空間方向の画素数を間引く等することによって、ＳＤ画像データを生成するようにしたが、その他、ＳＤ画像データは、例えば、ＨＤ画像データの時間方向の画素数を間引く等して生成することも可能である。
【０７０９】
次に、図５５は、図４４の送信処理部４０１の第３の構成例を示している。
【０７１０】
図５５の実施の形態では、送信処理部４０１において、画像データがベクトル量子化されることにより符号化されて送信されるようになっている。
【０７１１】
即ち、ベクトル化部５０１には、ＣＣＤカメラ６５（図２３）で撮像された画像データや、ＨＤＤ２１５に記憶された画像データが供給される。ベクトル化部５０１は、そこに供給される画像データをベクトル化する。即ち、ベクトル化部５０１は、例えば、そこに供給される各フレームの画像データを、水平方向×垂直方向の画素数が、例えば、３×３画素のブロックにブロック化し、そのブロックの９画素（３×３画素）の画素値を所定の順番で並べたものを、ベクトルのコンポーネントとするベクトル（以下、適宜、画像ベクトルという）を生成する。ベクトル化部５０１において得られた画像ベクトルは、ベクトル量子化部５０２および差分演算部５０４に供給される。
【０７１２】
ベクトル量子化部５０２は、コードブック記憶部５０９に記憶されたコードブックを参照し、ベクトル化部５０１から供給される画像ベクトルをベクトル量子化する。
【０７１３】
即ち、ベクトル量子化部５０２は、コードブック記憶部５０９に記憶されたコードブックに登録されているすべてのコードベクトルと、ベクトル化部５０１からの画像ベクトルとの距離を計算し、その距離を最小にするコードベクトルに対応するコードを、ベクトル量子化結果として出力する。ベクトル量子化部５０２が出力するコードは、ローカルデコード部５０３、エントロピー符号化部５０５、および更新部５０６に供給される。
【０７１４】
ローカルデコード部５０３は、ベクトル量子化部５０２から供給されるコードを、コードブック記憶部５０９に記憶されたコードブック（ベクトル量子化部５０２がベクトル量子化に用いたのと同一のコードブック）を用いてベクトル逆量子化する。即ち、ローカルデコード部５０３は、コードブック記憶部５０９に記憶されたコードブックにおいて、ベクトル量子化部５０２が出力するコードに対応するコードベクトルを、ベクトル逆量子化結果として出力する。このベクトル逆量子化結果としてのコードベクトルは、ローカルデコード部５０３から差分演算部５０４に供給される。
【０７１５】
差分演算部５０４は、ベクトル化部５０１から供給される画像ベクトルと、ローカルデコード部５０３から供給される、その画像ベクトルをベクトル量子化し、さらにベクトル逆量子化して得られるコードベクトルとの差分を計算し、その結果得られるベクトル（以下、適宜、差分ベクトルという）を、エントロピー符号化部５０５および更新部５０６に供給する。
【０７１６】
エントロピー符号化部５０５は、ベクトル量子化部５０２から供給される、画像ベクトルのベクトル量子化結果としてのコードと、差分演算部５０４から供給される、その画像ベクトルについて得られた差分ベクトルとをエントロピー符号化し、送信データとして出力する。この送信データは、通信Ｉ／Ｆ２１８（図２３）を介して、受信処理部４０２（受信装置としてのＰＤＡ１０３）に送信される。
【０７１７】
なお、差分ベクトルは、上述のように、エントロピー符号化部５０５においてエントロピー符号化されるため、差分ベクトルが０となる頻度が高い場合には、送信データ量のデータ量を低減することができる。即ち、ベクトル量子化部５０２が、コードブック記憶部５０９に記憶されたコードブックを用いて行うベクトル量子化（符号化）による量子化誤差が０となる頻度が高い場合には、送信データのデータ量を低減することができる。このことは、同一画質の画像を得るための送信データのデータ量が少なくなることを意味するから、見方を変えて、送信データのデータ量が一定であると仮定すれば、上述のデータ量の低減する分だけ、画質を向上させることができることと等価である。
【０７１８】
更新部５０６は、ベクトル量子化部５０２から供給されるコードと、差分演算部５０４から供給される差分ベクトルとに基づいて、コードブックデータベース５０７に記憶されたコードブックを更新する。なお、更新部５０６には、選択部５０８から、コードブック記憶部５０９に記憶されているコードブック、即ち、現在のベクトル量子化に用いられているコードブックを特定する情報（後述するコードブック番号）が供給されるようになっており、更新部５０６は、そのコードブック番号に基づいて、更新対象のコードブックを特定する。
【０７１９】
コードブックデータベース５０７は、画像ベクトルをベクトル量子化するのに用いられる１以上のコードブックを記憶している。
【０７２０】
なお、コードブックデータベース５０７は、コードブックの初期値として、例えば、あらかじめ用意された学習用の多量の画像データを用い、ＬＢＧ(Linde Buzo Gray)アルゴリズム等によって求められたコードブックを記憶しており、更新部５０６は、その初期値のコードブックを、後述するように、適宜更新していく。
【０７２１】
また、コードブックデータベース５０７は、１以上のコードブックを記憶するが、その記憶方法としては、例えば、最初に、初期値のコードブックを１以上記憶しておくようにしても良いし、また、最初は、初期値のコードブックを１つだけ記憶しておき、その後、必要に応じて、その初期値のコードブックをコピーするようにしても良い。
【０７２２】
選択部５０８には、選択情報が供給されるようになっており、選択部５０８は、その選択情報にしたがい、コードブックデータベース５０７に記憶された１以上のコードブックの中から、ベクトル量子化に用いるものを選択する。そして、選択部５０８は、その選択したコードブックを、コードブックデータベース５０７から読み出し、コードブック記憶部５０９に供給して上書きする形で記憶させる。
【０７２３】
ここで、選択情報としては、例えば、ユーザ入力や、通信相手の情報、ベクトル化部５０１に供給されるのと同一の画像データ等を採用することができる。ユーザ入力は、ユーザが操作部２２４（図２３）等を操作することにより与えられる。また、通信相手の情報は、送信装置としてのＰＤＡ１０１が、受信装置としてのＰＤＡ１０３と通信を開始するときに、受信装置としてのＰＤＡ１０３から送信されることにより、アンテナ６４および通信Ｉ／Ｆ２１８（図２３）を介して与えられる。さらに、ベクトル化部５０１に供給されるのと同一の画像データは、ＣＣＤカメラ６５やＨＤＤ２１５（図２３）から与えられる。
【０７２４】
選択部５０８は、ユーザ入力が、選択情報として与えられる場合、そのユーザ入力にしたがって、コードブックデータベース５０７に記憶された１以上のコードブックから、１つのコードブックを選択する。従って、この場合、ベクトル量子化部５０２では、ユーザが指示したコードブックを用いて、画像データのベクトル量子化が行われることになる。
【０７２５】
また、選択部５０８は、通信相手の情報が、選択情報として与えられる場合、その通信相手の情報にしたがって、コードブックデータベース５０７に記憶された１以上のコードブックから、１つのコードブックを選択する。従って、この場合、ベクトル量子化部５０２では、通信相手によって異なる（一人の通信相手ごとに異なる、あるいは、複数のユーザをグループとし、グループごとに異なる）コードブックを用いて、画像データのベクトル量子化が行われることになる。
【０７２６】
さらに、選択部５０８は、画像データが、選択情報として与えられる場合、その画像データにしたがって、コードブックデータベース５０７に記憶された１以上のコードブックから、１つのコードブックを選択する。従って、この場合、ベクトル量子化部５０２では、画像データの特性（例えば、画像データの絵柄を表すアクティビティや、明るさ、動きなど）ごとに異なるコードブックを用いて、その画像データのベクトル量子化が行われることになる。
【０７２７】
なお、選択部５０８は、コードブックデータベース５０７に記憶された１以上のコードブックから、１つのコードブックを選択し、コードブック記憶部５０９に記憶させるとき、即ち、ベクトル量子化部５０２でベクトル量子化に用いられるコードブックを変更するとき、その変更後のコードブック（コードブックデータベース５０７から選択したコードブック）を特定するコードブック番号を、更新部５０６に供給する。
【０７２８】
以上のように構成される送信処理部４０１では、画像データをベクトル量子化して送信する画像データ送信処理、そのベクトル量子化に用いるコードブックを選択するコードブック選択処理、およびコードブックデータベース５０７の記憶内容を更新する更新処理が行われる。
【０７２９】
そこで、まず、図５６のフローチャートを参照して、図５５の送信処理部４０１が行う画像データ送信処理とコードブック選択処理について説明する。
【０７３０】
まず最初に、図５６（Ａ）のフローチャートを参照して、画像データ送信処理について説明する。
【０７３１】
画像データ送信処理では、送信すべき画像データが、例えば、１フレーム単位で、ベクトル化部５０１に供給され、ベクトル化部５０１は、その画像データを受信する。
【０７３２】
そして、ステップＳ３０１において、ベクトル化部５０１は、そこに供給される１フレームの画像データをベクトル化する。即ち、ベクトル化部５０１は、１フレームの画像データを、例えば、３×３画素のブロックにブロック化し、そのブロックの９画素（３×３画素）の画素値を所定の順番で並べたものをコンポーネントとする画像ベクトルを生成する。ベクトル化部５０１において、１フレームの画像データについて得られた画像ベクトルは、ベクトル量子化部５０２および差分演算部５０４に供給される。
【０７３３】
なお、以下説明するステップＳ３０２乃至Ｓ３０５の処理は、１フレームの画像データについて得られた画像ベクトルそれぞれについて行われる。
【０７３４】
ベクトル量子化部５０２は、ベクトル化部５０１から画像ベクトルを受信すると、ステップＳ３０２において、その画像ベクトルを、コードブック記憶部５０９に記憶されたコードブックを用いてベクトル量子化し、その結果得られるコードを、ローカルデコード部５０３、エントロピー符号化部５０５、および更新部５０６に供給して、ステップＳ３０３に進む。
【０７３５】
ステップＳ３０３では、ローカルデコード部５０３が、ベクトル量子化部５０２から供給されるコードを、コードブック記憶部５０９に記憶されたコードブックを用いてベクトル逆量子化し、その結果得られるコードベクトルを、差分演算部５０４に供給して、ステップＳ３０４に進む。
【０７３６】
ステップＳ３０４では、差分演算部５０４が、ベクトル化部５０１から供給される画像ベクトルと、ローカルデコード部５０３から供給されるコードベクトルとの差分を計算し、その結果得られる差分ベクトルを、エントロピー符号化部５０５および更新部５０６に供給して、ステップＳ３０５に進む。
【０７３７】
ステップＳ３０５では、エントロピー符号化部５０５が、ベクトル量子化部５０２から供給される、画像ベクトルのベクトル量子化結果としてのコードと、差分演算部５０４から供給される、その画像ベクトルについて得られた差分ベクトルとをエントロピー符号化し、送信データとして出力する。この送信データは、通信Ｉ／Ｆ２１８（図２３）を介して、受信処理部４０２（受信装置としてのＰＤＡ１０３）に送信される。
【０７３８】
その後、ステップＳ３０６に進み、ベクトル化部５０１は、次のフレームの画像データがあるかどうかを判定し、あると判定した場合、ステップＳ３０１に戻り、以下、同様の処理が繰り返される。
【０７３９】
また、ステップＳ３０６において、次のフレームの画像データがないと判定された場合、処理を終了する。
【０７４０】
次に、図５６（Ｂ）のフローチャートを参照して、コードブック選択処理について説明する。
【０７４１】
なお、コードブック選択処理は、例えば、画像データ送信処理（図５６（Ａ））が開始される直前に開始される。
【０７４２】
コードブック選択処理では、まず最初に、ステップＳ３１１において、選択部５０８が、コードブックデータベース５０７に記憶された１以上のコードブックの中から、デフォルトのコードブックを選択し、コードブック記憶部５０９に供給して記憶させる。
【０７４３】
ここで、デフォルトのコードブックとしては、例えば、上述した初期値のコードブックを採用することができる。
【０７４４】
その後、ステップＳ３１２に進み、選択部５０８は、選択情報が供給されたかどうかを判定し、供給されていないと判定した場合、ステップＳ３１３をスキップして、ステップＳ３１４に進む。
【０７４５】
また、ステップＳ３１２において、選択情報が供給されたと判定された場合、ステップＳ３１３に進み、選択部５０８は、その選択情報にしたがい、コードブックデータベース５０７に記憶された１以上のコードブックの中から、ベクトル量子化に用いるものを選択し、コードブック記憶部５０９に供給して記憶させる。
【０７４６】
そして、ステップＳ３１４に進み、選択部５０８は、今回のコードブック選択処理を開始した直後に開始された画像データ送信処理（図５６（Ａ））による画像データの送信が終了したかどうかを判定し、終了していないと判定した場合、ステップＳ３１２に戻り、以下、同様の処理を繰り返す。
【０７４７】
従って、この場合、画像データ送信処理による画像データの送信が行われている間に、選択部５０８に対して、新たな選択情報が供給されると、コードブック記憶部５０９に記憶されるコードブック、即ち、ベクトル量子化部５０２におけるベクトル量子化で用いられるコードブックが、その新たな選択情報に基づいて変更され得る。
【０７４８】
一方、ステップＳ３１４において、画像データの送信が終了したと判定された場合、処理を終了する。
【０７４９】
なお、図５６（Ｂ）の実施の形態では、上述したように、画像データ送信処理による画像データの送信が行われている間に、選択部５０８に対して、新たな選択情報が供給された場合に、ベクトル量子化で用いられるコードブックが、その新たな選択情報に基づいて、随時変更され得るようにしたが、画像データ送信処理による画像データの送信が行われている間のコードブックの変更は、１回だけに制限することが可能である。即ち、コードブックの変更は、デフォルトのコードブックから、最初に供給された選択情報に基づくコードブックへの変更のみに制限することが可能である。
【０７５０】
次に、図５５の送信処理部４０１において行われる、コードブックデータベース５０７の記憶内容を更新する更新処理について説明するが、その前に、コードブックデータベース５０７に記憶されているコードブックと、更新部５０６の構成について説明する。
【０７５１】
図５７は、図５５のコードブックデータベース５０７に記憶されているコードブックの例を示している。
【０７５２】
コードブックには、コードブック番号およびコードブックバージョンが付されている。
【０７５３】
コードブック番号は、そのコードブックを特定するユニークな番号であり、従って、コードブック番号によれば、コードブックが一意に特定される。コードブックバージョンは、そのコードブックのバージョンを表す情報であり、例えば、そのコードブックが更新された年月日および時刻と更新回数等で表される。
【０７５４】
図５７の実施の形態において、コードブックは、一般のコードブックと同様に、コードｎとコードベクトルＶ_n（＝（ａ_n，ｂ_n，・・・））とが対応付けられて構成されている。なお、図５７の実施の形態では、コード（コードベクトル）の数は、ｎ＋１個とされており、コードとして、０からＮまでの整数値が用いられている。
【０７５５】
さらに、図５７の実施の形態のコードブックにおいては、各コード＃ｎに、コードベクトルＶ_nの他、前回の更新時までの頻度Ａ_n、差分ベクトルの加算値Σ△_n（＝（ａ’_n，ｂ’_n，・・・））、および前回の更新時から現在までの頻度Ｂ_nも対応付けられている。
【０７５６】
ここで、前回の更新時までの頻度Ａ_nとは、前回のコードブックの更新時までのベクトル量子化において、コード＃ｎが、ベクトル量子化結果として出力された頻度を表す。
【０７５７】
また、差分ベクトルの加算値Σ△_nは、前回のコードブックの更新が行われた直後からいままでのベクトル量子化において、コード＃ｎのベクトル量子化結果が得られたときの差分ベクトル△_nの総和を表す。
【０７５８】
さらに、前回の更新時から現在までの頻度Ｂ_nは、前回のコードブックの更新が行われた直後からいままでのベクトル量子化において、コード＃ｎが、ベクトル量子化結果として出力された頻度を表す。
【０７５９】
なお、前回の更新時までの頻度Ａ_n、差分ベクトルの加算値Σ△_n、および前回の更新時から現在までの頻度Ｂ_nは、そのコードブックを用いて行われたベクトル量子化についてのものであり、他のコードブックを用いて行われたベクトル量子化についての前回の更新時までの頻度Ａ_n、差分ベクトルの加算値Σ△_n、および前回の更新時から現在までの頻度Ｂ_nは、その、他のコードブックに登録される。
【０７６０】
また、初期値のコードブックについては、前回の更新時までの頻度Ａ_n、差分ベクトルの加算値Σ△_n、および前回の更新時から現在までの頻度Ｂ_nは、例えば、すべて０とされている。
【０７６１】
次に、図５８は、図５５の更新部５０６の構成例を示している。
【０７６２】
更新部５０６は、図５８に示すように、データ更新部５２１とコードブック更新部５２２とから構成される。
【０７６３】
データ更新部５２１には、選択部５０８から、コードブック記憶部５０９に記憶されているコードブック、即ち、ベクトル量子化部５０２で使用されているコードブックを特定する情報としてのコードブック番号が供給されるようになっている。さらに、データ更新部５２１には、ベクトル量子化部５０２が出力する画像ベクトルのベクトル量子化結果としてのコードも供給されるようになっている。また、データ更新部５２１には、差分演算部５０４が出力する、ベクトル量子化部５０２でベクトル量子化が行われた画像ベクトルについて求められた差分ベクトルも供給されるようになっている。
【０７６４】
そして、データ更新部５２１は、コードブックデータベース５０７に記憶された１以上のコードブックの中から、いまベクトル量子化に用いられているコードブックを、選択部５０８（図５５）から供給されるコードブック番号によって特定する。さらに、データ更新部５２１は、その特定したコードブックを、注目コードブックとして、その注目コードブックの差分ベクトルの加算値Σ△_n、および前回の更新時から現在までの頻度Ｂ_nを、ベクトル量子化部５０２からのコードと、差分演算部５０４からの差分ベクトルによって更新する。
【０７６５】
コードブック更新部５２２は、コードブックデータベース５０７に記憶された１以上のコードブックそれぞれを、各コードブックに記憶されている前回の更新時までの頻度Ａ_n、差分ベクトルの加算値Σ△_n、および前回の更新時から現在までの頻度Ｂ_nに基づいて更新し、その更新後のコードブックを、コードブックデータベース５０７に上書きする形で記憶させる。
【０７６６】
以上のように構成される更新部５０６で行われる更新処理は、コードブックデータベース５０７に記憶された差分ベクトルの加算値Σ△_n、および前回の更新時から現在までの頻度Ｂ_nを更新するデータ更新処理と、コードブックのコードベクトルを、そのコードブックに登録されている前回の更新時までの頻度Ａ_n、差分ベクトルの加算値Σ△_n、および前回の更新時から現在までの頻度Ｂ_nに基づいて更新するコードブック更新処理とからなる。
【０７６７】
そこで、図５９のフローチャートを参照して、データ更新処理とコードブック更新処理について説明する。
【０７６８】
まず最初に、図５９（Ａ）のフローチャートを参照して、データ更新処理について説明する。
【０７６９】
データ更新処理では、まず最初に、ステップＳ３２１において、データ更新部５２１が、選択部５０８（図５５）からコードブック番号を受信したかどうかを判定する。
【０７７０】
ステップＳ３２１において、選択部５０８からコードブック番号を受信したと判定された場合、即ち、ベクトル量子化部５０２でのベクトル量子化に用いられるコードブックが変更された場合、ステップＳ３２２に進み、データ更新部５２１は、コードブックデータベース５０７に記憶された１以上のコードブックのうち、選択部５０８から供給されるコードブック番号によって特定されるコードブックを、注目コードブックとして、ステップＳ３２３に進む。
【０７７１】
また、ステップＳ３２１において、選択部５０８からコードブック番号を受信していないと判定された場合、即ち、ベクトル量子化部５０２でのベクトル量子化に用いられるコードブックが変更されず、従って、いままでのベクトル量子化で用いられていたものが、そのまま注目コードブックとされる場合、ステップＳ３２２をスキップして、ステップＳ３２３に進む。
【０７７２】
ステップＳ３２３では、データ更新部５２１は、ベクトル量子化部５０２（図５５）から、画像ベクトルのベクトル量子化結果としてのコードが供給されるとともに、差分演算部５０４から、その画像ベクトルについて求められた差分ベクトルが供給されたかどうか判定する。
【０７７３】
ステップＳ３２３において、コードおよび差分ベクトルが供給されていないと判定された場合、ステップＳ３２１に戻り、以下、同様の処理を繰り返す。
【０７７４】
また、ステップＳ３２３において、コードおよび差分ベクトルが供給されたと判定された場合、ステップＳ３２４に進み、データ更新部５２１は、コードブックデータベース５０７の注目コードブックに登録された差分ベクトルの加算値Σ△_n、および前回の更新時から現在までの頻度Ｂ_nを更新する。
【０７７５】
即ち、データ更新部５２１は、注目コードブックの、ベクトル量子化部５０２から供給されたコード＃ｎのエントリを、注目エントリとして、その注目エントリにおける前回の更新時から現在までの頻度Ｂ_nを１だけインクリメントする。さらに、データ更新部５２１は、注目エントリにおける差分ベクトルの加算値Σ△_nに、差分演算部５０４から供給された差分ベクトルを加算し、その加算値を、新たな差分ベクトルの加算値Σ△_nとして、注目コードブックの注目エントリに上書きする。
【０７７６】
そして、ステップＳ３２１に戻り、以下、同様の処理が繰り返される。
【０７７７】
次に、図５９（Ｂ）のフローチャートを参照して、コードブック更新処理について説明する。
【０７７８】
コードブック更新処理は、定期または不定期に、任意のタイミングで開始される。
【０７７９】
なお、コードブック更新処理において、現に更新の対象となっている注目コードブックについては、その内容の整合性を保つため、図５９（Ａ）のデータ更新処理の対象から除外される（いわゆる排他制御される）ようになっている。
【０７８０】
コードブック更新処理では、まず最初に、ステップＳ３３１において、コードブック更新部５２２が、コードブック番号を表す変数ｉを、例えば、１に初期化し、ステップＳ３３２に進む。ステップＳ３３２では、コードブック更新部５２２は、コードブックデータベース５０７に記憶された１以上のコードブックのうち、ｉ番目のコードブックを、注目コードブックとして、ステップＳ３３３に進む。ステップＳ３３３では、コードブック更新部５２２が、注目コードブックにおけるコードを表す変数ｎを、例えば、０に初期化し、ステップＳ３３４に進む。
【０７８１】
ステップＳ３３４では、コードブック更新部５２２が、注目コードブックのコード＃ｎのエントリを注目エントリとして、例えば、その注目エントリの前回の更新時までの頻度Ａ_nおよび前回の更新時から現在までの頻度Ｂ_nを重みとして用い、注目エントリのコードベクトルＶ_nと、差分ベクトルの加算値Σ△_nとを重み付け加算することにより、注目エントリのコードベクトルＶ_nを更新する。
【０７８２】
即ち、コードブック更新部５２２は、例えば、次式にしたがって、注目エントリのコードベクトルＶ_nを更新する。
【０７８３】
Ｖ_n＝Ｖ_n＋Ｂ_n×Σ△_n／（Ａ_n＋Ｂ_n）
【０７８４】
そして、コードブック更新部５２２は、ステップＳ３３５に進み、注目エントリにおける前回の更新時までの頻度Ａ_n、差分ベクトルの加算値Σ△_n、および前回の更新時から現在までの頻度Ｂ_nを更新する。
【０７８５】
即ち、コードブック更新部５２２は、前回の更新時までの頻度Ａ_nと、前回の更新時から現在までの頻度Ｂ_nとを加算し、その加算値を、新たな前回の更新時までの頻度Ａ_nとする。さらに、コードブック更新部５２２は、差分ベクトルの加算値Σ△_n、および前回の更新時から現在までの頻度Ｂ_nを、いずれも０に初期化し、ステップＳ３３６に進む。
【０７８６】
ステップＳ３３６では、コードブック更新部５２２は、注目コードブックにおけるコードを表す変数ｎが、その最大値であるＮに等しいかどうかを判定する。ステップＳ３３６において、変数ｎがＮに等しくないと判定された場合、ステップＳ３３７に進み、コードブック更新部５２２は、変数ｎを１だけインクリメントする。そして、ステップＳ３３４に戻り、以下、同様の処理が繰り返される。
【０７８７】
また、ステップＳ３３６において、変数ｎがＮに等しいと判定された場合、即ち、注目コードブックのすべてのエントリを更新した場合、ステップＳ３３８に進み、変数ｉが、コードブックデータベース５０７に記憶されたコードブックの数を表すＩに等しいかどうかを判定する。
【０７８８】
ステップＳ３３８において、変数ｉがＩに等しくないと判定された場合、ステップＳ３３９に進み、コードブック更新部５２２は、変数ｉを１だけインクリメントする。そして、ステップＳ３３２に戻り、以下、同様の処理を繰り返す。
【０７８９】
一方、ステップＳ３３８において、変数ｉがＩに等しいと判定された場合、即ち、コードブックデータベース５０７に記憶されたすべてのコードブックの更新が終了した場合、処理を終了する。
【０７９０】
なお、上述したように、コードブックデータベース５０７においては、最初は、初期値のコードブックを１つだけ記憶しておき、その後、必要に応じて、その初期値のコードブックをコピーすることが可能であるが、この場合は、そのコピー元となるコードブックを更新してしまうと、その後に、初期値のコードブックのコピーを生成することができなくなる。そこで、コピー元となるコードブックは、図５９に示したデータ更新処理およびコードブック更新処理の対象外とされる。
【０７９１】
コードブックデータベース５０７に記憶されたコードブックは、上述のように、そのコードブックを用いて行われたベクトル量子化結果としてのコードと、そのコードに対応する差分ベクトル（そのコードが得られた画像ベクトルについて求められた差分ベクトル）とに基づいて更新されていくので、画質の向上を図ることが可能となる。
【０７９２】
即ち、ベクトル量子化部５０２でのベクトル量子化に用いられるコードブックは、上述したように、例えば、通信相手によって選択される。この場合、ベクトル量子化に用いられるコードブックは、通信相手ごとに異なり、従って、あるコードブックに注目すれば、その注目コードブックは、特定の通信相手に送信される画像データから得られるコードと差分ベクトルによって更新されることになる。
【０７９３】
その結果、注目コードブックは、特定の通信相手に送信することの多い画像の特性（例えば、周波数特性等）に応じて、差分ベクトルが０となる頻度が高くなるように更新されていくことになり、送信データのデータ量が低減されることになる。そして、この送信データのデータ量の低減により、上述したように、画質を向上させることができる。
【０７９４】
また、ベクトル量子化部５０２でのベクトル量子化に用いられるコードブックは、上述したように、例えば、そのベクトル量子化される画像データの絵柄を表すアクティビティによって選択される。この場合、ベクトル量子化に用いられるコードブックは、画像データの絵柄ごとに異なり、従って、あるコードブックに注目すれば、その注目コードブックは、特定の絵柄の画像データから得られるコードと差分ベクトルによって更新されることになる。
【０７９５】
その結果、注目コードブックは、特定の絵柄の画像について、差分ベクトルが０となる頻度が高くなるように更新されていくことになり、送信データのデータ量が低減されることになる。そして、この送信データのデータ量の低減により、上述したように、画質を向上させることができる。
【０７９６】
次に、図６０は、図４４の受信処理部４０２の第３の構成例を示している。即ち、図６０は、送信処理部４０１が図５５に示したように構成される場合の受信処理部４０２の構成例を示している。
【０７９７】
図５５のエントロピー符号化部５０５が出力する送信データは、受信装置としてのＰＤＡ１０３のアンテナ６４（図２３）で受信され、通信Ｉ／Ｆ２１８を介して、エントロピー復号部５３１と、エラー検出部５４０に供給されるようになっている。
【０７９８】
エントロピー復号部５３１は、そこに供給される受信データを、コードと差分ベクトルにエントロピー復号し、コードを、ベクトル逆量子化部５３２および更新部５３９に供給するとともに、差分ベクトルを、加算部５３３および更新部５３９に供給する。
【０７９９】
ベクトル逆量子化部５３２は、エントロピー復号部５３１から供給されるコードを、コードブック記憶部５３８に記憶されたコードブックを用いてベクトル逆量子化する。即ち、ベクトル逆量子化部５３２は、コードブック記憶部５３８に記憶されたコードブックにおけるコードベクトルのうち、エントロピー復号部５３１からのコードに対応するものを、ベクトル逆量子化結果として出力する。ベクトル逆量子化部５３２が出力するベクトル逆量子化結果としてのコードベクトルは、加算部５３３に供給される。
【０８００】
加算部５３３は、エントロピー復号部５３１から供給される差分ベクトルと、ベクトル逆量子化部４３２から供給されるコードベクトルとを加算し、これにより、元の画像ベクトルを復号する。この画像ベクトルは、エラー訂正部５３４に供給される。
【０８０１】
エラー訂正部５３４は、エラー検出部５４０から、画像ベクトルの符号化データとしてのコードや差分ベクトルについて、欠落等のエラーがある旨のエラーメッセージを受信すると、そのエラーを、コードブック記憶部５３８に記憶されたコードブックを参照して訂正する。そして、エラー訂正部５３４は、エラー訂正後の画像ベクトルを、スカラ化部５３５に供給する。
【０８０２】
スカラ化部５３５は、画像ベクトルの各コンポーネントを、画像の画素値として、元の位置に配置し、元の１フレームの画像データを構成するスカラ化を行い、その結果得られる画像データを出力する。
【０８０３】
コードブックデータベース５３６は、画像ベクトルをベクトル量子化して得られるコードをベクトル逆量子化するのに用いられる１以上のコードブックを記憶している。なお、コードブックデータベース５３６に記憶されたコードブックも、例えば、図５７に示したのと同一フォーマットとされている。
【０８０４】
選択部５３７には、選択情報が供給されるようになっており、選択部５３７は、その選択情報にしたがい、コードブックデータベース５３６に記憶された１以上のコードブックの中から、ベクトル量子化に用いるものを選択する。そして、選択部５３７は、その選択したコードブックを、コードブックデータベース５３６から読み出し、コードブック記憶部５３８に供給して上書きする形で記憶させる。
【０８０５】
ここで、選択情報としては、例えば、ユーザ入力や、通信相手の情報、エントロピー復号部５３１が出力するコードと差分ベクトルから復号された画像データ等を採用することができる。ユーザ入力は、ユーザが操作部２２４（図２３）等を操作することにより与えられる。また、通信相手の情報は、受信装置としてのＰＤＡ１０３が、送信装置としてのＰＤＡ１０１と通信を開始するときに、送信装置としてのＰＤＡ１０１から送信されることにより、アンテナ６４および通信Ｉ／Ｆ２１８（図２３）を介して与えられる。さらに、エントロピー復号部５３１が出力するコードと差分ベクトルから復号された画像データは、スカラ化部５３５からから与えられる。
【０８０６】
選択部５３７は、ユーザ入力が、選択情報として与えられる場合、そのユーザ入力にしたがって、コードブックデータベース５３６に記憶された１以上のコードブックから、１つのコードブックを選択する。従って、この場合、ベクトル逆量子化部５３２では、ユーザが指示したコードブックを用いて、ベクトル逆量子化が行われることになる。
【０８０７】
また、選択部５３７は、通信相手の情報が、選択情報として与えられる場合、その通信相手の情報にしたがって、コードブックデータベース５３６に記憶された１以上のコードブックから、１つのコードブックを選択する。従って、この場合、ベクトル逆量子化部５３２では、通信相手によって異なるコードブックを用いて、ベクトル逆量子化が行われることになる。
【０８０８】
さらに、選択部５３７は、画像データが、選択情報として与えられる場合、その画像データにしたがって、コードブックデータベース５３６に記憶された１以上のコードブックから、１つのコードブックを選択する。従って、この場合、ベクトル逆量子化部５３２では、復号される画像データの特性ごとに異なるコードブックを用いて、ベクトル逆量子化が行われることになる。
【０８０９】
なお、選択部５３７は、コードブックデータベース５３６に記憶された１以上のコードブックから、１つのコードブックを選択し、コードブック記憶部５３８に記憶させるとき、即ち、ベクトル逆量子化部５３２でベクトル量子化に用いられるコードブックを変更するとき、その変更後のコードブック（コードブックデータベース５３６から選択したコードブック）を特定するコードブック番号を、更新部５３９に供給する。
【０８１０】
更新部５３９は、図５８の更新部５３９と同様に構成され、エントロピー復号部５３１から供給されるコードと差分ベクトルとに基づいて、コードブックデータベース５３６に記憶されたコードブックを更新する。なお、上述したように、更新部５３９には、選択部５３７から、コードブック記憶部５３８に記憶されているコードブック、即ち、現在のベクトル逆量子化に用いられているコードブックを特定する情報としてのコードブック番号が供給されるようになっており、更新部５３９は、そのコードブック番号に基づいて、更新対象のコードブックを特定する。
【０８１１】
なお、コードブックデータベース５３６も、図５５のコードブックデータベース５０７と同様に、コードブックの初期値として、例えば、あらかじめ用意された学習用の多量の画像データを用い、ＬＢＧアルゴリズム等によって求められたコードブックを記憶しており、更新部５３９は、その初期値のコードブックを順次更新していく。
【０８１２】
また、コードブックデータベース５３６における、１以上のコードブックを記憶する記憶方法としては、図５５のコードブックデータベース５０７における場合と同様に、最初に、初期値のコードブックを１以上記憶しておくようにしても良いし、また、最初は、初期値のコードブックを１つだけ記憶しておき、その後、必要に応じて、その初期値のコードブックをコピーするようにしても良い。
【０８１３】
エラー検出部５４０は、受信データに生じているデータの欠落等のエラーを有無を検出し、エラーを検出した場合には、その旨のエラーメッセージを、エラー訂正部５３４に出力する。
【０８１４】
なお、例えば、図５５のエントロピー符号化部５０５は、送信データに、エラー検出用の符号（エラー検出符号）を付加するようになっており、エラー検出部５４０は、そのエラー検出符号に基づいて、エラーの有無を検出する。
【０８１５】
以上のように構成される受信処理部４０２では、受信データを受信して画像データに復号する画像データ受信処理、ベクトル逆量子化部５３２でのベクトル逆量子化に用いられるコードブックを選択するコードブック選択処理、およびコードブックデータベース５３６の記憶内容を更新する更新処理が行われる。
【０８１６】
但し、更新処理（データ更新処理とコードブック更新処理）は、更新部５３９において、エントロピー復号部５３１から供給されるコードおよび差分ベクトルを用いて、図５９のフローチャートで説明した場合と同様に行われるため、その説明は省略する。
【０８１７】
そこで、図６１のフローチャートを参照して、図６０の受信処理部４０２が行う画像データ受信処理およびコードブック選択処理について説明する。
【０８１８】
まず最初に、図６１（Ａ）のフローチャートを参照して、画像データ受信処理について説明する。
【０８１９】
画像データ受信処理は、受信データが、エントロピー復号部５３１およびエラー検出部５４０に供給されると開始される。
【０８２０】
即ち、画像データ受信処理では、まず最初に、ステップＳ３５１において、エントロピー復号部５３１が、受信データをエントロピー復号し、その結果得られるコードと差分ベクトルを出力する。コードは、ベクトル逆量子化部５３２および更新部５３９に供給され、差分ベクトルは、加算部５３３および更新部５３９に供給される。
【０８２１】
ここで、更新部５３９では、このようにして、エントロピー復号部５３１から供給されるコードと差分ベクトルに基づいて、図５９で説明したように、コードブックデータベース５３６のコードブックが更新される。
【０８２２】
従って、図５５の送信処理部４０１と、図６０の受信処理部４０２において、例えば、通信相手ごとに異なるコードブックが、ベクトル量子化とベクトル逆量子化それぞれに用いるコードブックとして選択されるとすると、あるユーザＡのＰＤＡ１０１から、他のユーザＢのＰＤＡ１０３に対して、画像データが送信される場合には、ユーザＡのＰＤＡ１０１における送信処理部４０１では、ユーザＢに対応するコードブックが、ベクトル量子化に用いられるとともに、ユーザＢのＰＤＡ１０３における受信処理部４０２では、ユーザＡに対応するコードブックが、ベクトル逆量子化に用いられることになる。
【０８２３】
その結果、送信処理部４０１（図５５）の更新部５０６と、受信処理部４０２（図６０）の更新部５３９とでは、ユーザＡとＢとの間で通信が行われるたびに、コードブックが、同じように更新される。即ち、ユーザＡとＢとの間では（他のユーザどうしの間でも同様）、基本的には、同一のコードブックを用いて、ベクトル量子化とベクトル逆量子化とがそれぞれ行われ、さらに、そのベクトル量子化で用いられるコードブックと、ベクトル逆量子化で用いられるコードブックとは、同一に更新されていく。
【０８２４】
そして、コードブックの更新は、図５９で説明したように、データ量を低減するように、または画質を向上させるように行われるから、図６０の受信処理部４０２では、画質の向上した画像データの復号を精度良く行うことができる。
【０８２５】
エントロピー復号部５３１において、例えば、１フレーム分の画像データについてのエントロピー復号が終了すると、ステップＳ３５２に進み、ベクトル量子化部５３２は、エントロピー復号部５３１から供給される１フレーム分のコードを、コードブック記憶部５３８に記憶されたコードブックを用いて、ベクトル逆量子化し、１フレーム分のコードそれぞれについて、コードベクトルを得る。このコードベクトルは、加算部５３３に供給される。
【０８２６】
加算部５３３は、ステップＳ３５３において、ベクトル逆量子化部５３２から供給される１フレーム分のコードベクトルそれぞれに、エントロピー復号部５３１から供給される１フレーム分の差分ベクトルの対応するものを加算することで、１フレーム分の画像ベクトルを復号する。加算部５３３において得られた１フレーム分の画像ベクトルは、順次、エラー訂正部５３４に供給される。
【０８２７】
エラー訂正部５３４は、ステップＳ３５４において、加算部５３３から供給される画像ベクトルに対して、後述するようなエラー訂正処理を施し、スカラ化部５３５に供給する。
【０８２８】
スカラ化部５３５は、ステップＳ３５５において、エラー訂正部５３４から供給される１フレーム分の画像ベクトルを、上述したようにスカラ化し、その結果得られる１フレームの画像データを出力して、ステップＳ３５６に進む。
【０８２９】
ステップＳ３５６では、エントロピー復号部５３１が、次のフレームの受信データが送信されてきたかどうかを判定し、送信されてきたと判定された場合、ステップＳ３５１に戻り、その次のフレームの受信データを対象に、以下、同様の処理が繰り返される。
【０８３０】
また、ステップＳ３５６において、次のフレームの受信データが送信されてきていないと判定された場合、処理を終了する。
【０８３１】
次に、図６１（Ｂ）のフローチャートを参照して、コードブック選択処理について説明する。
【０８３２】
なお、コードブック選択処理は、例えば、画像データ受信処理（図６１（Ａ））が開始される直前に開始される。
【０８３３】
コードブック選択処理では、まず最初に、ステップＳ３６１において、選択部５３７が、コードブックデータベース５３６に記憶された１以上のコードブックの中から、デフォルトのコードブックを選択し、コードブック記憶部５３８に供給して記憶させる。
【０８３４】
ここで、デフォルトのコードブックとしては、例えば、上述した初期値のコードブックを採用することができる。
【０８３５】
その後、ステップＳ３６２に進み、選択部５３７は、選択情報が供給されたかどうかを判定し、供給されていないと判定した場合、ステップＳ３６３をスキップして、ステップＳ３６４に進む。
【０８３６】
また、ステップＳ３６２において、選択情報が供給されたと判定された場合、ステップＳ３６３に進み、選択部５３７は、その選択情報にしたがい、コードブックデータベース５３６に記憶された１以上のコードブックの中から、ベクトル量子化に用いるものを選択し、コードブック記憶部５３８に供給して記憶させる。
【０８３７】
そして、ステップＳ３６４に進み、選択部５３７は、今回のコードブック選択処理を開始した直後に開始された画像データ受信処理（図６１（Ａ））による画像データの受信が終了したかどうかを判定し、終了していないと判定した場合、ステップＳ３６２に戻り、以下、同様の処理を繰り返す。
【０８３８】
従って、この場合、画像データ受信処理による画像データの受信が行われている間に、選択部５３７に対して、新たな選択情報が供給されると、コードブック記憶部５３８に記憶されるコードブック、即ち、ベクトル逆量子化部５３２におけるベクトル逆量子化で用いられるコードブックが、その新たな選択情報に基づいて変更され得る。
【０８３９】
一方、ステップＳ３６４において、画像データの受信が終了したと判定された場合、処理を終了する。
【０８４０】
なお、図６１（Ｂ）の実施の形態では、上述したように、画像データ受信処理による画像データの送信が行われている間に、選択部５３７に対して、新たな選択情報が供給された場合に、ベクトル逆量子化で用いられるコードブックが、その新たな選択情報に基づいて、随時変更され得るようにしたが、画像データ受信処理による画像データの受信が行われている間のコードブックの変更は、１回だけに制限することが可能である。即ち、コードブックの変更は、デフォルトのコードブックから、最初に供給された選択情報に基づくコードブックへの変更のみに制限することが可能である。
【０８４１】
次に、図６２は、図６０のエラー訂正部５３４の構成例を示している。
【０８４２】
加算部５３３が出力する画像ベクトルは、書き込み部５５０に供給されるようになっており、書き込み部５５０は、その画像ベクトルを、メモリ５５１に供給して書き込む。メモリ５５１は、書き込み部５５０から供給される画像ベクトルを記憶する。
【０８４３】
ここで、メモリ５５１は、例えば、少なくとも１フレーム分の画像ベクトルを記憶することのできる記憶容量を有している。
【０８４４】
また、画像ベクトルは、上述したように、例えば、３×３画素のブロックにおける９画素の画素値をコンポーネントとしたベクトルであるが、書き込み部５５０は、画像ベクトルを、その画像ベクトルとしてのブロックに対応するメモリ５５１のアドレスに書き込むようになっている。
【０８４５】
読み出し部５５２は、メモリ５５１に、例えば、１フレーム分の画像ベクトルが記憶されると、その１フレーム分の画像ベクトルを、メモリ５５１から読み出し、スカラ化部５３５（図６０）に供給する。
【０８４６】
また、読み出し部５５２には、エラー検出部５４０（図６０）からエラーメッセージが供給されるようになっており、読み出し部５５２は、そのエラーメッセージに基づき、部分ベクトル推定部５５３を制御する。
【０８４７】
即ち、エラー検出部５４０が出力するエラーメッセージには、エラーが生じている画像ベクトルとしてのブロックの位置を表す情報が含まれており、読み出し部５５２は、そのエラーが生じている画像ベクトル（以下、適宜、エラーベクトルという）について、後述する部分ベクトルを推定するように、部分ベクトル推定部５５３を制御する。
【０８４８】
部分ベクトル推定部５５３は、読み出し部５５２の制御にしたがい、エラーベクトルについて、例えば、図６３に示すように、その一部のコンポーネントからなる部分ベクトルを推定する。
【０８４９】
即ち、いまの場合、画像ベクトルは、図６３（Ａ）に示すように、３×３画素からなるブロックを構成する９画素の画素値をコンポーネントとするベクトルであるから、画像ベクトル（画像ベクトルから得られたコードや差分ベクトル）が欠落等しているということは、図６３（Ｂ）に示すように、その画像ベクトルとしてのブロックの３×３画素を復号することができなことになる。
【０８５０】
そこで、部分ベクトル推定部５５３では、図６３（Ｃ）に示すように、エラーベクトルとしてのブロック（以下、適宜、エラーブロックという）の、他のブロックの画素と隣接する８画素を、その隣接する画素により補完する。
【０８５１】
即ち、図６３（Ｃ）では、エラーブロックの３×３画素のうち、中心の画素ｐ₉を除く８画素ｐ₁乃至ｐ₈が、それぞれに隣接する他のブロックの画素によって補完されている。なお、中心の画素ｐ₉を除く８画素ｐ₁乃至ｐ₈のうち、他のブロックの１画素にのみ隣接するｐ₂，ｐ₄，ｐ₆，ｐ₈には、例えば、それぞれに隣接する他のブロックの画素の画素値がコピーされる。また、他のブロックの２画素に隣接する画素ｐ₁，ｐ₃，ｐ₅，ｐ₇には、例えば、それぞれに隣接する他のブロックの２つの画素の平均値、またはいずれか一方の値がコピーされる。
【０８５２】
部分ベクトル推定部５５３は、以上のように、エラーベクトルについて、ブロックの中心の画素ｐ₉に対応するコンポーネントがないベクトル、即ち、本来ならば９画素の画素値をコンポーネントとするベクトルについて、ブロックの中心の画素ｐ₉に対応するコンポーネントがない部分ベクトルを生成する。そして、部分ベクトル推定部５５３は、その部分ベクトルを、エラーベクトルに対応する真の画像ベクトルの一部（部分ベクトル）の推定値として出力する。
【０８５３】
図６２に戻り、部分ベクトル推定部５５３が出力する部分ベクトルは、画像ベクトル推定部５５４に供給される。
【０８５４】
画像ベクトル推定部５５４は、部分ベクトル推定部５５３からの部分ベクトルと、コードブック記憶部５３８に記憶されているコードブックとから、エラーベクトルに対応する真の画像ベクトルを推定することによりエラー訂正を行い、その推定した画像ベクトル（以下、適宜、推定ベクトルという）を、メモリ５５１の対応するアドレスに記憶させる。
【０８５５】
次に、図６４のフローチャートを参照して、図６２のエラー訂正部５３４が、図６１（Ａ）のステップＳ３５４において行うエラー訂正処理について説明する。
【０８５６】
エラー訂正処理では、まず最初に、ステップＳ３７１において、書き込み部５５０が、画像ベクトルが、加算部５３３（図６０）から供給されたかどうかを判定する。ステップＳ３７１において、画像ベクトルが供給されたと判定された場合、ステップＳ３７２に進み、書き込み部５５０は、その画像ベクトルを、メモリ５５１に供給して記憶させ、ステップＳ３７５に進む。
【０８５７】
また、ステップＳ３７１において、画像ベクトルが供給されていないと判定された場合、ステップＳ３７３に進み、読み出し部５５２は、エラー検出部５４０からエラーメッセージが供給されたかどうかを判定する。
【０８５８】
ステップＳ３７３において、エラーメッセージが供給されていないと判定された場合、ステップＳ３７４をスキップして、ステップＳ３７５に進む。
【０８５９】
また、ステップＳ３７３において、エラーメッセージが供給されたと判定された場合、ステップＳ３７４に進み、読み出し部５５２は、そのエラーメッセージに基づき、エラーが生じているブロック（画像ベクトル）の位置（以下、適宜、エラー位置という）を認識し、その内蔵するメモリ（図示せず）に一時記憶する。
【０８６０】
そして、ステップＳ３７５に進み、読み出し部５５２は、１フレーム分の画像ベクトル（エラーベクトルを含む）が、メモリ５５１に記憶されたかどうかを判定する。
【０８６１】
ステップＳ３７５において、１フレーム分の画像ベクトルがメモリ５５１に記憶されていないと判定された場合、ステップＳ３７１に戻り、以下、同様の処理を繰り返す。
【０８６２】
また、ステップＳ３７５において、１フレーム分の画像ベクトルがメモリ５５１に記憶されたと判定された場合、ステップＳ３７６に進み、読み出し部５５２は、その内蔵するメモリに、エラー位置が記憶されているかどうかを判定する。
【０８６３】
ステップＳ３７６において、エラー位置が記憶されていると判定された場合、ステップＳ３７７に進み、読み出し部５５２は、その内蔵するメモリに記憶されている１以上のエラー位置のうちの１つを注目エラー位置とし、その注目エラー位置におけるエラーベクトルついて、部分ベクトルを推定するように、部分ベクトル推定部５５３を制御する。これにより、ステップＳ３７７では、部分ベクトル推定部５５３が、図６３で説明したように、部分ベクトルを推定し、画像ベクトル推定部５５４に出力する。
【０８６４】
画像ベクトル推定部５５４は、ステップＳ３７８において、部分ベクトル推定部５５３からの部分ベクトルと、コードブック記憶部５３８（図６０）に記憶されているコードブックとから、エラーベクトルに対応する真の画像ベクトルを推定するエラー訂正を行い、ステップＳ３７９に進み、そのエラー訂正により得られる推定ベクトルを、メモリ５５１の注目エラー位置に対応するアドレスに記憶させる。
【０８６５】
即ち、画像ベクトル推定部５５４は、部分ベクトルの各コンポーネント（ここでは、図６３で説明したように、８つのコンポーネントそれぞれ）に対応するコンポーネントが、部分ベクトルの各コンポーネントに最も近似するコードベクトルを、コードブック記憶部５３８に記憶されているコードブックから抽出し、そのコードベクトルを、推定ベクトルとする。
【０８６６】
具体的には、画像ベクトル推定部５５４は、例えば、部分ベクトルの各コンポーネントと、コードブックのコードベクトルの対応するコンポーネントとの差分の自乗和を計算し、その自乗和を最小にするコードベクトルを検出する。そして、画像ベクトル推定部５５４は、そのコードベクトルを、注目エラー位置の画像ベクトルの推定ベクトルとし、メモリ５５１に書き込む。
【０８６７】
その後、読み出し部５５２は、注目エラー位置を、その内蔵するメモリから消去して、ステップＳ３７６に戻り、以下、エラー位置が、その内蔵するメモリに記憶されていない状態となるまで、ステップＳ３７６乃至Ｓ３７９の処理を繰り返す。
【０８６８】
一方、ステップＳ３７６において、エラー位置が記憶されていないと判定された場合、即ち、メモリ５５１に、１フレーム分の画像ベクトルがエラーのない状態（エラーの訂正がされた状態）で記憶された場合、ステップＳ３８０に進み、読み出し部５５１は、メモリ５５１から、その１フレーム分の画像ベクトルを読み出し、スカラ化部５３５（図６０）に供給して、エラー訂正処理を終了する。
【０８６９】
なお、図５５乃至図６４で説明したようなコードブックを用いて符号化／復号を行って、データを送受信する通信は、ＰＤＡ１０１と他のＰＤＡ１０３との間だけではなく、ベース基地コンピュータ１０２（図２２）と、ＰＤＡ１０３や、他の図示せぬベース基地コンピュータとの間で行うことも可能である。
【０８７０】
ところで、あるユーザＡが、ＰＤＡ１０１とベース基地コンピュータ１０２を所有し、他のユーザＢが、ＰＤＡ１０３を所有する場合には、ユーザＡが、ユーザＢのＰＤＡ１０３とデータのやりとりを、ＰＤＡ１０１を用いて行うときもあれば、ベース基地コンピュータ１０２を用いて行うときもある。
【０８７１】
このように、ＰＤＡ１０１またはベース基地コンピュータ１０２それぞれと、ユーザＢのＰＤＡ１０３との間でデータのやりとりをすると、ＰＤＡ１０１とベース基地コンピュータ１０２の両方に、ユーザＢとの通信用のコードブックが生成される。
【０８７２】
しかしながら、ＰＤＡ１０１と１０３との間でやりとりされるデータと、ベース基地コンピュータ１０２とＰＤＡ１０３との間でやりとりされるデータとは、同一であるとは限らず、むしろ異なるデータであることが多いため、ＰＤＡ１０１とベース基地コンピュータ１０２とでは、ユーザＢとの通信用のコードブックとして、異なるコードブックが生成される。
【０８７３】
そこで、図６５に示すように、ＰＤＡ１０１におけるコードブックと、ベース基地コンピュータ１０２におけるコードブックとは、統合して同一のコードブックとすることができるようになっている。
【０８７４】
即ち、いま、図６５に示すように、ＰＤＡ１０１が、図５５に示した送信処理部４０１と、図５０に示した受信処理部４０２とを有し、ベース基地コンピュータ１０２が、図５５の送信処理部４０１と同一構成の送信処理部４０１’と、図５０の受信処理部４０２と同一構成の受信処理部４０２’とを有するとした場合には、ＰＤＡ１０１とベース基地コンピュータ１０２とが、上述したようにして通信を行うことで、それぞれが有するコードブックがやりとりされる。これにより、ＰＤＡ１０１の送信処理部４０１とベース基地コンピュータ１０２の送信処理部４０１’とにおける対応するコードブックが、同一のコードブックに統合される。さらに、ＰＤＡ１０１の受信処理部４０２とベース基地コンピュータ１０２の受信処理部４０２’とにおける対応するコードブックも、同一のコードブックに統合される。
【０８７５】
なお、２つのコードブックの統合方法としては、例えば、一方のコードブックのコードベクトルを、統合後のコードブックのコードベクトルとして採用する方法や、２つのコードブックのコードベクトルの平均値（平均ベクトル）を、統合後のコードブックのコードベクトルとして採用する方法その他がある。
【０８７６】
なお、図４５乃至図６５の実施の形態においては、画像の画質を向上させる場合について説明したが、図４５乃至図６５の実施の形態は、その他、例えば、音声の音質を向上させる場合にも適用可能である。
【０８７７】
次に、ＰＤＡ１０１においては、他の１以上のＰＤＡから、データを高品質化する高品質化データを取得し、その高品質化データと、既に有している高品質化データとに基づいて、新たな高品質化データを生成し、その新たな高品質化データによって、既に有している高品質化データを更新することができる。さらに、ＰＤＡ１０１において、その更新後の高品質化データ（新たな高品質化データ）に基づき、データを処理し、より高品質なデータを求めることができる。
【０８７８】
図６６は、そのようなＰＤＡ１０１の機能的構成例を示している。なお、ここでは、高品質化の対象とするデータを、音声データとして、以下、説明を行う。但し、高品質化の対象とするデータは、音声データ以外の、例えば画像データとすることも可能である。
【０８７９】
また、図６６においては、図２３のハードウエア構成と対応する部分については、同一の符号を付してあり、以下では、その説明は、適宜省略する。
【０８８０】
音声復号部６００は、例えば、ＨＤＤ２１５（図２３）に記憶された、符号化されている音声データを復号し、復号音声データを、音質向上部６０１に供給する。
【０８８１】
音質向上部６０１は、学習部６０２から供給される高品質化データと、復号音声データとに基づき、その復号音声データの音質を向上させた音質向上データを求め、Ｄ／Ａ変換器２１２に供給する。
【０８８２】
学習部６０２は、アンプ２０９から供給される音声データに基づき、学習を行うことで、高品質化データを求める。さらに、学習部６０２は、他の１以上のＰＤＡから送信されてくる、後述する学習情報を受信し、必要に応じて、その学習情報にも基づいて学習を行うことで、新たな高品質化データを求める。そして、学習部６０２は、新たに求めた高品質化データを、音質向上部６０１に供給する。
【０８８３】
また、学習部６０２には、操作部２２４（図２３）から操作信号が供給されるようになっている。学習部６０２は、操作部２２４からの操作信号が、他のＰＤＡに対して、学習情報を要求する操作を表すものである場合、学習情報を要求するリクエスト信号を生成して出力する。このリクエスト信号は、通信Ｉ／Ｆ２１８（図２３）を介して、例えば、アンテナ６４から送信される。
【０８８４】
さらに、学習部６０２には、他のＰＤＡからのリクエスト信号が供給されるようになっている。即ち、他のＰＤＡがリクエスト信号を送信すると、そのリクエスト信号は、アンテナ６４（図２３）で受信され、通信Ｉ／Ｆ２１８を介して、学習部６０２に供給される。学習部６０２は、他のＰＤＡからのリクエスト信号を受信すると、自身が有している学習情報を、通信Ｉ／Ｆ２１８およびアンテナ６４を介して、リクエスト信号を送信してきた他のＰＤＡに送信する。
【０８８５】
次に、図６７は、図６６の音質向上部６０１の構成例を示している。
【０８８６】
図６７の実施の形態においては、音質向上部６０１は、例えば、上述した適応処理によって、音声復号部６００からの復号音声データの音質を向上させるようになっており、従って、ここでは、高品質化データとして、タップ係数が用いられるようになっている。
【０８８７】
即ち、音声復号部６００が出力する復号音声データは、バッファ６１１に供給されるようになっており、バッファ６１１は、そこに供給される復号音声データを一時記憶する。
【０８８８】
予測タップ抽出部６１２は、復号音声データの音質を向上させた音質向上データを、順次、注目データとして、その注目データの予測値を、式（１）の線形一次予測演算により求めるのに用いる予測タップを、バッファ６１１に記憶された復号音声データのうちの幾つかの音声サンプルを抽出することによって構成（生成）し、積和演算部６１６に供給する。
【０８８９】
なお、予測タップ抽出部６１２は、後述する図６９の予測タップ抽出部６２４が構成するのと同一の予測タップを構成する。
【０８９０】
クラスタップ抽出部６１３は、バッファ６１１に記憶された復号音声データのうちの幾つかの音声サンプルを抽出することによって、注目データについて、クラスタップを構成（生成）し、クラス分類部６１４に供給する。
【０８９１】
なお、クラスタップ抽出部６１３は、後述する図６９のクラスタップ抽出部６２５と同一のクラスタップを構成する。
【０８９２】
クラス分類部６１４は、クラスタップ抽出部６１３からのクラスタップを用いてクラス分類を行い、その結果得られるクラスコードを、係数メモリ６１５に供給する。
【０８９３】
なお、クラス分類部６１４は、後述する図６９のクラス分類部６２６と同一のクラス分類を行う。
【０８９４】
係数メモリ６１５は、学習部６０２から供給される高品質化データとしてのクラスごとのタップ係数を、そのクラスに対応するアドレスに記憶する。さらに、係数メモリ６１５は、クラス分類部６１４から供給されるクラスコードに対応するアドレスに記憶されているタップ係数を、積和演算部６１６に供給する。
【０８９５】
積和演算部６１６は、予測タップ抽出部６１２が出力する予測タップと、係数メモリ６１５が出力するタップ係数とを取得し、その予測タップとタップ係数とを用いて、式（１）に示した線形予測演算を行う。これにより、積和演算部６１６は、注目データとしての音質向上データ（の予測値）を求め、Ｄ／Ａ変換器２１２（図６６）に供給する。
【０８９６】
次に、図６８のフローチャートを参照して、図６７の音質向上部６０１の処理（音質向上処理）について説明する。
【０８９７】
バッファ６１１は、音声復号部６００（図６６）が出力する復号音声データを、順次記憶していく。
【０８９８】
そして、まず最初に、ステップＳ４０１において、予測タップ抽出部６１２が、復号音声データの音質を向上させた音質向上データのうち、例えば、時系列順で、まだ注目データとしていない最も過去の音声サンプルを、注目データとし、その注目データについて、バッファ６１１から復号音声データのうちの幾つかの音声サンプルを読み出すことにより、予測タップを構成して、積和演算部６１６に供給する。
【０８９９】
さらに、ステップＳ４０１では、クラスタップ抽出部６１３が、バッファ６１１に記憶された復号音声データのうちの幾つかの音声サンプルを読み出すことにより、注目データについて、クラスタップを構成し、クラス分類部６１４に供給する。
【０９００】
クラス分類部６１４は、クラスタップ抽出部６１３からクラスタップを受信すると、ステップＳ４０２に進み、そのクラスタップを用いてクラス分類を行い、その結果得られるクラスコードを、係数メモリ６１５に供給して、ステップＳ４０３に進む。
【０９０１】
ステップＳ４０３では、係数メモリ６１５は、クラス分類部６１４からのクラスコードに対応するアドレスに記憶されているタップ係数を読み出し、積和演算部６１６に供給して、ステップＳ４０４に進む。
【０９０２】
ステップＳ４０４では、積和演算部６１６は、係数メモリ６１５が出力するタップ係数を取得し、そのタップ係数と、予測タップ抽出部６１２からの予測タップとを用いて、式（１）に示した積和演算を行い、音質向上データ（の予測値）を得る。
【０９０３】
以上のようにして得られた音質向上データは、積和演算部６１６から、Ｄ／Ａ変換器２１２（図６６）およびアンプ２０８を介して、スピーカ１０等に供給され、これにより、スピーカ１０等からは、高音質の音声が出力される。
【０９０４】
ステップＳ４０４の処理後は、ステップＳ４０５に進み、まだ、注目データとして処理すべき音質向上データがあるかどうかが判定され、あると判定された場合、ステップＳ４０１に戻り、以下、同様の処理が繰り返される。また、ステップＳ４０５において、注目データとして処理すべき音質向上データがないと判定された場合、処理を終了する。
【０９０５】
次に、図６９は、図６６の学習部６０２の構成例を示している。
【０９０６】
教師データメモリ６２１には、アンプ２０９（図６６）が出力する音声データが、学習用のデータとして供給されるようになっており、教師データメモリ６２１は、その音声データを、学習の教師となる教師データとして一時記憶する。
【０９０７】
生徒データ生成部６２２は、教師データメモリ６２１に記憶された教師データとしての音声データから、学習の生徒となる生徒データを生成する。
【０９０８】
即ち、生徒データ生成部６２２は、音声符号化部６２２Ｅと音声復号部６２２Ｄとから構成されている。音声符号化部６２２Ｅは、音声復号部６００（図６６）における復号方式に対応する符号化方式によって音声データを符号化するもので、そのような符号化方式によって、教師データメモリ６２１に記憶された教師データを符号化し、符号化音声データを出力する。音声復号部６２２Ｄは、音声復号部６００と同様に構成されており、音声符号化部６２２Ｅが出力する符号化音声データを復号し、その結果得られる復号音声データを、生徒データとして出力する。
【０９０９】
なお、ここでは、教師データを、符号化音声データに符号化し、さらに、その符号化音声データを復号することによって、生徒データを生成するようにしたが、その他、生徒データは、例えば、教師データとしての音声データを、ローパスフィルタによってフィルタリングすること等で、その音質を劣化させることによって生成すること等が可能である。
【０９１０】
生徒データメモリ６２３は、生徒データ生成部６２２の音声復号部６２２Ｄが出力する生徒データを一時記憶する。
【０９１１】
予測タップ抽出部６２４は、教師データメモリ６２１に記憶された教師データの音声サンプルを、順次、注目データとし、さらに、その注目データを予測するのに用いる生徒データとしての幾つかの音声サンプルを、生徒データメモリ６２３に記憶された生徒データから抽出することにより、予測タップ（注目データの予測値を求めるためのタップ）を生成する。この予測タップは、予測タップ抽出部６２４から足し込み部６２７に供給される。
【０９１２】
クラスタップ抽出部６２５は、注目データをクラス分けするクラス分類に用いる生徒データとしての幾つかの音声サンプルを、生徒データメモリ６２３に記憶された生徒データから抽出することにより、クラスタップ（クラス分類に用いるタップ）を生成する。このクラスタップは、クラスタップ抽出部６２５からクラス分類部６２６に供給される。
【０９１３】
ここで、予測タップやクラスタップを構成する音声サンプルとしては、例えば、注目データとなっている教師データの音声サンプルに対応する生徒データの音声サンプルに対して時間的に近い位置にある複数の生徒データとしての音声サンプルを用いることができる。
【０９１４】
また、予測タップとクラスタップを構成する音声サンプルとしては、同一の音声サンプルを用いることもできるし、異なる音声サンプルを用いることも可能である。
【０９１５】
クラス分類部６２６は、クラスタップ抽出部６２５からのクラスタップに基づき、注目データをクラス分類し、その結果得られるクラスに対応するクラスコードを、足し込み部６２７に出力する。
【０９１６】
なお、クラス分類部６２６におけるクラス分類の方法としては、例えば、上述した場合と同様に、ADRC等を採用することができる。
【０９１７】
足し込み部６２７は、教師データメモリ６２１から、注目データとなっている教師データの音声サンプルを読み出し、予測タップ抽出部６２４からの予測タップを構成する生徒データ、および注目データとしての教師データを対象とした足し込みを、コンポーネントデータベース６３０の記憶内容を必要に応じて用いながら、クラス分類部６２６から供給されるクラスごとに行う。
【０９１８】
即ち、足し込み部６２７は、基本的には、クラス分類部６２６から供給されるクラスコードに対応するクラスごとに、予測タップ（生徒データ）を用い、式（８）の行列Ａにおける各コンポーネントとなっている、生徒データどうしの乗算（ｘ_inｘ_im）と、サメーション（Σ）に相当する演算を行う。
【０９１９】
さらに、足し込み部６２７は、やはり、クラス分類部６２６から供給されるクラスコードに対応するクラスごとに、予測タップ（生徒データ）および注目データ（教師データ）を用い、式（８）のベクトルｖにおける各コンポーネントとなっている、生徒データと教師データの乗算（ｘ_inｙ_i）と、サメーション（Σ）に相当する演算を行う。
【０９２０】
一方、コンポーネントデータベース６３０は、足し込み部６２７において前回の学習で求められた式（８）における行列Ａのコンポーネントと、ベクトルｖのコンポーネントを、クラスごとに記憶する。
【０９２１】
足し込み部６２７は、新たに入力された音声データを用いて学習を行う場合、コンポーネントデータベース６３０に記憶された、前回の学習で求められた式（８）における行列Ａのコンポーネントと、ベクトルｖのコンポーネントを読み出し、その行列Ａまたはベクトルｖのコンポーネントに対して、新たに入力された音声データから得られる教師データおよび生徒データを用いて計算される、対応するコンポーネントｘ_inｘ_imまたはｘ_inｙ_iを足し込むことにより（行列Ａ、ベクトルｖにおけるサメーションで表される加算を行うことにより）、新たな行列Ａとベクトルｖのコンポーネントを求め、各クラスについて、式（８）に示した正規方程式をたてる。
【０９２２】
従って、足し込み部６２７では、新たに入力された音声データだけではなく、過去の学習に用いられた音声データにも基づいて、式（８）の正規方程式がたてられる。
【０９２３】
足し込み部６２７は、上述のようにして、新たに入力された音声データから得られた行列Ａおよびベクトルｖのコンポーネントと、コンポーネントデータベース６３０に記憶された行列Ａおよびベクトルｖのコンポーネントと用いて、新たに、クラスごとの行列Ａおよびベクトルｖのコンポーネントを求めると、それらのコンポーネントを、コンポーネントデータベース６３０に供給し、上書きする形で記憶させる。
【０９２４】
また、足し込み部６２７は、音声データが新たに入力されていない場合であっても、必要に応じて、コンポーネントデータベース６３０に記憶された各クラスの行列Ａのコンポーネントと、ベクトルｖのコンポーネントを読み出し、その行列Ａまたはベクトルｖのコンポーネントを用いて、各クラスについて、式（８）に示した正規方程式をたてる。
【０９２５】
即ち、コンポーネントデータベース６３０に記憶された式（８）における各クラスの行列Ａのコンポーネントおよびベクトルｖのコンポーネントは、足し込み部６２７において、新たに入力された音声データを用いて学習が行われることにより、上述したように、その学習の途中で得られる新たな各クラスの行列Ａのコンポーネントおよびベクトルｖのコンポーネントによって更新される（上書きされる）他、後述する統合部６３１により更新されるようにもなっている。
【０９２６】
このように、コンポーネントデータベース６３０に記憶された式（８）における各クラスの行列Ａのコンポーネントおよびベクトルｖのコンポーネントが、統合部６３１によって更新された場合には、足し込み部６２７は、コンポーネントデータベース６３０に記憶された、更新後の各クラスの行列Ａのコンポーネントと、ベクトルｖのコンポーネントを読み出し、その行列Ａまたはベクトルｖのコンポーネントを用いて、各クラスについて、式（８）に示した正規方程式をたてるようになっている。
【０９２７】
足し込み部６２７は、上述のようにして、クラスごとの行列Ａおよびベクトルｖのコンポーネントで構成される式（８）の正規方程式を新たにたてると、そのクラスごとの正規方程式を、タップ係数決定部６２８に供給する。
【０９２８】
タップ係数決定部６２８は、足し込み部６２７から供給されるクラスごとの正規方程式を解くことにより、クラスごとに、タップ係数を求め、このクラスごとのタップ係数を、高品質化データとして、タップ係数メモリ６２９に供給し、各クラスに対応するアドレスに、上書きする形で記憶させる。
【０９２９】
タップ係数メモリ６２９に記憶された高品質化データとしてのクラスごとのタップ係数は、音質向上部６０１（図６６）に供給されるようになっている。
【０９３０】
コンポーネントデータベース６３０は、上述したように、クラスごとの式（８）における行列Ａおよびベクトルｖのコンポーネントを記憶する。
【０９３１】
統合部６３１は、他のＰＤＡから送信されてくる、学習情報としての、クラスごとの式（８）における行列Ａおよびベクトルｖのコンポーネントを、アンテナ６４（図２３）および通信Ｉ／Ｆ２１８を介して受信する。そして、統合部６３１は、受信した各クラスの行列Ａおよびベクトルｖのコンポーネントと、コンポーネントデータベース６３０に記憶された各クラスの行列Ａおよびベクトルｖのコンポーネントと用いて、新たな各クラスの行列Ａおよびベクトルｖのコンポーネントを求める。
【０９３２】
即ち、統合部６３１は、他のＰＤＡから各クラスの行列Ａおよびベクトルｖのコンポーネントを受信すると、コンポーネントデータベース６３０に記憶されている各クラスの行列Ａのコンポーネントと、ベクトルｖのコンポーネントを読み出し、その各クラスの行列Ａまたはベクトルｖのコンポーネントに対して、対応するクラスの、他のＰＤＡから受信した行列Ａまたはベクトルｖのコンポーネントをそれぞれ足し込むことにより（行列Ａ、ベクトルｖにおけるサメーションで表される加算を行うことにより）、各クラスについて、新たな行列Ａのコンポーネントと、ベクトルｖのコンポーネントを求める。
【０９３３】
そして、統合部６３１は、各クラスについて求めた新たな行列Ａおよびベクトルｖのコンポーネントを、コンポーネントデータベース６３０に上書きする形で記憶させることにより、そのコンポーネントデータベース６３０に記憶されている各クラスの行列Ａおよびベクトルｖのコンポーネントを更新する。
【０９３４】
学習情報送信部６３２は、イベント検出部６３３からの要求に応じて、コンポーネントデータベース６３０から各クラスの行列Ａおよびベクトルｖのコンポーネントを読み出し、通信Ｉ／Ｆ２１８（図２３）およびアンテナ６４を介して、学習情報として送信する。
【０９３５】
ここで、上述の統合部６３１では、他のＰＤＡにおける学習情報送信部６３２から送信されてくる学習情報としての各クラスの行列Ａおよびベクトルｖのコンポーネントが受信される。
【０９３６】
イベント検出部６３３は、他のＰＤＡから、学習情報の送信を要求するリクエスト信号が送信されてくると、そのリクエスト信号が送信されてきたことを、所定のイベントして検出する。即ち、他のＰＤＡから送信されてくるリクエスト信号は、アンテナ６４（図２３）を介して、通信Ｉ／Ｆ２１８で受信され、イベント検出部６３３に供給される。イベント検出部６３３は、このようにして供給されるリクエスト信号を検出すると、所定のイベントが発生したとして、コンポーネントデータベース６３０に記憶されている学習情報としての各クラスの行列Ａおよびベクトルｖのコンポーネントを送信するように、学習情報送信部６３２を制御する。
【０９３７】
リクエスト信号送信部６３４には、操作部２２４（図２３）等から操作信号が供給されるようになっており、リクエスト信号送信部６３４は、学習情報を要求するように、操作部２２４の操作が行われたことを表す操作信号を受信すると、その操作信号の受信を、所定のイベントとして、学習情報を要求するリクエスト信号を、通信Ｉ／Ｆ２１８（図２３）およびアンテナ６４を介して送信する。
【０９３８】
ここで、このようにして送信されるリクエスト信号を受信した１以上の他のＰＤＡでは、そのリクエスト信号の受信（検出）により所定のイベントが発生したとして、自身が有する学習情報としての各クラスの行列Ａおよびベクトルｖのコンポーネントの送信が行われる。そして、このようにして他の１以上のＰＤＡから送信されてくる学習情報としての各クラスの行列Ａおよびベクトルｖのコンポーネントが、統合部６３１において受信されることになる。
【０９３９】
以上のように構成される学習部６０２では、新たに入力された音声データを用いて、クラスごとのタップ係数を求める学習処理、コンポーネントデータベース６３０に記憶された学習情報としての各クラスの行列Ａおよびベクトルｖのコンポーネントを送信する学習情報送信処理、および他のＰＤＡから送信されてきた学習情報としての各クラスの行列Ａおよびベクトルｖのコンポーネントによって、コンポーネントデータベース６３０に記憶された学習情報を更新するコンポーネントデータ統合処理が行われるようになっている。
【０９４０】
そこで、図７０のフローチャートを参照して、図６９の学習部６０２が行う学習処理、学習情報送信処理、およびコンポーネントデータ統合処理について説明する。
【０９４１】
まず、図７０（Ａ）のフローチャートを参照して、学習処理について説明する。
【０９４２】
学習処理は、例えば、教師データメモリ６２１に、所定のデータ量以上の新たな音声データが記憶されると開始される。
【０９４３】
即ち、教師データメモリ６２１に、新たな音声データが、所定のデータ量以上記憶されると、ステップＳ４１１において、足し込み部６２７が、コンポーネントデータベース６３０に記憶されている各クラスの行列Ａおよびベクトルｖのコンポーネントを読み出し、ステップＳ４１２に進む。
【０９４４】
ステップＳ４１２では、生徒データ生成部６２２が、教師データメモリ６２１に記憶された音声データを教師データとして、その教師データを読み出し、その教師データから生徒データを生成する。そして、生徒データ生成部６２２は、得られた生徒データを、生徒データメモリ６２３に供給して記憶させ、ステップＳ４１３に進む。
【０９４５】
ステップＳ４１３では、予測タップ抽出部６２４は、教師データメモリ６２１に記憶された教師データとしての音声サンプルのうち、まだ注目データとしていないものの１つを注目データとして、その注目データについて、生徒データメモリ６２３に記憶された生徒データとしての音声サンプルの幾つかを読み出すことにより、予測タップを生成して、足し込み部６２７に供給する。
【０９４６】
さらに、ステップＳ４１３では、クラスタップ抽出部６２５が、予測タップ抽出部６２４における場合と同様にして、注目データについて、クラスタップを生成し、クラス分類部６２６に供給する。
【０９４７】
ステップＳ４１３の処理後は、ステップＳ４１４に進み、クラス分類部６２６が、クラスタップ抽出部６２５からのクラスタップに基づいて、クラス分類を行い、その結果得られるクラスコードを、足し込み部６２７に供給する。
【０９４８】
そして、ステップＳ４１５に進み、足し込み部６２７は、教師データメモリ６２１から注目データを読み出し、その注目データと、予測タップ抽出部６２４からの予測タップを用いて、行列Ａとベクトルｖのコンポーネントを計算する。さらに、足し込み部６２７は、コンポーネントデータベース６３０から読み出した行列Ａとベクトルｖのコンポーネントのうち、クラス分類部６２６からのクラスコードに対応するものに対して、注目データと予測タップから求められた行列Ａとベクトルｖのコンポーネントを足し込み、これにより、新たな行列Ａとベクトルｖのコンポーネントを求めて、ステップＳ４１６に進む。
【０９４９】
ステップＳ４１６では、予測タップ抽出部６２４が、教師データメモリ６２１に、まだ、注目データとしていない教師データが存在するかどうかを判定し、存在すると判定した場合、ステップＳ４１３に戻り、まだ、注目データとされていない教師データを、新たに注目データとして、以下、同様の処理が繰り返される。
【０９５０】
また、ステップＳ４１６において、教師データメモリ６２１に、注目データとしていない教師データが存在しないと判定された場合、ステップＳ４１７に進み、足し込み部６２７は、ステップＳ４１３乃至Ｓ４１６の処理を繰り返すことにより得られた新たな各クラスの行列Ａとベクトルｖのコンポーネントを、コンポーネントデータベース３１０に上書きする形で記憶させる。さらに、足し込み部６２７は、新たな各クラスの行列Ａおよびベクトルｖのコンポーネントで構成される式（８）の正規方程式を、タップ係数決定部６２８に供給し、ステップＳ４１８に進む。
【０９５１】
ステップＳ４１８では、タップ係数決定部６２８は、足し込み部６２７から供給される各クラスごとの正規方程式を解くことにより、各クラスごとに、タップ係数を求める。さらに、ステップＳ４１８では、タップ係数決定部６２８は、各クラスごとのタップ係数を、タップ係数メモリ６２９に供給し、上書きする形で記憶させ、処理を終了する。
【０９５２】
次に、図７０（Ｂ）のフローチャートを参照して、学習情報送信処理について説明する。
【０９５３】
学習情報送信処理では、まず最初に、ステップＳ４３１において、イベント検出部６３３が、所定のイベントが発生したかどうかを判定し、発生していないと判定した場合、ステップＳ４３１に戻る。
【０９５４】
また、ステップＳ４３１において、所定のイベントが発生したと判定された場合、即ち、イベント検出部６３３が、他のＰＤＡから送信されてきたリクエスト信号を受信した場合、イベント検出部６３３は、学習情報を送信するように、学習情報送信部６３２を制御し、ステップＳ４３２に進む。
【０９５５】
ステップＳ４３２では、学習情報送信部６３２は、イベント検出部６３３の制御にしたがい、コンポーネントデータベース６３０から、学習情報としての各クラスの行列Ａとベクトルｖのコンポーネントを読み出し、ステップＳ４３３に進む。
【０９５６】
ステップＳ４３３では、学習情報送信部６３２は、コンポーネントデータベース６３０から読み出した学習情報としての各クラスの行列Ａとベクトルｖのコンポーネントを、イベント検出部６３３が受信したリクエスト信号を送信してきた他のＰＤＡに送信する。そして、ステップＳ４３１に戻り、以下、同様の処理が繰り返される。
【０９５７】
次に、図７０（Ｃ）のフローチャートを参照して、コンポーネントデータ統合処理について説明する。
【０９５８】
コンポーネントデータ統合処理では、まず最初に、ステップＳ４４１において、リクエスト信号送信部６３４が、学習情報を要求する旨の操作信号の受信を、所定のイベントとして、そのような所定のイベントが発生したかどうかを判定し、発生していないと判定した場合、ステップＳ４４１に戻る。
【０９５９】
また、ステップＳ４４１において、所定のイベントが発生したと判定された場合、即ち、ユーザによって、学習情報を要求するように、操作部２２４（図２３）が操作され、その旨を表す操作信号を、リクエスト信号送信部６３４が受信した場合、ステップＳ４４２に進み、リクエスト信号送信部６３４は、任意の１以上のＰＤＡ宛に、学習情報を要求するリクエスト信号を送信する（例えば、ブロードキャストする）。
【０９６０】
そして、ステップＳ４４３に進み、統合部６３１は、ステップＳ４４２で送信されたリクエスト信号に応じて、他のＰＤＡから学習情報が送信されてきたかどうかを判定する。
【０９６１】
ステップＳ４４３において、学習情報が送信されてきたと判定された場合、ステップＳ４４４に進み、統合部６３１は、その学習情報を受信し、その内蔵するメモリ（図示せず）に一時記憶して、ステップＳ４４５に進む。
【０９６２】
また、ステップＳ４４３において、学習情報が送信されてきていないと判定された場合、ステップＳ４４５に進み、統合部６３１は、リクエスト信号送信部６３４が、ステップＳ４４２でリクエスト信号を送信してから所定時間が経過したかどうかを判定する。
【０９６３】
ステップＳ４４５において、リクエスト信号を送信してから、まだ、所定時間が経過していないと判定された場合、ステップＳ４４３に戻り、以下、同様の処理が繰り返される。
【０９６４】
また、ステップＳ４４５において、リクエスト信号を送信してから所定時間が経過したと判定された場合、ステップＳ４４６に進み、統合部６３１は、ステップＳ４４４において記憶した学習情報としての各クラスの行列Ａとベクトルｖのコンポーネントと、コンポーネントデータベース６３０に記憶されている各クラスの行列Ａとベクトルｖのコンポーネントとを、クラスごとに加算し、新たな各クラスの行列Ａとベクトルｖのコンポーネントを求める。さらに、ステップＳ４４６において、統合部６３１は、新たな各クラスの行列Ａとベクトルｖのコンポーネントを、コンポーネントデータベース６３０に上書きすることで、その更新を行い、ステップＳ４４７に進む。
【０９６５】
ステップＳ４４７では、足し込み部６２７が、コンポーネントデータベース６３０から、新たな各クラスの行列Ａおよびベクトルｖのコンポーネントを読み出し、そのコンポーネントで構成される式（８）の正規方程式をたてて、タップ係数決定部６２８に供給する。さらに、ステップＳ４４７では、タップ係数決定部６２８が、足し込み部６２７から供給された各クラスごとの正規方程式を解くことにより、各クラスごとに、タップ係数を求め、タップ係数メモリ６２９に供給して、上書きする形で記憶させる。そして、ステップＳ４４１に戻り、以下、同様の処理が繰り返される。
【０９６６】
以上のように、学習部６０２では、新たに入力された音声データの他、過去の学習に用いられた音声データに基づく学習処理が行われるので、ユーザが発話を行うほど、学習が進行したタップ係数が求められることになる。従って、音質向上部６０１において、そのようなタップ係数を用いて、復号音声データを処理することにより、より音質の向上した音声データ（音質向上データ）を得ることが可能となる。
【０９６７】
さらに、学習部６０２では、１以上の他のＰＤＡから収集した学習情報としての各クラスの行列Ａおよびベクトルｖのコンポーネントにより、コンポーネントデータベース６３０に記憶された各クラスの行列Ａおよびベクトルｖのコンポーネントが更新されていく。そして、そのような更新された各クラスの行列Ａおよびベクトルｖのコンポーネントからたてられた式（８）の正規方程式を解くことで、タップ係数が求められる。従って、音質向上部６０１において、そのようなタップ係数を用いて、復号音声データを処理することにより、やはり、より音質の向上した音声データを得ることが可能となる。
【０９６８】
さらに、学習部６０２では、上述したように、１以上の他のＰＤＡから、学習情報としての各クラスの行列Ａおよびベクトルｖのコンポーネントが収集されるため、音質の向上した音声データを得るためのタップ係数を、迅速に得ることができる。
【０９６９】
即ち、式（８）の正規方程式を、ユーザから入力された音声データだけを用いてたてる場合には、入力された音声データのサンプル数が十分でないこと等に起因して、タップ係数を求めるのに必要な数の正規方程式が得られないクラスが生じる場合があり得るが、上述のように、１以上の他のＰＤＡから、学習情報としての各クラスの行列Ａおよびベクトルｖのコンポーネントを収集する場合には、各クラスについて、式（８）の正規方程式をたてるための行列Ａとベクトルｖのコンポーネントを、迅速に得ることができ、その結果、音質の向上した音声データを得るためのタップ係数を、迅速に得ることができる。
【０９７０】
また、複数のＰＤＡどうしの間で、上述したように、それぞれが有する学習情報としての各クラスの行列Ａおよびベクトルｖのコンポーネントをやりとりすることで、その複数のＰＤＡでは、同一音質で、かつ高音質の音声データを得ることができるようになる。
【０９７１】
なお、図６９および図７０の実施の形態では、リクエスト信号送信部６３４において、操作信号を受信した場合に、リクエスト信号を送信するようにしたが、リクエスト信号は、その他、例えば、定期的または不定期に、任意のタイミングで送信するようにすることが可能である。
【０９７２】
また、図６６のＰＤＡ１０１においては、学習の途中で得られる各クラスの行列Ａおよびベクトルｖのコンポーネントを学習情報として、１以上の他のＰＤＡとの間でやりとりするようにしたが、その他、例えば、タップ係数自体を学習情報としてやりとりすることも可能である。
【０９７３】
ＰＤＡ１０１において、１以上の他のＰＤＡとの間で、タップ係数をやりとりする場合には、タップ係数の更新は、例えば、自身が有しているタップ係数と、他のＰＤＡから受信したタップ係数との重み付け加算等によって行うことが可能である。
【０９７４】
なお、この場合の重みとしては、タップ係数を求めるのに用いられた音声サンプル数を用いることができる。
【０９７５】
即ち、タップ係数を求めるためには、音声データから得られる教師データｙ_iおよび生徒データｘ_inを用いて計算される、行列Ａのコンポーネントとなるｘ_inｘ_imと、ベクトルｖのコンポーネントとなるｘ_inｙ_iを足し込んでいく必要があるが、その足し込み回数（これは、教師データとなった音声サンプル数に等しい）を、重みとして用いることが可能である。
【０９７６】
この場合、例えば、ＰＤＡ１０１が有しているタップ係数を、ｗ_a＝｛ｗ_a1，ｗ_a2，ｗ_a3，・・・｝と、そのタップ係数を求めるのに用いられた音声サンプル数を、αと、他のＰＤＡから受信したタップ係数を、ｗ_b＝｛ｗ_b1，ｗ_b2，ｗ_b3，・・・｝と、そのタップ係数を求めるのに用いられた音声サンプル数を、βと、それぞれするとき、新たなタップ係数は、式（αｗ_a＋βｗ_b）／（α＋β）で求められることになる。
【０９７７】
さらに、ＰＤＡ１０１において、１以上の他のＰＤＡとの間で、タップ係数をやりとりする場合には、タップ係数の更新は、例えば、自身がタップ係数を有していないクラスのタップ係数を、他のＰＤＡから受信したタップ係数によって補完することにより行うことが可能である。
【０９７８】
即ち、ＰＤＡ１０１において、式（８）の正規方程式を、そのユーザから入力された音声データだけを用いてたてる場合には、入力された音声データのサンプル数が十分でないこと等に起因して、タップ係数を求めるのに必要な数の正規方程式が得られないクラスが生じることがあり得るが、他のユーザのＰＤＡにおいては、そのクラスについて十分な数の正規方程式が得られていることがあり得る。
【０９７９】
そこで、ＰＤＡ１０１では、他のＰＤＡとの間でタップ係数をやりとりすることで、自身だけでは得られなかったクラスのタップ係数を、他のＰＤＡで得られたものによって補完するようにすることが可能である。
【０９８０】
以上、本発明をＰＤＡに適用した場合について説明したが、本発明は、ＰＤＡ以外の情報処理装置等にも適用可能である。
【０９８１】
なお、上述した一連の処理は、ハードウェアにより行うこともできるし、ソフトウェアにより行うこともできる。
【０９８２】
一連の処理をソフトウェアによって行う場合には、そのソフトウェアを構成するプログラムが、ＰＤＡ１０１やベース基地コンピュータ１０２にインストールされる。
【０９８３】
プログラムは、コンピュータに内蔵されている記録媒体としてのＨＤＤ２１５（図２３）やＨＤＤ３３３（図２６）に予め記録しておくことができる。
【０９８４】
あるいはまた、プログラムは、フレキシブルディスク、CD-ROM(Compact Disc Read Only Memory)，MO(Magneto Optical)ディスク，DVD(Digital Versatile Disc)、磁気ディスク、半導体メモリなどのリムーバブル記録媒体に、一時的あるいは永続的に格納（記録）しておくことができる。このようなリムーバブル記録媒体は、いわゆるパッケージソフトウエアとして提供することができる。
【０９８５】
なお、プログラムは、上述したようなリムーバブル記録媒体からＰＤＡ１０１やベース基地コンピュータ１０２にインストールする他、ダウンロードサイトから、ディジタル衛星放送用の人工衛星を介して、ＰＤＡ１０１やベース基地コンピュータ１０２に無線で転送したり、LAN(Local Area Network)、インターネットといったネットワークを介して、ＰＤＡ１０１やベース基地コンピュータ１０２に有線で転送し、ＰＤＡ１０１やベース基地コンピュータ１０２では、そのようにして転送されてくるプログラムを、通信部１０８で受信してインストールすることができる。
【０９８６】
ここで、本明細書において、ＰＤＡ１０１のＣＰＵ２０２（図２３）や、ベース基地コンピュータ１０２のＣＰＵ３１２（図２６）に各種の処理を行わせるためのプログラムを記述する処理ステップは、必ずしもフローチャートとして記載された順序に沿って時系列に処理する必要はなく、並列的あるいは個別に実行される処理（例えば、並列処理あるいはオブジェクトによる処理）も含むものである。
【０９８７】
また、プログラムは、１のＣＰＵにより処理されるものであっても良いし、複数のＣＰＵによって分散処理されるものであっても良い。さらに、プログラムは、遠方のコンピュータ等に転送されて実行されるものであっても良い。
【０９８８】
【発明の効果】
以上の如く、本発明の表示装置および表示方法、並びにプログラムによれば、複数の表示手段それぞれごとに、１つの情報が表示され、複数の表示手段それぞれに表示された情報のうちのいずれが選択されたかが検出される。そして、その選択された情報に対応付けられている複数の情報それぞれが、複数の表示手段それぞれに表示される。従って、分かりやすい情報の表示や、正確な操作が可能となる。
【図面の簡単な説明】
【図１】本発明を適用したＰＤＡ１０１の第１実施の形態の外観構成例を示す斜視図である。
【図２】ＰＤＡ１０１のメインブロック２の第１の外観構成例を示す斜視図である。
【図３】蓋部２０を開いた状態のメインブロック２の外観構成例を示す斜視図である。
【図４】蓋部２０、並びにサブパネル１５および１６を開いた状態のＰＤＡ１０１の外観構成例を示す斜視図である。
【図５】ＰＤＡ１０１における配線の状態を示す斜視図である。
【図６】ヒンジ部１３の構成例を示す斜視図である。
【図７】ＰＤＡ１０１のメインブロック２の第２の外観構成例を示す斜視図である。
【図８】メインブロック２の第２の外観構成例を示す上面図と側面図である。
【図９】蓋部２０を開いた状態のメインブロック２の外観構成例を示す斜視図である。
【図１０】蓋部２０、並びにサブパネル１５および１６を開いた状態のＰＤＡ１０１の外観構成例を示す斜視図である。
【図１１】ヒンジ部７１を分解した状態を示す斜視図である。
【図１２】ヒンジ部７１を分解した状態のより詳細を示す斜視図である。
【図１３】ＰＤＡ１０１のメインブロック２の第３の外観構成例を示す斜視図である。
【図１４】メインブロック２における配線の状態を示す断面図である。
【図１５】メインブロック２における配線の状態を示す断面図である。
【図１６】本発明を適用したＰＤＡ１０１の第２実施の形態の外観構成例を示す斜視図である。
【図１７】ＰＤＡ１０１の第２実施の形態の構成例を示す平面図である。
【図１８】ＰＤＡ１０１のメインブロック２の第４の外観構成例を示す斜視図である。
【図１９】メインブロック２における配線の状態を説明するための斜視図である。
【図２０】ＰＤＡ１０１のメインブロック２の第５の外観構成例を示す斜視図である。
【図２１】ＰＤＡ１０１のメインブロック２の第６の外観構成例を示す斜視図である。
【図２２】ＰＤＡ１０１を用いたＰＤＡシステムの一実施の形態の構成例を示す図である。
【図２３】ＰＤＡ１０１のハードウエア構成例を示すブロック図である。
【図２４】ＰＤＡ１０１の第１の機能的構成例を示すブロック図である。
【図２５】ベース基地コンピュータ１０２の外観構成例を示す斜視図である。
【図２６】ベース基地コンピュータ１０２のハードウエア構成例を示すブロック図である。
【図２７】ベース基地コンピュータ１０２の機能的構成例を示すブロック図である。
【図２８】ＰＤＡ１０１とベース基地コンピュータ１０２とが接続された状態を示すブロック図である。
【図２９】ＰＤＡ１０１が行う通話処理と発呼処理を説明するフローチャートである。
【図３０】ＰＤＡ１０１が行うメール送受信処理およびメール送信イベント処理を説明するフローチャートである。
【図３１】ＰＤＡ１０１が行うデータ送受信処理およびデータ送信イベント処理を説明するフローチャートである。
【図３２】ＰＤＡ１０１が行うデータ再生処理を説明するフローチャートである。
【図３３】ＰＤＡ１０１が行うストリーミング再生処理を説明するフローチャートである。
【図３４】ＰＤＡ１０１からベース基地コンピュータ１０２に対してファイルが送信される場合の通信手順を説明する図である。
【図３５】ベース基地コンピュータ１０２からＰＤＡ１０１に対してファイルが送信される場合の通信手順を説明するフローチャートである。
【図３６】ＬＣＤ４₁，５₁，１２，２１乃至２３における画面の表示例を示す図である。
【図３７】ＬＣＤ４₁，５₁，１２，２１乃至２３における画面の表示例を示す図である。
【図３８】ＬＣＤ４₁，５₁，１２，２１乃至２３における画面の表示例を示す図である。
【図３９】ＬＣＤ４₁乃至４₄，５₁乃至５₄，１２，２１乃至２３における画面の表示例を示す図である。
【図４０】ＬＣＤ３，１２，２１乃至２３のオン／オフ制御の処理を説明するフローチャートである。
【図４１】階層表示制御処理を説明するフローチャートである。
【図４２】階層構造に構造化された情報のフォーマットを示す図である。
【図４３】ＰＤＡ１０１が、他のＰＤＡ１０３との間で通信を行っている状態を示す図である。
【図４４】ＰＤＡ１０１の第２の機能的構成例を示すブロック図である。
【図４５】送信処理部４０１の第１の構成例を示すブロック図である。
【図４６】送信処理部４０１が行う画像データ送信処理、クラスコード生成処理、およびクラスコード送信処理を説明するフローチャートである。
【図４７】受信処理部４０２の第１の構成例を示すブロック図である。
【図４８】適応処理部４４７の構成例を示すブロック図である。
【図４９】受信処理部４０２が行う画像データ受信処理、リクエスト信号送信処理、および適応処理を説明するフローチャートである。
【図５０】送信処理部４０１の第２の構成例を示すブロック図である。
【図５１】学習部４１４の構成例を示すブロック図である。
【図５２】送信処理部４０１が行う学習処理およびタップ係数送信処理を説明するフローチャートである。
【図５３】受信処理部４０２の第２の構成例を示すブロック図である。
【図５４】適応処理部４４８の構成例を示すブロック図である。
【図５５】送信処理部４０１の第３の構成例を示すブロック図である。
【図５６】送信処理部４０１が行う画像データ送信処理およびコードブック選択処理を説明するフローチャートである。
【図５７】コードブックのフォーマットを示す図である。
【図５８】更新部５０６（５３９）の構成例を示すブロック図である。
【図５９】更新部５０６が行うデータ更新処理およびコードブック更新処理を説明するフローチャートである。
【図６０】受信処理部４０２の第３の構成例を示すブロック図である。
【図６１】受信処理部４０２が行う画像データ受信処理、およびコードブック選択処理を説明するフローチャートである。
【図６２】エラー訂正部５３４の構成例を示すブロック図である。
【図６３】部分ベクトル推定部５５３の処理を説明するための図である。
【図６４】エラー訂正処理を説明するフローチャートである。
【図６５】ＰＤＡ１０１とべーす基地コンピュータ１０２との間で行われるコードブックの統合を説明するための図である。
【図６６】ＰＤＡ１０１の第３の機能的構成例を示すブロック図である。
【図６７】音質向上部６０１の構成例を示すブロック図である。
【図６８】音質向上処理を説明するフローチャートである。
【図６９】学習部６０２の構成例を示すブロック図である。
【図７０】学習部６０２で行われる学習処理、学習情報送信処理、およびコンポーネントデータ統合処理を説明するフローチャートである。
【符号の説明】
１ 時計バンド， ２ メインブロック， ３ ＬＣＤ， ３Ａ タッチパネル， ４ サイドパネル， ４₁乃至４₄ ＬＣＤ， ４Ａ₁乃至４Ａ₄ タッチパネル， ５ サイドパネル， ５₁乃至５₄ ＬＣＤ， ５Ａ₁乃至５Ａ₄ タッチパネル， ６ ジョグダイヤル， ７ イヤフォンマイクジャック， ８ コネクタ部， ８₁乃至８₄ コネクタ， ９ マイク， １０ スピーカ， １１ 本体， １２ ＬＣＤ， １２Ａ タッチパネル， １３ ヒンジ部， １４ メインパネル， １５，１６ サブパネル， １７，１８ ヒンジ部， ２０ 蓋部， ２１ ＬＣＤ， ２１Ａ タッチパネル， ２２ ＬＣＤ， ２２Ａ タッチパネル， ２３ ＬＣＤ， ２３Ａ タッチパネル， ３１，３２ 軸，３３，３４ 軸受け部， ３５乃至４０ 通し穴， ４１乃至４３ 回路ブロック， ５１ 係止部， ５２ 溝， ６１ ホールドスイッチ， ６２ 電源スイッチ， ６３ 無線通信部， ６４ アンテナ， ６５ ＣＣＤカメラ， ７１乃至７３ ヒンジ部， ８１ ヒンジ金具， ８２ 軸， ８３ ヒンジ金具， ９１，９２ ヒンジカバー， １０１ ＰＤＡ， １０２ ベース基地コンピュータ， １０３ ＰＤＡ， １０４ 公衆網， １０５ インターネット， １１１ 回路基板， １１２，１１３ フレキ， １１４，１１５ 穴， １３１ アーム部， １３１Ａ，１３１Ｂ ピン部， １３２ アーム部， １３２Ａ，１３２Ｂ ピン部， １３３ アーム部， １３３Ａ，１３３Ｂ ピン部， １４３ 係止部， １４４ 穴， １４５乃至１４７ フレキ， ２０１バス， ２０２ ＣＰＵ， ２０３ ＲＯＭ， ２０４ ＲＡＭ， ２０５（２０５₁乃至２０５₇） タッチパネルドライバ， ２０６（２０６₁乃至２０６₇） ＬＣＤドライバ， ２０７乃至２０９ アンプ， ２１０，２１１ Ａ／Ｄ変換器， ２１２ Ｄ／Ａ変換器， ２１３ ＭＰＥＧエンコーダ／デコーダ，２１４ ＡＴＲＡＣエンコーダ／デコーダ， ２１５ ＨＤＤ， ２１６ フラッシュメモリ， ２１７ ＤＲＡＭ， ２１８ 通信Ｉ／Ｆ， ２１９ ドライバ， ２２０ 受光部， ２２１ 発光部， ２２２ バッテリ， ２２３ 入力Ｉ／Ｆ， ２２４ 操作部， ２２５ スイッチ部， ２２６ バス， ２３１ ＲＦ処理部， ２３２ チャネル復号部， ２３３ チャネル符号化部，２３４ エンコード／デコード部， ２３５ 画像符号化部， ２３６ 画像復号部， ２３７ 音声符号化部， ２３８ 音声復号部， ２３９ 制御部，２４０ 変復調部， ２４１ メモリ， ２４２ アラーム部， ２４３ 開閉検出部， ２４４ 表示制御部， ２４５ 表示部， ２５１乃至２５５ 表示制御部， ２５６乃至２６１ 表示部， ３０１ キーボード， ３０２ 表示部， ３０３ ＰＤＡ装着部， ３０４ 無線通信部， ３０５ IEEE1394端子， ３０６ USB端子， ３１１ バス， ３１２ ＣＰＵ， ３１３ ＲＯＭ， ３１４ ＲＡＭ， ３１５ フラッシュメモリ， ３１６ ＬＣＤ ， ３１７ ＬＣＤドライバ， ３１８ キーボードＩ／Ｆ， ３１９ USBインタフェース， ３２０ IEEE1394インタフェース， ３２１ PCMCIAＡドライバ，３２２ PCMCIAスロット， ３２３ ＡＴＲＡＣエンコーダ／デコーダ， ３２４ Ａ／Ｄ変換器， ３２５ Ｄ／Ａ変換器， ３２６，３２７ アンプ， ３２８ マイク， ３２９ スピーカ， ３３０ マイクジャック， ３３１ イヤフォンジャック， ３３２ CD-RWドライブ， ３３３ ＨＤＤ， ３３４フラッシュメモリ， ３３５ ＤＲＡＭ， ３３６ 通信Ｉ／Ｆ， ３３７ コネクタ部， ３３７₁乃至３３７₄ コネクタ， ３３８ ドライバ， ３３９受光部， ３４０ 発光部， ３４１ ＬＡＮボード， ３４２ モデム／ＴＡ／ＤＳＵ， ３４３ バス， ３４４ 充電回路， ３４５ バス， ３５１制御部， ３５２ 音声画像符号化復号部， ３５３ 操作部， ３５４ 変復調部， ３５５ 表示制御部， ３５６ 表示部， ３５７ メモリ， ４０１，４０１’ 送信処理部， ４０２，４０２’ 受信処理部， ４１１ 符号化部， ４１２ イベント検出部， ４１３ ストレージ， ４１４ 学習部，４１５ タップ係数バッファ， ４１６ ストレージ， ４２１ データ圧縮部， ４２２ 送信制御部， ４２３ クラスタップ抽出部， ４２４ クラス分類部， ４２５ クラスコードデータベース， ４３１ 復号部， ４３２ 画質判定部， ４３３ リクエスト信号送信部， ４４１ 受信制御部， ４４２ 受信バッファ， ４４３ 登録部， ４４４ ストレージ， ４４５ 選択部， ４４６ クラスコードデータベース， ４４７，４４８ 適応処理部， ４５１ バッファ， ４５２ 予測タップ抽出部， ４５３ 積和演算部， ４５４ クラスコード読み出し部， ４５５ 係数メモリ， ４５６ 登録部， ４６１ 教師データメモリ， ４６２ データ圧縮部， ４６３ 生徒データメモリ， ４６４ 予測タップ抽出部， ４６５ クラスタップ抽出部， ４６６クラス分類部， ４６７ 足し込み部， ４６８ タップ係数決定部， ５０１ ベクトル化部， ５０２ ベクトル量子化部， ５０３ ローカルデコード部， ５０４ 差分演算部， ５０５ エントロピー符号化部， ５０６ 更新部， ５０７ コードブックデータベース， ５０８ 選択部， ５０９ コードブック記憶部， ５２１ データ更新部， ５２２ コードブック更新部， ５３１ エントロピー復号部， ５３２ ベクトル逆量子化部， ５３３ 加算部， ５３４ エラー訂正部， ５３５ スカラ化部， ５３６ コードブックデータベース， ５３７ 選択部， ５３８ コードブック記憶部， ５３９ 更新部， ５４０ エラー検出部， ５５０ 書き込み部， ５５１ メモリ，５５２ 読み出し部， ５５３ 部分ベクトル推定部， ５５４ 画像ベクトル推定部， ６００ 音声復号部， ６０１ 音質向上部， ６０２ 学習部，６１１ バッファ， ６１２ 予測タップ抽出部， ６１３ クラスタップ抽出部， ６１４ クラス分類部， ６１５ 係数メモリ， ６１６ 積和演算部， ６２１ 教師データメモリ， ６２２ 生徒データ生成部， ６２２Ｄ 音声復号部， ６２２Ｅ 音声符号化部， ６２３ 生徒データメモリ， ６２４予測タップ抽出部， ６２５ クラスタップ抽出部， ６２６ クラス分類部， ６２７ 足し込み部， ６２８ タップ係数決定部， ６２９ タップ係数メモリ， ６３０ コンポーネントデータベース， ６３１ 統合部， ６３２学習情報送信部， ６３３ イベント検出部， ６３４ リクエスト信号送信部[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a display device, a display method, a program, and a recording medium. In particular, in a small information processing device such as a PDA (Personal Digital Assistance), easy-to-understand information display, accurate operation, etc. The present invention relates to a display device, a display method, a program, and a recording medium.
[0002]
[Prior art]
In recent years, PDAs have been manufactured and sold by many manufacturers.
[0003]
[Problems to be solved by the invention]
By the way, since the PDA is a so-called palm-sized information processing apparatus, a small LCD (Liquid Crystal Display) or the like for displaying information is adopted. Therefore, when displaying a large amount of information, the information must be displayed in a small size on a small LCD, and it may be difficult for the user to recognize the information without staring at the LCD. .
[0004]
Furthermore, as a PDA, an LCD and a touch panel (in this specification, including a tablet that can be operated with a finger as well as a tablet operated with a dedicated pen) are integrally configured, and buttons are provided on the LCD. The display is realized and the button operation is detected by the touch panel.
[0005]
However, as described above, since the LCD provided in the PDA is small, the buttons to be displayed on the LCD must be displayed in a small size, and a button that is different from the button to be operated is erroneously operated. There was a case.
[0006]
The present invention has been made in view of such a situation, and makes it possible to display easy-to-understand information and to enable an accurate operation.
[0007]
[Means for Solving the Problems]
The display device according to the present invention includes a display control unit that performs control to display one piece of information on each of a plurality of display units, and a detection that detects which of the information displayed on each of the plurality of display units is selected. Each of the plurality of pieces of information associated with the selected information includes: a means for detecting the state of the cover portion; and a panel state detecting means for detecting the state of the panel. Are displayed on each of the plurality of display means, and the plurality of display means are turned on / off according to the state of the lid detected by the lid state detection means and the state of the panel detected by the panel state detection means. In addition, when displaying a plurality of menu items on a plurality of display means, the display means and the menu items are displayed in a one-to-one correspondence according to user selection. And re-display, Displaying menu items across a plurality of display means without causing one-to-one correspondence between display means and menu items It is characterized by controlling switching to multi-screen display.
[0008]
The display method of the present invention includes a display control step for performing control for displaying one piece of information on each of a plurality of display means, and detection for detecting which of the information displayed on each of the plurality of display means is selected. A plurality of pieces of information associated with the selected information in the display control step, each including a step, a lid state detection step for detecting the state of the lid, and a panel state detection step for detecting the state of the panel Is displayed on each of the plurality of display means, and on / off of the plurality of display means is controlled in accordance with the state of the lid and the state of the panel, and a plurality of menu items are displayed on the plurality of display means. At the time, according to the user selection, the directory display for displaying the display means and the menu items in a one-to-one correspondence, Displaying menu items across a plurality of display means without causing one-to-one correspondence between display means and menu items It is characterized by controlling switching to multi-screen display.
[0009]
The program of the present invention includes a display control step for performing control for displaying one piece of information on each of a plurality of display means, and a detection step for detecting which of the information displayed on each of the plurality of display means is selected. And a lid state detection step for detecting the state of the lid portion, and a panel state detection step for detecting the state of the panel. In the display control step, each of a plurality of pieces of information associated with the selected information is displayed. When displaying on each of the plurality of display means, controlling on / off of the plurality of display means according to the state of the lid and the state of the panel, and further displaying a plurality of menu items on the plurality of display means , According to user selection, a directory display for displaying display means and menu items in a one-to-one correspondence; Displaying menu items across a plurality of display means without causing one-to-one correspondence between display means and menu items It is characterized by controlling switching to multi-screen display.
[0010]
The recording medium of the present invention includes a display control step for performing control to display one piece of information on each of a plurality of display means, and detection for detecting which of the information displayed on each of the plurality of display means has been selected. A plurality of pieces of information associated with the selected information in the display control step, each including a step, a lid state detection step for detecting the state of the lid, and a panel state detection step for detecting the state of the panel Is displayed on each of the plurality of display means, and on / off of the plurality of display means is controlled in accordance with the state of the lid and the state of the panel, and a plurality of menu items are displayed on the plurality of display means. At the time, according to the user selection, the directory display for displaying the display means and the menu items in a one-to-one correspondence, Displaying menu items across a plurality of display means without causing one-to-one correspondence between display means and menu items It is characterized by controlling switching to multi-screen display.
[0011]
In the display device, the display method, and the program of the present invention, one piece of information is displayed on each of the plurality of display means, and it is detected which of the information displayed on each of the plurality of display means is selected, and the lid The state of the panel is detected, and the state of the panel is detected. Each of the plurality of pieces of information associated with the selected information is displayed on each of the plurality of display units, and on / off of the plurality of display units is controlled according to the state of the lid and the state of the panel. In addition, when displaying a plurality of menu items on a plurality of display means, according to user selection, a directory display for displaying the display means and the menu items in a one-to-one correspondence, Displaying menu items across a plurality of display means without causing one-to-one correspondence between display means and menu items Switching to multi-screen display is controlled.
[0012]
DETAILED DESCRIPTION OF THE INVENTION
1 to 4 show an example of the external configuration of an embodiment of a PDA to which the present invention is applied.
[0013]
As shown in FIG. 1, the PDA is of a wrist watch type, and is configured by providing a main block 2 on a watch band 1. The user can easily carry the PDA by attaching the watch band 1 to the wrist part of the left arm or the right arm as in the case of a wristwatch.
[0014]
The main block 2 corresponds to a main body portion of a watch in the case of a wristwatch, and includes a main body 11 and a lid 20 that is rotatably attached to one end of the main body.
[0015]
The lid portion 20 has an LCD (Liquid Crystal Display) 3 provided so as to be exposed on the upper surface in the closed state. In the embodiment of FIG. 1, the LDC 3 includes a timepiece composed of a long hand and a short hand. Is displayed.
[0016]
Further, the watch band 1 of the PDA is provided with side panels 4 and 5 on the upper side and the lower side of the main block 2, respectively, as shown in FIG. The side panel 4 has an LCD 4 ₁ Is provided so as to be exposed in the upper surface direction. ₁ Is provided so as to be exposed in the upper surface direction.
[0017]
The LCD 3 is configured integrally with a transparent touch panel 3A, and operations on buttons and the like displayed on the LCD 3 are detected by the touch panel 3A. Similarly, LCD4 ₁ Touch panel 4A ₁ And integrated with the LCD5 ₁ Touch panel 5A ₁ It is configured integrally with.
[0018]
The PDA has various functions as will be described later, such as a telephone function, and the LCD 3 and the like of the lid 20 change the display as necessary according to the function mode that provides each function. When the function mode is, for example, the clock mode, as shown in FIG. 1, the LCD 3 displays a clock, but when the function mode is, for example, the telephone mode, the display on the LCD 3 is as shown in FIG. As shown, the display changes to a button operated to dial (enter a telephone number). Note that the operation on this button is detected by the touch panel 3A configured integrally with the LCD 3, as described above.
[0019]
Note that the switching of the function mode is performed according to a user operation, or the PDA is performed based on a predetermined event.
[0020]
As shown in FIG. 2, a jog dial 6, an earphone microphone jack 7, a connector 8, and a microphone 9 are provided on the lower side surface of the main body 11. In FIG. 1 and the like, the jog dial 6 and the like are not shown in order to avoid complication of the drawing.
[0021]
The jog dial 6 can be rotated left and right. For example, when the function mode is the telephone mode, the other party who is going to make a call from a list display such as a telephone number displayed on the LCD 3 or the like. This is operated when selecting a telephone number. Further, the jog dial 6 can be pressed in the internal direction of the main body 11, and the selection is confirmed by the pressing operation of the jog dial 6. That is, for example, the user selects the telephone number of the other party to be called from the list display such as the telephone number displayed on the LCD 3 or the like by rotating the jog dial 6, and further presses the jog dial 6. When operated, the selection of the telephone number is confirmed and a call is made to the telephone number.
[0022]
The jog dial 6 is urged outward from the inside of the main body 11. Therefore, when the user applies a force to the internal direction of the main body 11, the jog dial 6 moves in the internal direction of the main body 11. However, when the user stops applying force, the jog dial 6 returns to the original position by the biasing force.
[0023]
The earphone microphone jack 7 includes, for example, a jack provided on a headset when a so-called headset (not shown), which is an integrally configured earphone and microphone (microphone), is connected to the main body 11. The main body 11 and the headset are thereby electrically connected.
[0024]
The connector unit 8 is engaged with the connector unit 337 (FIG. 26) of the base base computer 102 when performing data communication with the base base computer 102 (FIG. 22) to be described later. The main body 11 (PDA) and the base base computer 102 are electrically connected.
[0025]
The microphone 9 captures the user's voice and converts it into a voice signal as an electrical signal. The microphone 9 captures the user's voice transmitted to the other party in a voice call when the PDA function mode is, for example, a telephone mode.
[0026]
A speaker 10 is provided on the upper surface of the upper surface in a state where the lid 20 is closed. For example, when the function mode is set to the telephone mode, the speaker 10 outputs the voice transmitted from the other party.
[0027]
A hold switch 61 and a power switch 62 are provided on the left side surface of the main body 11. In FIG. 1 and the like, the hold switch 61 and the power switch 62 are not shown in order to avoid making the figure complicated.
[0028]
The hold switch 61 is operated to enable / disable the operation of buttons displayed on the jog dial 6 or the LCD 3. When the hold switch 61 is operated so as to invalidate the operation of the button displayed on the jog dial 6 or the LCD 3, etc., when the PDA is carried in the back etc., the PDA is put in the back, etc. It is possible to prevent an erroneous operation from being performed by colliding with the object.
[0029]
The power switch 62 is operated when turning on / off the power of the PDA.
[0030]
A hinge portion 13 is provided at the upper and lower portions of the right end of the main body 11, and the lid portion 20 can be rotated about the hinge portion 13 as a rotation center. As shown in FIG. 3, the lid portion 20 can be rotated with the hinge portion 13 of the main body 11 as a rotation center to a position where the upper surface is substantially horizontal with the upper surface or the bottom surface of the main body 11. Thereby, the cover part 20 will be in the open state.
[0031]
As shown in FIG. 3, an LCD 12 is provided on the upper surface of the main body 11 so as to face the lid portion 20 in a closed state. The LCD 12 is in a stored state when the lid portion 20 is in a closed state. However, when the lid portion 20 is in an open state, it is exposed on the upper surface. The LCD 12 is also configured integrally with the transparent touch panel 12A, and operations on buttons and the like displayed on the LCD 12 are detected by the touch panel 12A.
[0032]
The lid 20 has a main panel 14 and two sub panels 15 and 16.
[0033]
A hinge portion 17 is provided at the upper left end and the right end of the main panel 14, and a hinge portion 18 is provided at the lower left end and the right end thereof, respectively. The sub panel 15 is attached so as to be rotatable about the hinge portion 17 as a rotation center, and the sub panel 16 is attached so as to be rotatable about the hinge portion 18 as a rotation center.
[0034]
If the vertical direction is the vertical direction and the horizontal direction is the horizontal direction, the length of the sub-panels 15 and 16 in the horizontal direction is slightly shorter than the length of the main panel 14 in the horizontal direction. Further, the vertical length of the sub-panels 15 and 16 is about ½ or less of the vertical length of the main panel 14.
[0035]
The sub-panel 15 rotates about the hinge portion 17 as the center of rotation, and the sub-panel 16 rotates about the hinge portion 18 as the center of rotation to a position that is almost horizontal with the upper surface or the back surface of the main panel 14. As a result, the sub-panels 15 and 16 are opened as shown in FIG.
[0036]
As shown in FIG. 4, the LCD 21 is placed on the upper surface of the main panel 14, that is, the back surface of the lid 20 when the subpanel 15, 16 is open. Is provided. Note that an LCD 3 (FIG. 1) is provided on the surface of the main panel 14 opposite to the surface on which the LCD 21 is provided. Therefore, when the lid 20 is closed, the LCD 3 is exposed (upward), and the LCD 14 is stored (down), and the lid 20 is open. The LCD 3 is in the stowed state, and the LCD 21 is exposed in the upper surface direction.
[0037]
The sub-panel 15 has an upper surface when it is open, LCD22 Is provided on the upper surface of the sub-panel 16 when it is opened. LCD23 Is provided. Therefore, the LCD 22 of the sub-panel 15 is exposed when the sub-panel 15 is in an open state, and when the sub-panel 15 is in a closed state, the LCD 22 is in a stowed state facing the LCD 21 of the main panel 14. Similarly, the LCD 23 of the sub-panel 16 is exposed when the sub-panel 16 is in an open state, and is in a stowed state facing the LCD 21 of the main panel 14 when the sub-panel 16 is closed.
[0038]
Note that the LCD 21 of the main panel 14 is configured integrally with a transparent touch panel 21A, and operations on buttons and the like displayed on the main panel 14 are detected by the touch panel 21A. Similarly, the LCD 22 of the sub-panel 15 and the touch panel 22A and the LCD 23 of the sub-panel 16 are integrally configured with the touch panel 23A, respectively.
[0039]
As described above, in the main block 2, when the lid portion 20 is closed, only one LCD 3 is exposed. Accordingly, in this case, the LCD 4 of the side panel 4 provided in the watch band 1 is used. ₁ And the LCD 5 of the side panel 5 ₁ In combination, the PDA has three LCDs 3, 4 as shown in FIG. ₁ , 5 ₁ Information can be displayed on the screen and presented to the user.
[0040]
On the other hand, when the lid 20 is opened and the sub-panels 15 and 16 of the lid 20 are further opened, the four LCDs 12, 21, 22, and 23 are exposed. Accordingly, in this case, the LCD 4 of the side panel 4 provided in the watch band 1 is used. ₁ And the LCD 5 of the side panel 5 ₁ In combination, the PDA has six LCDs 4 as shown in FIG. ₁ , 5 ₁ , 12, 21, 22, and 23, information can be displayed and presented to the user.
[0041]
By the way, in the main block 2 configured as described above, the control of the LCDs 3 and 21 included in the main panel 14 of the lid 20 (including the control of the touch panels 3A and 21A) from the electric circuit built in the main body 11; In addition, control of the LCD 22 included in the sub-panel 15 of the lid 20 (including control of the touch panel 22A) and control of the LCD 23 included in the sub-panel 16 (including control of the touch panel 23A) are performed. Therefore, it is necessary to perform wiring from the main body 11 to the main panel 14 of the lid portion 20 and further to the sub panels 15 and 16.
[0042]
Therefore, with reference to FIG. 5, wiring from the main body 11 to the main panel 14 and the sub panels 15 and 16 will be described.
[0043]
FIG. 5 shows an enlarged view of a portion surrounded by a dotted line in FIG. In FIG. 5, the inside of the main block 2 is indicated by a dotted line.
[0044]
The hinge portion 13 includes a shaft 31 and a bearing portion 33. The shaft 31 is fixed to one side of the main panel 14, and an end portion thereof is inserted into a hole provided in the bearing portion 33. That is, the bearing portion 33 is provided with a hole having a diameter slightly larger than the diameter of the shaft 31, and the end portion of the shaft 31 is inserted into the hole. Accordingly, the shaft 31 is rotatably supported by the bearing portion 33, so that the main panel 14 fixed to the shaft 31 can rotate about the shaft 31.
[0045]
The hinge portion 17 is also composed of a shaft 32 configured in the same manner as the shaft 31 and a bearing portion 34 configured in the same manner as the bearing portion 33, and the shaft 32 is fixed to one side of the sub-panel 15. Accordingly, the shaft 32 is rotatably supported by the bearing portion 34, so that the sub-panel 15 fixed to the shaft 32 can rotate about the shaft 32.
[0046]
A part of the inside of the bearing portion 33 constituting the hinge portion 13 is hollow and is fixed to a corner portion of the main body 11. A through hole 40 is formed in a portion where the bearing portion 33 of the main body 11 is fixed, and flexible (flexible cables) 146 and 147 as wirings extending from the circuit block 43 as an electric circuit built in the main body 11 are provided. , Through the through hole 40 and reach the inside of the bearing portion 33.
[0047]
Further, a through hole 35 is provided in a part of the end of the shaft 31 inserted into the bearing portion 33, and a part of the portion fixed to the main panel 14 of the shaft 31 is A through hole 36 is provided. The inside of the shaft 31 is hollow, and the flexes 146 and 147 reaching the inside of the bearing portion 33 pass through the through hole 35, the inside of the shaft 31, and the through hole 36, and Reached inside.
[0048]
Inside the main panel 14, a flexible cable 147 is connected to a circuit block 42 as an electric circuit built in the main panel 14.
[0049]
Similarly to the bearing portion 33, a part of the bearing portion 34 constituting the hinge portion 17 is hollow and is fixed to a corner portion of the main panel 14. A through hole 39 is formed in a portion of the main panel 14 where the bearing portion 34 is fixed, and the flexible cable 146 reaching the inside of the main panel 14 reaches the inside of the bearing portion 34 through the through hole 39. is doing.
[0050]
The shaft 32 inserted into the bearing portion 34 is provided with a through hole 37 at a part of the end thereof, as with the shaft 31, and further at a part of the portion fixed to the sub-panel 15. A through hole 38 is provided. The inside of the shaft 32 is hollow, and the flexible cable 146 reaching the inside of the bearing portion 34 reaches the inside of the sub-panel 15 through the through hole 37, the inside of the shaft 32, and the through hole 39. is doing.
[0051]
Inside the sub panel 15, a flex 146 is connected to a circuit block 41 as an electric circuit built in the sub panel 15.
[0052]
As described above, the circuit block 43 built in the main body 11 and the circuit block 42 built in the main panel 14 or the circuit block 41 built in the sub panel 15 are electrically connected to each other.
[0053]
The main body 11 and the sub panel 16 are also electrically connected in the same manner as the main body 11 and the sub panel 15.
[0054]
Next, in the hinge portion 13, when the shaft 31 is simply inserted into the hole of the bearing portion 33, when the user applies a certain force to rotate the lid portion 20, the shaft 31 is moved. However, there is a possibility that it will be easily removed from the hole of the bearing portion 33.
[0055]
Therefore, the hinge portion 13 is configured as shown in FIG. 6, thereby preventing the shaft 31 from easily coming out of the hole of the bearing portion 33. In FIG. 6, illustration of the through holes 35 and 36 is omitted.
[0056]
That is, at the end of the shaft 31 (the portion inserted into the hole of the bearing portion 33), as shown in the perspective view of FIG. A locking portion 51 is provided in which the remaining portion is formed so as to be able to vibrate. Accordingly, a part of the locking portion 51 is connected to the shaft 31 and is pushed into the inner side by applying a force in the inner direction of the shaft 31, but when the force is stopped, the locking portion 51 itself It is designed to return to its original state by the elastic force.
[0057]
Further, the engaging portion 51 has a convex portion having a substantially triangular cross section at one end opposite to one end connected to the shaft 31.
[0058]
The shaft 31 is provided with a locking portion similar to the locking portion 51 at a position 180 degrees opposite to the locking portion 51.
[0059]
On the other hand, in a part of the inside of the hole of the bearing portion 33, a groove having a depth shallower than the height of the convex portion formed on the locking portion 51 of the shaft 31, as shown in the sectional view of FIG. 52 is provided over the inner periphery of the hole.
[0060]
When the shaft 31 is inserted into the hole of the bearing portion 33, the engaging portion 51 is pushed into the inner direction of the shaft 31 by contacting the inner wall of the hole of the bearing portion 33 in the portion without the groove 52. After that, when the locking portion 51 reaches the groove 52, the locking portion 51 pushed into the shaft 31 is provided in the locking portion 51 so as to return to the original state by its elastic force. The projecting portion is fitted into the groove 52, and the shaft 31 is not easily removed from the hole of the bearing portion 33.
[0061]
Here, the groove 52 is formed to such a depth that when the convex portion of the locking portion 51 is fitted therein, the convex portion can be pushed somewhat into the inner direction of the shaft 31. Therefore, in a state where the convex portion provided in the locking portion 51 is fitted in the groove 52, the locking portion 51 is slightly pushed into the inner side of the shaft 31.
[0062]
Accordingly, when a rotational torque is applied to the shaft 31 due to the influence of gravity or the like when the lid portion 20 tries to rotate, the space between the convex portion provided on the locking portion 51 of the shaft 31 and the inner wall of the groove 52. A frictional force that prevents the rotation is generated.
[0063]
Thereby, when the cover part 20 is opened at an arbitrary angle (0 to 180 degrees) with respect to the main body 11, the state of the cover part 20 is maintained by the above-described frictional force. However, this frictional force is approximately the same as the rotational torque generated by the dead weight of the lid part 20, and does not hinder the turning operation of the lid part 20 by the user.
[0064]
Note that the shaft 32 and the bearing portion 34 constituting the hinge portion 17 are also configured in the same manner as in the hinge portion 13 shown in FIG. Moreover, the hinge part 18 is also comprised similarly to the hinge part 13 of FIG.
[0065]
Next, FIGS. 7 to 10 are external views showing other configuration examples of the main block 2. In the figure, portions corresponding to those shown in FIGS. 1 to 4 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate.
[0066]
Here, FIG. 7 shows a perspective view of the main block 2. 8A is a front (top) view of the main block 2, FIG. 8B is a side view of the upper side of the main block 2, and FIG. 8C is a lower side of the main block 2. 8D shows a left side view of the main block 2, and FIG. 8E shows a right side view of the main block 2. Further, FIG. 9 shows a perspective view of the main block 2 with the lid portion 20 opened, and FIG. 10 further shows a perspective view of the lid portion 20 with the sub-panels 15 and 16 opened. .
[0067]
As shown in FIG. 8B, a radio communication unit 63, an antenna 64, and a CCD (Charge Coupled Device) camera 65 are provided on the upper side surface of the main body 11 in the main block 2.
[0068]
The wireless communication unit 63 emits infrared rays and receives infrared rays when performing wireless communication using infrared rays or the like, for example.
[0069]
For example, the antenna 64 emits radio waves and receives radio waves when performing wireless communication using radio waves. The antenna 64 can transmit and receive radio waves for wireless communication in accordance with, for example, Bluetooth (trademark) and other standards in addition to transmission and reception of radio waves for performing voice calls over the telephone. It has become.
[0070]
The CCD camera 65 receives light incident thereon and photoelectrically converts it to output an image signal corresponding to the light. That is, the CCD camera 65 captures an image.
[0071]
The wireless communication unit 63, the antenna 64, and the CCD camera 65 are also provided in the main block 2 shown in FIG.
[0072]
The main block 2 shown in FIGS. 7 to 10 and the main block 2 shown in FIGS. 1 to 4 have the same function.
[0073]
However, the main block 2 shown in FIGS. 7 to 10 is different from the main block 2 shown in FIGS. 1 to 4 in the hinge mechanism.
[0074]
That is, in the main block 2 shown in FIGS. 7 to 10, the lid portion 20 has a hinge portion 71 (FIG. 8A, FIG. 8E, FIG. 9) provided on the right side surface of the main block 2. , FIG. 10) is rotated about the rotation center. Further, the sub panel 15 rotates around a hinge portion 72 (FIGS. 8B, 9 and 10) provided on the upper portion of the main panel 14, and the sub panel 16 is positioned below the main panel 14. The hinge 73 (FIGS. 7, 8C, 9 and 10) provided is rotated around the rotation center.
[0075]
FIG. 11 shows an exploded view of the hinge portion 71.
[0076]
As shown in FIG. 11, the hinge portion 71 includes a hinge fitting 81 fixed to the main body 11, a hinge fitting 83 fixed to the lid portion 20, and a shaft 82 that is rotatably coupled to both.
[0077]
As shown in FIG. 12, the hinge fittings 81 and 83 are formed of a spring steel plate, and the end portions thereof are bent (curled) and formed into a cylindrical shape. The hole diameters of the hinge fitting 81 and the hinge fitting 83 are the same. The shaft 82 is a stainless steel cylinder having a predetermined length, and the shaft diameter is slightly larger than the hole diameter of the hinge fittings 81 and 83. Both ends of the shaft 82 are press-fitted into the hinge fitting 81 and the hinge fitting 83, respectively.
[0078]
As described above, the shaft diameter of the shaft 82 is larger than the hole diameter of the hinge fittings 81 and 83. Therefore, when the shaft 82 is press-fitted, the hole diameter of the hinge fittings 81 and 83 is expanded (elastically deformed). ), The hinge fittings 81 and 83 are in a state of supporting the shaft 82 elastically and rotatably.
[0079]
In this state, the shaft 82 is not completely fixed by the hinge fittings 81 and 83, and when a rotational torque is applied to the shaft 82 due to the influence of gravity or the like, the shaft 82 is rotated. A frictional force is generated between the hinge 82 and the hinge fittings 81 and 83 to prevent the rotation.
[0080]
Therefore, when the lid 20 is opened at an arbitrary angle (0 to 180 degrees) with respect to the main body 11, the state of the lid 20 is maintained by the above-described frictional force. However, this frictional force is approximately the same as the rotational torque generated by the dead weight of the lid part 20, and does not hinder the turning operation of the lid part 20 by the user.
[0081]
In addition, the hinge parts 72 and 73 are also comprised similarly to the hinge part 71 shown in FIG. 11 and FIG.
[0082]
Next, the wiring from the main body 11 to the main panel 14 and the sub panels 15 and 16 when the hinge portions 71 to 73 are employed in the main block 2 will be described with reference to FIGS.
[0083]
Although not shown in FIGS. 7 to 10 in order to avoid complication of the drawing, the main block 2 has a hinge portion 71 for connecting the main body 11 and the lid portion 20 as shown in FIG. A covering hinge cover 91 is provided.
[0084]
Further, as shown in FIG. 13, the lid 20 is also provided with a hinge cover 92 that covers the hinge 73 that connects the main panel 14 and the sub-panel 16.
[0085]
Although not shown, the lid portion 20 is further provided with a hinge cover that covers the hinge portion 72 that couples the main panel 14 and the sub panel 15.
[0086]
14 shows a cross-sectional view of the main block 2 when viewed from the direction of the lower side surface indicated by the arrow A in FIG.
[0087]
Inside the main body 11, a circuit board 111 on which various electric circuits (electronic circuits) are formed is provided. The circuit board 111 includes an LCD 12 (including a touch panel 12A) provided on the upper surface of the main body 11. And are electrically connected via the flex 112.
[0088]
Further, a flexible board 113 is connected to the circuit board 111. The flexible board 113 passes through a hole 114 provided on a side surface to which the hinge portion 71 of the main body 11 is fixed, and then goes out to the outside. The lid 20 reaches the inside of the main panel 14 through a hole 115 provided in a side surface to which the hinge portion 71 of the main panel 14 is fixed. Inside the main panel 14, the flex 113 is connected to the LCD 3 (including the tablet 3 </ b> A) provided on the upper surface thereof and the LCD 21 (including the tablet 21 </ b> A) provided on the rear surface thereof. As a result, the LCDs 3 and 21 of the main panel 14 are electrically connected to the circuit board 111.
[0089]
The hinge cover 91 that covers the hinge portion 71 is provided in a shape that protects the flex 113 that is exposed to the outside between the holes 114 and 115, which may cause the flex 113 to be damaged or the like. Is to prevent.
[0090]
FIG. 15 is a cross-sectional view of the main block 2 when viewed from the right side surface direction indicated by an arrow B in FIG.
[0091]
The flex 113 (a part) reaching the inside of the main panel 14 has once gone out through the hole 122 provided on the side surface to which the hinge portion 73 of the main panel 14 is fixed, Further, the sub-panel 16 reaches the inside of the sub-panel 16 through a hole 123 provided on a side surface to which the hinge portion 73 is fixed. Inside the sub panel 16, the flex 113 is connected to the LCD 23 (including the touch panel 23 </ b> A) provided on the sub panel 16. Thereby, the LCD 23 of the sub-panel 16 is electrically connected to the circuit board 111.
[0092]
The hinge cover 92 covering the hinge portion 73 is provided in such a shape as to protect the flexible cable 113 exposed to the outside between the holes 122 and 123, and the flexible film 113 may be damaged. Is to prevent.
[0093]
Note that the flex 113 reaches the inside of the sub panel 15 in the same manner as in the sub panel 16, so that the LCD 22 (including the touch panel 22 </ b> A) provided on the sub panel 15 is electrically connected to the circuit board 111. It is connected to the.
[0094]
Next, in the above embodiment, one LCD 4 is respectively provided on the upper side and the lower side of the main block 2 of the watch band 1 of the PDA. ₁ And 5 ₁ The watch band 1 can be provided with more LCDs.
[0095]
FIG. 16 and FIG. 17 show an example of an external configuration of a PDA in which more LCDs are provided on the watch band 1.
[0096]
In the embodiment of FIGS. 16 and 17, four LCDs 4 are provided on the upper side of the main block 2 of the watch band 1. ₁ , 4 ₂ , 4 _Three , 4 _Four And four LCDs 5 on the lower side of the main block 2 ₁ , 5 ₂ , 5 _Three , 5 _Four Is provided.
[0097]
Note that, as shown in FIG. 17A, fasteners 1A and 1B are provided at two ends of the watch band 1, respectively. When the watch band 1 is wound around the user's arm and the fasteners 1A and 1B are coupled, the watch band 1 becomes ring-shaped and maintains the state of being worn on the user's arm as shown in FIG.
[0098]
FIG. 17A shows a state where the watch band 1 is removed from the state in which the watch band 1 is worn on the user's arm and is placed on a plane with the lid portion 20 closed, as shown in FIG. Yes.
[0099]
In this case, in addition to the LCD 3 provided on the lid 20 of the main block 2, the LCD 4 provided on the watch band 1. ₁ Thru 4 _Four And LCD5 ₁ 5 _Four In total, nine LCDs are exposed at the top.
[0100]
Therefore, in this case, the nine LCDs 3, 4 ₁ Thru 4 _Four And 5 ₁ 5 _Four Thus, information can be presented to the user.
[0101]
LCD4 ₂ , 4 _Three , 4 _Four , 5 ₂ , 5 _Three , 5 _Four LCD4 mentioned above ₁ And 5 ₁ Like the transparent touch panel 4A ₂ , 4A _Three , 4A _Four , 5A ₂ , 5A _Three , 5A _Four The LCD 4 is constructed integrally with each other. ₂ , 4 _Three , 4 _Four , 5 ₂ , 5 _Three , 5 _Four Button is displayed, and when the user operates the button, the operation on the button is performed by the touch panel 4A. ₂ , 4A _Three , 4A _Four , 5A ₂ , 5A _Three , 5A _Four Are detected respectively.
[0102]
FIG. 17B shows a state in which the lid 20 is opened and the side panels 15 and 16 are opened from the state shown in FIG.
[0103]
In this case, in the main block 2, the LCDs 12, 21, 22 and 23 are exposed at the top instead of the LCD 3.
[0104]
Therefore, in this case, the LCD 4 provided on the watch band 1 ₁ Thru 4 _Four And LCD5 ₁ 5 _Four Information can be presented to the user by a total of 12 LCDs including the LCD 12 provided in the main block 2 and the LCDs 21 to 23.
[0105]
Next, FIG. 18 to FIG. 21 show still another configuration example of the main block 2.
[0106]
In the embodiment of FIG. 18, the main panel 14 is located on the main body 11 and the sub-panel 15 on the main panel 14 as shown in FIG. And the main block 2 is configured so that the positions of 16 and 16 are located.
[0107]
That is, for example, in the embodiment of FIGS. 2 and 7, the sub-panels 15 and 16 are positioned on the main body 11 with the lid 20 closed, and the main panel is positioned on the sub-panels 15 and 16. However, in the embodiment shown in FIG. 18, the position of the main panel 14 and the positions of the sub panels 15 and 16 are interchanged. For this reason, in FIG. 18, the LCD 22 of the sub-panel 15 and the LCD 23 of the sub-panel 16 are exposed to the top when the lid 20 is closed. The LCDs 22 and 23 are, for example, In the main block 2 shown in FIG. 7, the LCD 3 exposed to the upper part is also used when the lid 20 is in a closed state.
[0108]
Further, in the embodiment shown in FIG. 18, the main body 11 and the lid part 20 (the main panel 14 thereof) comprise a so-called link comprising an arm part 131 and pin parts 131A and 131B, as shown in FIG. Connected by structure.
[0109]
Further, the main panel 14 and the sub panel 15 in the lid part 20 are coupled by a link structure including the arm part 132 and the pin parts 132A and 132B, and the main panel 14 and the sub panel 16 are connected to the arm part 133 and the pin part. They are connected by a link structure consisting of 133A and 133B.
[0110]
In FIG. 18, a link structure similar to the link structure including the arm portion 131 and the pin portions 131 </ b> A and 131 </ b> B is provided on the upper side surface of the main block 2. Further, on the left side surface of the main block 2, a link structure similar to the link structure including the arm part 132 and the pin parts 132A and 132B and a link structure including the arm part 133 and the pin parts 133A and 133B are similar. A link structure is provided.
[0111]
Pin portions 131A and 131B are rotatably provided at both ends of the arm portion 131, respectively. The pin portion 131A is inserted into the lower left corner of the lower side surface of the main panel 14 of the lid portion 20, and the pin portion 131B is inserted into the central lower portion of the lower side surface of the main body 11.
[0112]
Pin portions 132A and 132B are rotatably provided at both ends of the arm portion 132, respectively. The pin portion 132A is inserted into the upper right corner of the right side surface of the main panel 14, and the pin portion 132B is inserted into the upper center of the right side surface of the sub panel 15.
[0113]
Pin portions 133A and 133B are rotatably provided at both ends of the arm portion 133, respectively. The pin portion 133A is inserted into the upper left corner of the right side surface of the main panel 14, and the pin portion 133B is inserted into the upper center of the right side surface of the sub panel 16.
[0114]
Therefore, when the lid portion 20 is moved so as to slide to the right, the arm portion 131 rotates about the pin portion 131A and the pin portion 131B as the rotation center. As shown in FIG. 18 (B), the lid 20 moves to a position adjacent to the right side surface of the main body 11 so that the LCD 12 provided on the upper surface of the main body 11 is exposed.
[0115]
Further, when the sub panel 15 is moved so as to slide in the upward direction (depth direction), the arm portion 132 rotates about the pin portion 132A and rotates about the pin portion 132B. The sub panel 15 moves to a position adjacent to the upper side surface of the main panel, as shown in FIG.
[0116]
Further, when the sub-panel 16 is moved so as to slide downward (frontward), the arm part 133 rotates about the pin part 133A and rotates about the pin part 133B. The sub panel 16 moves to a position adjacent to the lower side surface of the main panel, as shown in FIG.
[0117]
Thus, the LCD 21 provided on the upper surface of the main panel 14 can be seen.
[0118]
In the main block 2 in the embodiment shown in FIG. 18, the sub-panels 15 and 16 are first slid from the state shown in FIG. 18A, and then the main panel 14 and the entire sub-panels 15 and 16 are moved. It is also possible to slide to the state shown in FIG.
[0119]
Next, with reference to FIG. 19, the wiring when the link structure as shown in FIG. 18 is adopted will be described.
[0120]
That is, FIG. 19 shows an enlarged view of a portion of the main panel 14 in FIG. 18 into which the pin 131A is inserted.
[0121]
The main panel 14 is provided with a hole 144 into which the pin portion 131A is inserted. The diameter of the hole 144 is slightly larger than the diameter of the pin portion 131A.
[0122]
On the other hand, the pin portion 131A is provided with a locking portion 143 made of an elastic body such as rubber at the end inserted into the hole 144, and the diameter of the portion of the locking portion 143 is as follows. The diameter of the hole 144 is slightly larger.
[0123]
Therefore, when the pin portion 131A is inserted into the hole 144, the locking portion 143 is caught in the hole 144. However, since the locking portion 143 is an elastic body, the pin portion 131A is inserted into the hole 144 by a large force. As a result, the elastic body as the locking portion 143 is deformed and passes through the hole 144. When the locking portion 143 passes through the hole 144 and reaches the inside of the main panel 14, the locking portion 143 returns to its original state due to its elastic force, so that the pin portion 131 </ b> A is not easily removed from the hole 144. Become.
[0124]
In addition, the inside of the arm 131 and the inside of the pin portion 131A are hollow, and the flexible board 145 that is a wiring extending from the main body 11 passes through the inside of the arm 131 and the inside of the pin portion 131A and passes through the main panel 14. To reach.
[0125]
The other link structure shown in FIG. 18 is also configured in the same manner as described with reference to FIG. 19, so that wiring from the main body 11 to the main panel 14 and further to the sub panels 15 and 16 is possible. It has become.
[0126]
Next, in the embodiment of FIG. 20, the positional relationship between the main panel 11 and the main panel 14 and the sub panels 15 and 16 constituting the lid 20 is the same as in the embodiment of FIG. Therefore, when the lid 20 is closed, the LCD 22 of the sub-panel 15 and the LCD 23 of the sub-panel 16 are exposed to the top as shown in FIG.
[0127]
However, in the embodiment of FIG. 20, the coupling structure between the main body 11 and the main panel 14, the coupling structure between the main panel 14 and the sub panel 15, and the coupling structure between the main panel 14 and the sub panel 16 are not link structures. Similar to the embodiment of FIGS. 2 and 7, a hinge structure is employed.
[0128]
Therefore, when the lid 20 is opened, as shown in FIG. 20B, the LCD 21 provided on the main panel 14 of the lid 20 is exposed to the top, but is exposed to the upper surface when the lid 20 is closed. The LCD 22 of the sub-panel 15 and the LCD 23 of the sub-panel 16 are in a state of facing downward as shown in FIG. 20B when the lid 20 is opened.
[0129]
When the sub panels 15 and 16 are further opened, the LCD 22 of the sub panel 15 and the LCD 23 of the sub panel 16 are exposed to the top as shown in FIG.
[0130]
Next, in the embodiment of FIG. 21, the main block 2 is basically configured in the same manner as in FIG. 2 and FIG. However, in the embodiment of FIGS. 2 and 7, the vertical lengths of the sub-panels 15 and 16 are about a half of the vertical length of the main panel 14. In the twenty-first embodiment, the vertical lengths of the sub-panels 15 and 16 are approximately the same (slightly shorter) as the vertical length of the main panel 14. Accordingly, in the embodiment of FIG. 21, the sub-panels 15 and 16 are approximately the same size as the main panel 14.
[0131]
In the main block 2 of FIG. 21, when the lid 20 is opened after the lid 20 shown in FIG. 21A is closed, the LCD 12 provided in the main body 11 is exposed as shown in FIG. 21B. Further, when the sub panels 15 and 16 are opened, the LCD 21 of the main panel 14, the LCD 22 of the sub panel 15, and the LCD 23 of the sub panel 16 are exposed as shown in FIG.
[0132]
As described above, in the embodiment of FIG. 21, the sub panels 15 and 16 are approximately the same size as the main panel 14. It is the same size.
[0133]
Therefore, in the embodiment of FIG. 21, it is possible to perform more display or larger display on the LCDs 22 and 23 as compared to the embodiments of FIG. 2 and FIG.
[0134]
In the above-described embodiment, the sub-panels 15 and 16 are provided on the lid 20, but the sub-panels 15 and 16 may be provided on the main body 11.
[0135]
In the above-described case, it is necessary to provide the two sub-panels 15 and 16 in the lid portion 20, but it is possible to provide only one sub-panel.
[0136]
Further, the sub-panel can be provided so as to open on the right side or the like instead of opening on the upper side or the lower side. In addition, the lid 20 can be provided with a sub-panel that opens on the right side in addition to the sub-panel 15 that opens on the upper side and the sub-panel 16 that opens on the lower side.
[0137]
Next, FIG. 22 shows a configuration example of an embodiment of a PDA system using the PDA described in FIGS.
[0138]
The PDA 101 is a PDA described with reference to FIGS. 1 to 21 and performs various data processing and also communicates with other PDAs 103, the Internet 105, and various other communication terminals 106 via the public network 104. Be able to.
[0139]
The various communication terminals 106 include telephones (including mobile phones), facsimiles, computers, and the like.
[0140]
Further, the PDA 101 can perform data communication with the base base computer 102 for exchanging various data.
[0141]
Here, as data exchanged between the PDA 101 and the base base computer 102, for example, image data (including moving images and still images), audio data (audio data), personal information such as e-mail addresses and telephone numbers, Various files such as programs and other binary files and text files, stored information downloaded from the Internet 105 and other PDAs 103 configured similarly to the PDA 101 via the public network 104, and other various information processing devices Data transmitted / received to / from the server.
[0142]
The base base computer 102 is configured based on, for example, a desktop or notebook computer, and serves as a base base for the PDA 101. That is, since the PDA 101 is configured in a small size for convenience of carrying, the PDA 101 is inferior in performance to a desktop or notebook computer that can be configured in a larger size. Therefore, the base base computer 102 acquires (receives) and processes the data held by the PDA 101 and provides (transmits) the processing result to the PDA 101, or the base base computer 102 acquires the data from the Internet 105 or the like. Data can be provided to the PDA 101 or the like.
[0143]
The basic configuration of the base base computer 102 is the same as that of a general desktop type or notebook type computer except that the base base computer 102 can be a base base of the PDA 101. It is possible to connect to the Internet 105 and execute various programs.
[0144]
Next, FIG. 23 shows a hardware configuration example of the PDA 101.
[0145]
A CPU (Central Processing Unit) 202 is connected to the bus 201 and controls each block connected to the bus 201. Further, the CPU 202 is connected to a ROM (Read Only Memory) 203 and a RAM (Random Access Memory) 204 via a bus 226, and executes a program stored in the ROM 203 or a program loaded into the RAM 204. Thus, various processes including the above-described control are performed.
[0146]
The ROM 203 stores programs necessary for activation, such as an IPL (Initial Program Loading) program. The RAM 204 loads programs and data transferred from the CPU 202 via the bus 226, and temporarily stores data necessary for the operation of the CPU 202.
[0147]
Touch panel driver 205 ₁ , 205 ₂ , 205 _Three , 205 _Four , 205 _Five , 205 ₆ , 205 ₇ Touch panel 3A, 4A ₁ , 5A ₁ , 12A, 21A, 22A, and 23A, respectively, by driving the touch panels 3A and 4A. ₁ , 5A ₁ , 12A, 21A, 22A, and 23A are detected and supplied to the CPU 202 via the bus 201. In FIG. 23, the touch panel driver 205 ₁ To 205 ₇ Are collectively represented as a touch panel driver 205.
[0148]
LCD driver 206 ₁ , 206 ₂ , 206 _Three , 206 _Four , 206 _Five , 206 ₆ , 206 ₇ LCD3, 4 according to the signal supplied via the bus 201 ₁ , 5 ₁ , 12, 21, 22, and 23, respectively, to drive LCDs 3, 4 ₁ , 5 ₁ , 12, 21, 22, and 23, a predetermined image is displayed. In FIG. 23, the LCD driver 206 ₁ To 206 ₇ Are collectively represented as an LCD driver 206.
[0149]
The amplifier 207 amplifies the image signal output from the CCD camera 65 and supplies it to an A / D (Analog / Digital) converter 210. The amplifier 208 amplifies the audio signal output from the D / A (Digital / Analog) converter 212 and outputs it to the speaker 10 or the earphone microphone jack 7. The amplifier 209 amplifies the audio signal input from the microphone 9 or the earphone microphone jack 7 and supplies the amplified audio signal to the A / D converter 211.
[0150]
The A / D converter 210 performs A / D conversion on the analog image signal supplied from the amplifier 207, and supplies the analog image signal to an MPEG (Moving Picture Experts Group) encoder / decoder 213 as digital image data. The A / D converter 211 performs A / D conversion on the analog audio signal supplied from the amplifier 209 and supplies the analog audio signal to an ATRAC (Adaptive TRansform Acoustic Coding) encoder / decoder 214 as digital audio data. The D / A converter 212 D / A converts the digital audio data supplied from the ATRAC encoder / decoder 214 and supplies the digital audio data to the amplifier 208 as an analog audio signal.
[0151]
The MPEG encoder / decoder 213 encodes the image data supplied from the A / D converter 210 or the bus 201 in accordance with the MPEG standard, and outputs the encoded data obtained as a result on the bus 201. The MPEG encoder / decoder 213 decodes the encoded data supplied from the bus 201 in accordance with the MPEG standard, and outputs the resulting image data on the bus 201.
[0152]
The MPEG encoder / decoder 213 can output the image data supplied from the A / D converter 210 to the bus 201 as it is without performing any particular processing. .
[0153]
The ATRAC encoder / decoder 214 encodes audio data supplied from the A / D converter 211 or the bus 201 in accordance with the ATRAC standard, and outputs the encoded data obtained as a result on the bus 201. The ATRAC encoder / decoder 214 decodes the encoded data supplied from the bus 201 in accordance with the ATRAC standard, and outputs the resulting audio data to the bus 201 or the D / A converter 212. .
[0154]
The ATRAC encoder / decoder 214 outputs the audio data from the bus 201 as it is to the D / A converter 212 as needed, and also outputs the audio data from the D / A converter 212 as it is to the bus. The data can also be output on 201.
[0155]
An HDD (Hard Disk Drive) 215 incorporates an HD (Hard Disk) (not shown), reads data (including a program) recorded on the HD under the control of the CPU 202, and outputs it to the bus 201. In addition, data supplied from the bus 201 is written to the HD.
[0156]
The flash memory 216 is connected to the bus 201 and stores data that needs to be stored via the bus 201 even when the power of the PDA is turned off. That is, for example, the flash memory 216 stores, for example, the internal state immediately before the PDA is turned off. Thereby, when the power supply of the PDA is turned on again, the internal state of the PDA can be returned to the internal state immediately before the power supply is turned off by referring to the stored contents of the flash memory 216. .
[0157]
A DRAM (Dynamic RAM) 217 temporarily stores data (for example, image data and audio data to be encoded, encoded image data and audio data, etc.) supplied via the bus 201.
[0158]
A communication I / F (interface) 218 is connected to the bus 201 and functions as an interface for performing various types of communication by radio (including radio waves, infrared rays, and the like) and wired communication.
[0159]
That is, the communication I / F 218 performs processing necessary for communication such as demodulation on the received signal supplied from the antenna 64, outputs the received signal to the bus 201, and modulates the data supplied via the bus 201 Processing necessary for communication is performed, and a transmission signal obtained as a result is supplied to the antenna 64.
[0160]
Further, the communication I / F 218 receives data supplied from the connector unit 8, performs necessary processing, outputs the data to the bus 201, performs predetermined processing on the data supplied via the bus 201, and Supply to the connector section 8.
[0161]
The communication I / F 218 receives data supplied from the driver 219, performs necessary processing, outputs the data to the bus 201, performs predetermined processing on the data supplied via the bus 201, and 219.
[0162]
The driver 219, the light receiving unit 220, and the light emitting unit 221 constitute a wireless communication unit 63. The driver 219 drives the light emitting unit 221 in accordance with data supplied from the communication I / F 219, and the light receiving unit. Data is extracted from the signal supplied from 220 and supplied to the communication I / F 218. For example, the light receiving unit 220 receives infrared light and supplies an electric signal corresponding to the amount of received light to the driver 219. The light emitting unit 221 is driven by the driver 219 and emits infrared light, for example.
[0163]
The battery 222 supplies necessary power to each block constituting the PDA 101.
[0164]
The input I / F 223 is connected to the bus 201 and functions as an interface for external operation input. That is, the input I / F 223 receives signals from the operation unit 224 and the switch unit 225 and outputs them on the bus 201.
[0165]
The switch unit 225 includes a switch group that detects the open / close state of the cover unit 20 and the open / close state of the sub-panels 15 and 16, and signals according to the open / close state of the cover unit 20 and the sub-panels 15 and 16. This is supplied to the input I / F 223.
[0166]
The operation unit 224 includes the jog dial 6 shown in FIG. 7 and the like, a hold switch 61, a power switch 62, and the like, and supplies operation signals corresponding to these operations to the input I / F 223.
[0167]
Next, FIG. 24 shows a functional configuration example of the PDA 101. Note that portions corresponding to the hardware configuration of the PDA 101 in FIG. 23 are given the same reference numerals as appropriate.
[0168]
An RF (Radio Frequency) processing unit 231 demodulates an RF signal as a reception signal supplied from the antenna 64 and supplies the demodulated signal to the channel decoding unit 232. Further, the RF processing unit 231 modulates the signal supplied from the channel encoding unit 223 into an RF signal and supplies the RF signal to the antenna 64.
[0169]
The channel decoding unit 232 performs channel decoding on the signal supplied from the RF processing unit 231 and supplies the signal to the encoding / decoding unit 234 and the control unit 239. The channel encoding unit 233 performs channel encoding on the signal supplied from the encoding / decoding unit 234 and the control unit 239 and supplies the signal to the RF processing unit 231.
[0170]
Note that the RF processing unit 231, the channel decoding unit 232, and the channel coding unit 233 correspond to the communication I / F 218 in FIG.
[0171]
The encoding / decoding unit 234 includes an image encoding unit 235, an image decoding unit 236, an audio encoding unit 237, and an audio decoding unit 238.
[0172]
The image encoding unit 235 encodes the image data supplied from the control unit 224 under the control of the control unit 224 and supplies the encoded image data to the control unit 224 or the channel encoding unit 233. The image decoding unit 236 decodes the encoded data of the image supplied from the channel decoding unit 232 or the control unit 239 and supplies the decoded data to the control unit 239 or the display control unit 236. The audio encoding unit 237 encodes audio data supplied from the amplifier 209 or the control unit 239 and supplies the encoded audio data to the channel encoding unit 233 or the control unit 239. The audio decoding unit 238 decodes audio encoded data supplied from the channel decoding unit 232 or the control unit 239 and supplies the decoded data to the speaker 208 or the control unit 239.
[0173]
The encode / decode unit 234 corresponds to the MPEG encoder / decoder 213 and the ATRAC encoder / decoder 214 shown in FIG. 23, and is realized by the CPU 202 executing a program.
[0174]
The control unit 239 is realized by the CPU 202 shown in FIG. 23 executing a program. The control unit 239 exchanges data via the connector unit 8, processes corresponding to operation signals from the operation unit 224, and other various processes (PDA101). (Including control of each block constituting).
[0175]
The modem unit 240 modulates data supplied from the control unit 239 and supplies the data to the wireless communication unit 63. The modem unit 240 demodulates the signal supplied from the wireless communication unit 63 and supplies the demodulated signal to the control unit 239. The modem unit 240 corresponds to the communication I / F 218 in FIG.
[0176]
The memory 241 stores data supplied from the control unit 239 and supplies the stored data to the control unit 239. In the present embodiment, the memory 241 is connected not only to the control unit 239 but also to the connector unit 8. The memory 241 corresponds to the flash memory 216, the DRAM 217, or the like in FIG.
[0177]
For example, the alarm unit 242 monitors the remaining capacity of the battery 222, and notifies the control unit 239 of the fact when a so-called low battery state is reached. Note that the alarm unit 242 is realized, for example, when the CPU 202 in FIG. 23 executes a program.
[0178]
The opening / closing detection unit 243 detects opening / closing of the lid unit 20 and opening / closing of the sub-panels 15 and 16, and supplies the detection result to the control unit 239. The open / close detection unit 243 corresponds to the switch unit 225 in FIG.
[0179]
The display control unit 244 performs control for causing the display unit 245 to display an image according to the control of the control unit 239 and an image supplied from the image decoding unit 236. Further, the display control unit 244 detects an operation on a button or the like displayed on the display unit 245 and supplies an operation signal corresponding to the operation to the control unit 239. In addition, the display control unit 244 supplies signals from the control unit 239 to the display control units 251, 252, 253, 254, and 255 as necessary, and signals from the display control units 251 to 255, respectively. Is supplied to the control unit 239.
[0180]
The display control unit 244 corresponds to the touch panel driver 205 and the LCD driver 206 in FIG. The same applies to display control units 251 to 255 described later.
[0181]
The display unit 245 displays an image under the control of the display control unit 244, detects an operation on the display screen, and supplies a signal indicating the operated position on the display screen to the display control unit 244. . The display unit 245 corresponds to the LCD 11 and the touch panel 12A (FIG. 10 and the like) provided integrally with the main body 11.
[0182]
The display control unit 251 controls the display unit 256 or 257 to display an image according to a control signal supplied from the control unit 239 via the display control unit 244 and is displayed on the display unit 256 or 257. An operation on a button or the like is detected, and an operation signal corresponding to the operation is supplied to the control unit 239 via the display control unit 244. The display units 256 and 257 display an image according to the control of the display control unit 251, detect an operation on the display screen, and send a signal indicating the operated position on the display screen to the display control unit 251. Supply. The display unit 256 corresponds to the LCD 3 and the touch panel 3A (FIG. 7 and the like) integrally provided on the main panel 14, and the display unit 257 is the LCD 21 and the touch panel 21A (integrally provided on the main panel 14). This corresponds to FIG.
[0183]
The display control unit 252 controls the display unit 258 to display an image in accordance with a control signal supplied from the control unit 239 via the display control unit 244, and operates the buttons displayed on the display unit 258. And an operation signal corresponding to the operation is supplied to the control unit 239 via the display control unit 244. The display unit 258 displays an image under the control of the display control unit 252, detects an operation on the display screen, and supplies a signal indicating the operated position on the display screen to the display control unit 252. . The display unit 258 corresponds to the LCD 22 and the touch panel 22A (FIG. 10 and the like) integrally provided on the sub panel 15.
[0184]
The display control unit 253 performs control for causing the display unit 259 to display an image according to a control signal supplied from the control unit 239 via the display control unit 244, and operates the buttons displayed on the display unit 259. And an operation signal corresponding to the operation is supplied to the control unit 239 via the display control unit 244. The display unit 259 displays an image according to the control of the display control unit 253, detects an operation on the display screen, and supplies a signal representing the operated position on the display screen to the display control unit 253. . The display unit 259 corresponds to the LCD 23 and the touch panel 23A (FIG. 10 and the like) provided integrally with the sub panel 16.
[0185]
The display control unit 254 controls the display unit 260 to display an image according to a control signal supplied from the control unit 239 via the display control unit 244, and operates the buttons and the like displayed on the display unit 260. And an operation signal corresponding to the operation is supplied to the control unit 239 via the display control unit 244. The display unit 260 displays an image according to the control of the display control unit 254, detects an operation on the display screen, and supplies a signal indicating the operated position on the display screen to the display control unit 254. . The display unit 260 is an LCD 4 provided integrally with the side panel 4. ₁ And touch panel 4A ₁ (FIG. 10 etc.).
[0186]
The display control unit 255 controls the display unit 261 to display an image according to a control signal supplied from the control unit 239 via the display control unit 244, and operates the buttons and the like displayed on the display unit 261. And an operation signal corresponding to the operation is supplied to the control unit 239 via the display control unit 244. The display unit 261 displays an image under the control of the display control unit 255, detects an operation on the display screen, and supplies a signal indicating the operated position on the display screen to the display control unit 255. . The display unit 261 is an LCD 5 provided integrally with the side panel 5. ₁ And touch panel 5A ₁ (FIG. 10 etc.).
[0187]
Next, FIG. 25 is a perspective view showing an external configuration example of the base base computer 102 of FIG.
[0188]
In the embodiment of FIG. 25, the base base computer 102 has a substantially flat plate shape, and the front side has a tapered shape having a predetermined taper angle. A keyboard 301 that is operated by the user is disposed in the tapered portion.
[0189]
Further, a display unit 302 composed of, for example, an LCD or the like is provided on the left side of the upper surface of the base base computer 201, and various information is displayed on the display unit 302.
[0190]
Furthermore, a PDA mounting unit 303 and a wireless communication unit 304 are provided slightly on the right side of the upper surface of the base base computer 201.
[0191]
The PDA mounting portion 303 is a concave slot in which the main block 2 of the PDA 101 can be mounted. In addition, a connector portion 337 illustrated in FIG. 26 described later is provided inside the PDA mounting portion 303. When the main block 2 of the PDA 101 is inserted so that the connector portion 8 (FIG. 7 or the like) faces the bottom surface of the concave portion of the PDA mounting portion 303 and is mounted on the PDA mounting portion 303, The PDA mounting unit 303 is electrically connected to the connector unit 337, so that the PDA 101 (main block 2) and the base base computer 102 can communicate with each other.
[0192]
The wireless communication unit 304 is configured to transmit and receive infrared rays and the like when communicating with the PDA 101 using infrared rays and the like.
[0193]
In addition, in the embodiment of FIG. 25, the base base computer 102 is connected to other devices on the right side surface when performing communication conforming to the IEEE (Institute of Electrical and Electronics Engineers) 1394 standard. IEEE 1394 terminal 305 and USB terminal 306 that is connected to other devices when communication conforming to the USB (Universal Serial Bus) standard is provided. The IEEE1394 terminal 305 is connected to a video camera, for example, as a device compliant with the IEEE1394 standard. The USB terminal 306 is connected to, for example, a mouse as a device compliant with the USB standard.
[0194]
Next, FIG. 26 shows a hardware configuration example of the base base computer 102.
[0195]
The CPU 312 is connected to the bus 311 and controls each block connected to the bus 311. Further, the CPU 312 is connected to the ROM 313, the RAM 314, and the flash memory 315 via the bus 345. The CPU 312 executes the above-described control by executing a program stored in the ROM 313 or a program loaded in the RAM 314. Various types of processing are performed.
[0196]
The ROM 313 stores programs necessary for startup, such as IPL programs. The RAM 314 loads programs and data transferred from the CPU 312 via the bus 343, and temporarily stores data necessary for the operation of the CPU 312. The flash memory 315 stores, for example, a BIOS (Basic Input Output System) program. In other words, in the present embodiment, a BIOS program is stored in the rewritable flash memory 315, which makes it possible to easily deal with BIOS upgrades and the like.
[0197]
The LCD 316 and the LCD driver 317 constitute a display unit 302. The LCD driver 316 causes the LCD 316 to display a predetermined image by driving the LCD 316 according to a signal supplied via the bus 311.
[0198]
The keyboard I / F 318 functions as an interface between the keyboard 301 and the bus 311, and outputs an operation signal corresponding to the operation of the keyboard 301 to the bus 311.
[0199]
The USB interface 319 is a communication interface compliant with the USB standard, receives data from the bus 311, transmits it from the USB terminal 306, receives data from the USB terminal 306, and outputs it on the bus 311. The IEEE1394 interface 320 is a communication interface compliant with the IEEE1394 standard, receives data from the bus 311, transmits it from the IEEE1394 terminal 305, receives data from the IEEE1394 terminal 305, and outputs it on the bus 311.
[0200]
A PCMCIA (Personal Computer Memory Card International Association) driver 321 is connected to the bus 311 and drives a PC card (not shown) mounted in the PCMCIA slot 322. For example, a PC card such as a flash memory card, a hard disk, a SCSI card, a LAN card, a modem card, and the like can be attached to and detached from the PCMCIA slot 322. In FIG. 25, the PCMCIA slot 322 is not shown.
[0201]
The ATRAC encoder / decoder 323 encodes audio data supplied from the A / D converter 324 or the bus 311 in accordance with the ATRAC standard, and outputs the encoded data obtained as a result on the bus 311. The ATRAC encoder / decoder 323 decodes the encoded data supplied from the bus 311 in accordance with the ATRAC standard, and outputs the resulting audio data to the bus 311 or the D / A converter 325. .
[0202]
The ATRAC encoder / decoder 323 outputs the audio data from the bus 311 as it is to the D / A converter 325 as needed, and the audio data from the A / D converter 324 as it is to the bus. 311 can also be output.
[0203]
The A / D converter 324 A / D converts the analog audio signal supplied from the amplifier 326 and supplies the analog audio signal to the ATRAC encoder / decoder 323 as digital audio data. The D / A converter 325 performs D / A conversion on the digital audio data supplied from the ATRAC encoder / decoder 323 and supplies the digital audio data to the amplifier 327 as an analog audio signal.
[0204]
The amplifier 326 amplifies the audio signal input from the microphone 328 or the microphone jack 330 and supplies the amplified audio signal to the A / D converter 324. The amplifier 327 amplifies the audio signal supplied from the D / A converter 325 and supplies the amplified audio signal to the speaker 329 or the earphone jack 331.
[0205]
The microphone 328 converts audio into an audio signal as an electric signal and supplies the audio signal to the amplifier 326. The speaker 329 outputs a sound corresponding to the audio signal from the amplifier 327. A microphone or the like for inputting sound is connected to the microphone jack 330, and an earphone or the like for outputting sound is connected to the earphone jack 331. In FIG. 25, the microphone 328, the speaker 329, the microphone jack 330, and the earphone jack 331 are not shown.
[0206]
A CD-RW (Compact Disc ReWritable) drive 332 drives a CD-RW (not shown), writes data supplied from the bus 311 to the CD-RW, and reproduces data from the CD-RW to generate a bus. 311 is output.
[0207]
The HDD 333 incorporates an HD (not shown), reads data (including a program) recorded on the HD under the control of the CPU 312, outputs the data to the bus 311, and receives data supplied from the bus 311. Write to HD.
[0208]
The flash memory 334 and the DRAM 335 temporarily store data supplied via the bus 311.
[0209]
The communication I / F 336 is connected to the bus 311 and functions as an interface when performing various types of wireless or wired communication.
[0210]
That is, the communication I / F 336 performs processing necessary for communication such as demodulation on the reception signal supplied from the antenna 343, outputs the signal to the bus 311, and modulates the data supplied via the bus 311. Processing necessary for communication is performed, and a transmission signal obtained as a result is supplied to the antenna 343.
[0211]
Further, the communication I / F 336 receives data supplied from the connector unit 337 included in the PDA mounting unit 303, performs necessary processing, outputs the data to the bus 311, and converts the data supplied via the bus 311 into data. A predetermined process is performed and supplied to the connector portion 337.
[0212]
In addition, the communication I / F 336 receives data supplied from the driver 338, performs necessary processing, outputs the data to the bus 311, performs predetermined processing on the data supplied via the bus 311, and 338.
[0213]
As described above, the connector portion 337 is provided on the bottom surface of the recess (slot) as the PDA mounting portion 303 shown in FIG. 25, and includes one or more connectors included in the connector portion 8 (FIG. 7 and the like) of the PDA 101. It has one or more connectors to be connected.
[0214]
The driver 338, the light receiving unit 339, and the light emitting unit 340 constitute a wireless communication unit 304. The driver 338 drives the light emitting unit 340 according to data supplied from the communication I / F 338, and also receives the light receiving unit. Data is extracted from the signal supplied from 339 and supplied to the communication I / F 336. For example, the light receiving unit 339 receives infrared rays and supplies an electric signal corresponding to the amount of received light to the driver 338. The light emitting unit 340 is driven by the driver 338 and emits infrared light, for example.
[0215]
A LAN cable is connected to a LAN (Local Area Network) board 341 when, for example, a LAN such as Ethernet (registered trademark) is configured. The LAN board 341 is connected to the LAN cable and the bus 311. Exchange data between them. The modem / TA / DSU (Terminal Adapter / Digital Service Unit) 342 is connected to a telephone line such as a PSTN (Public Switched Telephone Network) line or an ISDN (Integrated Service Digital Network) line. Data is exchanged between the telephone line and the bus 311.
[0216]
The antenna 343 transmits data from the communication I / F 336 by radio waves, receives radio waves transmitted thereto, and supplies the radio I / F 336 to the communication I / F 336. As a result, the antenna 343 transmits and receives radio waves for wireless communication according to, for example, Bluetooth (trademark) and other standards. Note that the antenna 343 is not shown in FIG.
[0217]
For example, when the PDA 101 is mounted on the PDA mounting unit 303, the charging circuit 344 charges the battery 222 (FIGS. 23 and 24) included in the PDA 101.
[0218]
Next, FIG. 27 shows a functional configuration example of the base base computer 102. Note that portions corresponding to the hardware configuration of the base base computer 102 in FIG. 26 are given the same reference numerals as appropriate.
[0219]
The control unit 351 is realized by the CPU 312 in FIG. 26 executing a program, and performs various processes (including control of each block configuring the base base computer 102).
[0220]
That is, the control unit 351 performs processing according to an operation signal from the operation unit 353, for example. Also, the control unit 351 supplies, for example, image data and audio data to the audio image encoding / decoding unit 352, and encodes the image data and audio data. Furthermore, the control unit 351 supplies, for example, encoded data obtained by encoding an image or audio to the audio image encoding / decoding unit 352, and causes the encoded data to be decoded into image data or audio data. Further, for example, the control unit 351 supplies data to be transmitted wirelessly to the modem unit 354 and receives data supplied from the modem unit 354. Further, the control unit 351 supplies image data to be displayed to the display control unit 355, for example. Furthermore, for example, the control unit 351 supplies and stores data that needs to be held to the memory 357 and reads out necessary data from the memory 357. The control unit 351 is, for example, a connector 337 among one or more connectors constituting the connector unit 337 included in the PDA mounting unit 303. _Three And 337 _Four The necessary data is transmitted / received via the IEEE1394 terminal 305 and the USB terminal 306. Furthermore, the control unit 351 controls the HDD 333 to write data and read necessary data, for example. Moreover, the control part 351 controls the charging circuit 344, for example.
[0221]
The audio image encoding / decoding unit 352 encodes the image data and audio data supplied from the control unit 351, and supplies the encoded data obtained as a result to the control unit 351. The audio image encoding / decoding unit 352 decodes the encoded data supplied from the control unit 351 and supplies the image data and audio data obtained as a result to the control unit 351.
[0222]
Note that the audio image encoding / decoding unit 352 corresponds to the ATRAC encoder / decoder 323 in FIG. 26 and is realized by the CPU 312 executing the program.
[0223]
The modem unit 354 modulates the data supplied from the control unit 351 and supplies the data to the wireless communication unit 304. Further, the modem unit 354 demodulates the signal supplied from the wireless communication unit 304 and supplies the demodulated signal to the control unit 351. The modem unit 354 corresponds to the communication I / F 336 in FIG.
[0224]
The display control unit 355 performs display control for causing the display unit 356 to display the image data supplied from the control unit 351. The display control unit 355 corresponds to the LCD driver 317 in FIG.
[0225]
The display unit 356 performs display in accordance with display control by the display control unit 355. The display unit 356 corresponds to the LCD 316 in FIG.
[0226]
The memory 357 stores data supplied from the control unit 351 and supplies the stored data to the control unit 351. Note that the memory 357 corresponds to the flash memory 334, the DRAM 335, and the like in FIG.
[0227]
Here, in FIG. ₁ , 337 ₂ , 337 _Three , 337 _Four Is a connector constituting the connector portion 337 of the PDA mounting portion 303, and the connector 337 ₁ And 337 ₂ Are connected to the + terminal and the − terminal of the charging circuit 344, respectively. Also, the connector 337 _Three And 337 _Four Is connected to the control unit 351.
[0228]
On the other hand, the + terminal and the − terminal of the battery 222 built in the PDA 101 are the connectors 8 in one or more connectors constituting the connector portion 8 (FIG. 23) provided in the PDA 101. ₁ And 8 ₂ Are connected to each.
[0229]
When the PDA 101 is mounted on the PDA mounting portion 303, the connector 8 on the PDA 101 side ₁ And 8 ₂ Is a connector 337 on the base base computer 102 side. ₁ And 337 ₂ Thus, when the PDA 101 is mounted on the PDA mounting unit 303, the charging circuit 344 of the base base computer 102 is connected to the connector 8 by this. ₁ And 337 ₁ And connector 8 ₂ And 337 ₂ The battery 222 of the PDA 101 is charged via
[0230]
Next, a connection between the PDA 101 and the base base computer 102 when the PDA 101 is attached to the PDA attachment unit 303 of the base base computer 102 will be described with reference to FIG.
[0231]
The connector 8 of the PDA 101 is the connector 8 shown in FIG. ₁ And 8 ₂ In addition, as shown in FIG. _Three And 8 _Four Also has a connector 8 _Three Is connected to the control unit 239 with the connector 8 _Four Are respectively connected to the memory 241.
[0232]
And connector 8 _Three And 8 _Four When the PDA 101 is attached to the PDA attachment part 303 of the base base computer 102, the connector 337 of the connector part 337 is provided. _Three And 337 _Four Are connected to each other.
[0233]
As described above, in the base base computer 102, the connector 337 of the connector portion 337 thereof. _Three And 337 _Four Are connected to the control unit 351, and therefore, the connector 8 constituting the connector unit 8 of the PDA 101. _Three Connector 337 _Three Through the connector 8 _Four Connector 337 _Four All of these are connected to the control unit 351 of the base base computer 102.
[0234]
As a result, the connector 8 in the PDA 101 _Three The control unit 239 connected to the connector 8 _Three And 337 _Three And is electrically connected to the control unit 351 of the base base computer 102. In the PDA 101, the connector 8 _Four The memory 241 connected to the connector 8 _Four And 337 _Four And is electrically connected to the control unit 351 of the base base computer 102.
[0235]
Therefore, the control unit 239 of the PDA 101 and the control unit 351 of the base base computer 102 are connected to the connector 8. _Three And 337 _Three It becomes possible to exchange data via the. Furthermore, the control unit 239 of the PDA 101 can read / write data from / to the memory 357 of the base base computer 102 by sending a request to the control unit 351 of the base base computer 102, and conversely The control unit 351 of 102 can also read / write data from / to the memory 241 of the PDA 101 by issuing a request to the control unit 239 of the PDA 101.
[0236]
The control unit 351 of the base base computer 102 is connected to the connector 337. _Four And 8 _Four Thus, data can be directly read from and written to the memory 241 without going through the control unit 239 of the PDA 101.
[0237]
That is, when the PDA 101 is mounted on the PDA mounting unit 303 of the base base computer 102, the memory 241 of the PDA 101 functions as a part of the base base computer 102. The control unit 351 can access the memory 241 of the PDA 101 as if it is a part of the memory 357 included in the PDA 101.
[0238]
When the PDA 101 and the base base computer 102 are connected via the connector units 8 and 337, other blocks of the PDA 101 can function as a part of the base base computer 102. That is, for example, the control unit 239 of the PDA 101 can function as a part of the control unit 351 of the base base computer 102.
[0239]
Next, the PDA 101 shown in FIG. 23 can perform a voice call by telephone with another PDA 103 and a communication terminal 106 (FIG. 22) capable of communication. That is, when the function mode is set to the telephone mode for performing a voice call by telephone, the PDA 101 performs a call process and a call process shown in the flowchart of FIG.
[0240]
First, the call process will be described with reference to the flowchart of FIG.
[0241]
In the call processing, in step S1, the communication I / F 218 (FIG. 23) stands by using the communication mode as the control channel mode.
[0242]
Here, the communication mode includes a call channel mode in which voice can be transmitted / received to / from a base station (not shown) using a voice call channel, and data is transmitted using a data channel for data transmission / reception. Data channel mode that can transmit / receive data, pilot channel and other control data are exchanged only via the control channel for control, and control channel mode that does not transmit / receive by other channels (so-called standby state) In step S1, the call mode is set to the control channel mode.
[0243]
After that, for example, control data notifying that an incoming call has been received is transmitted via the control channel, or an operation signal indicating that an operation for requesting an outgoing call has been performed. When some event such as that supplied from the unit 224 or the like occurs, the process proceeds to step S2, and the communication I / F 218 determines whether or not there is an incoming call.
[0244]
That is, the communication I / F 218 monitors the data of the control channel that is constantly received by the antenna 64, and in step S2, it is determined whether there is an incoming call based on the data of the control channel.
[0245]
If it is determined in step S <b> 2 that there is an incoming call, the communication I / F 218 supplies a message to that effect to the CPU 202 via the bus 201. When the CPU 202 receives a message indicating that an incoming call has been received, the CPU 202 controls the speaker 10 via the bus 201, the ATRAC encoder / decoder 214, the D / A converter 212, and the amplifier 208 to output a ring tone, Proceed to step S3.
[0246]
In step S3, the communication I / F 218 switches the communication mode from the control channel mode to a call channel mode that uses a call channel for transmitting and receiving voice, and proceeds to step S4. In step S4, the CPU 202 determines whether or not the user has operated the operation unit 224 or the like so as to be in an off-hook state, and returns to step S1 when determining that the user has not operated.
[0247]
If it is determined in step S4 that the operation unit 224 or the like has been operated so as to be in the off-hook state, the process proceeds to step S5, where the communication I / F 218 establishes a communication link with the other party who has received the call, Send and receive audio data for.
[0248]
Thereby, the sound input to the microphone 9 is transmitted as a radio wave from the antenna 64 via the amplifier 209, the A / D converter 211, the ATRAC encoder / decoder 214, the bus 201, and the communication I / F 218. The sound transmitted as radio waves is received by the antenna 64 and output from the speaker 10 via the communication I / F 218, the bus 201, the ATRAC encoder / decoder 214, the D / A converter 212, and the amplifier 208. The
[0249]
Thereafter, the process proceeds to step S6, in which the CPU 202 determines whether or not to end the call, that is, whether the user of the PDA 101 has operated the operation unit 224 to be in an on-hook state or whether the other party is in an on-hook state. If it is determined not to end the call, the process returns to step S5.
[0250]
If it is determined in step S6 that the call is to be terminated, the communication I / F 218 disconnects the communication link with the incoming party and returns to step S1.
[0251]
On the other hand, if it is determined in step S2 that there is no incoming call, the process proceeds to step S7, and the communication I / F 218 determines whether an event for requesting a call (hereinafter referred to as a call event as appropriate) has occurred. .
[0252]
Here, the occurrence of the call event will be described later with reference to the flowchart of FIG.
[0253]
If it is determined in step S7 that there is no call event, the process returns to step S1 and the same processing is repeated thereafter.
[0254]
If it is determined in step S7 that a call event has occurred, the process proceeds to step S8, and the communication I / F 218 switches the communication mode from the control channel mode to the call channel mode and is supplied together with the call event. The telephone number of the other party to call is transmitted from the antenna 64.
[0255]
Thereafter, when the communication partner corresponding to the telephone number enters an off-hook state, the communication I / F 218 establishes a communication link with the communication partner, and proceeds to step S9. In step S9, the communication I / F 218 performs transmission / reception of voice data for a voice call as in step S5.
[0256]
Thereafter, the process proceeds to step S10, and the CPU 202 determines whether or not to end the call as in the case of step S6, and returns to step S9 when it is determined not to end the call.
[0257]
If it is determined in step S10 that the call is to be terminated, the communication I / F 218 disconnects the communication link with the communication partner and returns to step S1.
[0258]
Next, the calling process will be described with reference to the flowchart in FIG.
[0259]
HDD 215 (or flash memory 216) of PDA 101 (FIG. 23) stores a telephone number list in which a telephone number registered in advance by the user and the name of the other party corresponding to the telephone number are associated. In the telephone mode, for example, when the user operates the operation unit 224 to display the telephone number list, the calling process is started.
[0260]
That is, in the calling process, first, in step S21, the CPU 202 reads the telephone number list stored in the HDD 215 via the bus 201, and supplies it to the LCD driver 206 via the bus 201. Proceed to S22.
[0261]
In step S22, the LCD driver 206 displays the telephone number list from the CPU 202 on the LCDs 3 and 4. ₁ , 5 ₁ , 12, 21, 22, or 23.
[0262]
Here, the LCDs 3 and 4 will be appropriately described below. ₁ , 5 ₁ , 12, 21, 22, or 23 is referred to as LCD 3 or the like.
[0263]
Thereafter, when the user selects a certain telephone number from the telephone number list displayed on the LCD 3 or the like, the process proceeds to step S23, and the CPU 202 recognizes the telephone number selected by the user as a telephone number to be called. .
[0264]
Here, on the LCD 3 or the like, a telephone number list is displayed together with a cursor. This cursor designates a certain telephone number on the telephone number list, and when the jog dial 6 (FIG. 7 etc.) as the operation unit 224 (FIG. 23) is rotated, the designated telephone number is changed. It is supposed to move. Further, regarding the telephone number designated by the cursor, when the jog dial 6 is pressed, the selection of the telephone number is confirmed.
[0265]
Therefore, when the user rotates the jog dial 6 to move the cursor to the position of the telephone number of the other party who wants to make a call, and further presses the jog dial 6, in step S23, the telephone number (the cursor is designated). Phone number) is recognized.
[0266]
When a large number of telephone numbers are registered in the telephone number list, it is difficult to display all the telephone numbers on the LCD 3 at a time, but in this case, the telephone numbers cannot be displayed. For telephone numbers, the telephone number list is scrolled and displayed by rotating the jog dial 6.
[0267]
The telephone number can be selected by operating the jog dial 6 or by directly touching the telephone number in the telephone number list displayed on the LCD 3 or the like. That is, when the user touches a telephone number in the telephone number list displayed on the LCD 3 or the like, the touch position is detected by the touch panel 3A or the like and the touch panel driver 205, and the telephone number displayed at the position is It is recognized as the telephone number selected by the user.
[0268]
When the CPU 202 recognizes the telephone number selected by the user in step S23, the CPU 202 proceeds to step S24, associates the telephone number with a call event (message representing the call event), supplies the call number to the communication I / F 218, and makes a call. The call processing is terminated.
[0269]
As described with reference to the flowchart of FIG. 29A, the communication I / F 218 detects that this call event has occurred in step S7, and issues a call to the telephone number associated with the call event. Make a call.
[0270]
Here, a call is made to the telephone number selected by the user from the telephone number list, but the telephone number to be called can also be directly input by the user.
[0271]
That is, on the LCD 3 or the like, as shown in FIG. 2 described above or as shown in FIG. 36 described later, a dial button for inputting a telephone number is displayed, and the operation on the dial button is performed by the touch panel 3A. , 4A ₁ , 5A ₁ , 12A, 21A, 22A, or 23A (hereinafter referred to as the touch panel 3A as appropriate) and the touch panel driver 205, and a call is made to the telephone number corresponding to the detected dial button operation. Is possible.
[0272]
Next, the PDA 101 shown in FIG. 23 transmits and receives e-mails to and from other PDAs 103, the communication terminal 106 (FIG. 22) capable of communication, and other computers on the public network 104 and the Internet 105. Can be done. That is, the PDA 101 performs the mail transmission / reception process and the mail transmission event process shown in the flowchart of FIG.
[0273]
First, the mail transmission / reception process will be described with reference to the flowchart of FIG.
[0274]
In the mail transmission / reception process, in step S31, the communication I / F 218 (FIG. 23) stands by with the communication mode as the control channel mode.
[0275]
Thereafter, when any event occurs, the process proceeds to step S <b> 32, and the communication I / F 218 determines whether the event is an event related to transmission or reception of an electronic mail (hereinafter referred to as a mail transmission / reception event as appropriate).
[0276]
Here, the communication I / F 218 monitors the data of the control channel that is constantly received by the antenna 64. For example, control data notifying that an e-mail has been transmitted is transmitted via the control channel. When it comes, a mail reception event occurs. Further, the CPU 202 generates a mail transmission event when an operation signal indicating that an operation for requesting transmission of an electronic mail is performed is supplied from the operation unit 224 or the like. The mail transmission event and the mail reception event are collectively referred to as a mail transmission / reception event.
[0277]
If it is determined in step S32 that the generated event is not a mail transmission / reception event, the process returns to step S31.
[0278]
If it is determined in step S32 that the generated event is a mail transmission / reception event, the process proceeds to step S33, and the communication I / F 218 uses the data channel for transmitting / receiving data from the control channel mode. Switch to the data channel mode to proceed to step S34. In step S34, the communication I / F 218 establishes a communication link with a base station (not shown) (mail server) (not shown), and transmits / receives e-mail data.
[0279]
Thereby, for example, when the mail transmission / reception event is one indicating that the control data notifying that the electronic mail has been transmitted is transmitted via the control channel (mail reception event), the communication I / O F218 requests an e-mail from the base station (the mail server), and receives e-mail data transmitted from the base station via the antenna 64 in response to the request.
[0280]
Further, for example, when the mail transmission / reception event is one indicating that an operation signal indicating that an operation for requesting transmission of an e-mail has been performed is supplied from the operation unit 224 or the like (mail transmission event), The communication I / F 218 transmits e-mail data supplied via the bus 201 to the base station (mail server) via the antenna 64.
[0281]
Thereafter, the process proceeds to step S35, in which the CPU 202 has received or transmitted e-mail data, that is, has received all e-mail data addressed to itself stored in the base station (mail server), or Then, it is determined whether or not all the e-mail data requested to be transmitted has been transmitted.
[0282]
If it is determined in step S35 that transmission / reception of e-mail data has not been completed, the process returns to step S34, and transmission / reception of e-mail data that has not been transmitted / received is continued.
[0283]
If it is determined in step S35 that the transmission / reception of e-mail data has been completed, the process returns to step S31, and the same processing is repeated thereafter.
[0284]
Next, mail transmission event processing will be described with reference to the flowchart of FIG.
[0285]
In the HDD 215 (or flash memory 216) of the PDA 101 (FIG. 23), in addition to the telephone number list described above, an email address registered in advance by the user and the name of the other party corresponding to the email address are associated with each other. In the e-mail mode, for example, when the user operates the operation unit 224 or the like to display the mail address list, the mail transmission event process is started.
[0286]
That is, in the mail transmission event process, first, in step S41, the CPU 202 reads the mail address list stored in the HDD 215 via the bus 201, and supplies it to the LCD driver 206 via the bus 201. Proceed to step S42.
[0287]
In step S42, the LCD driver 206 displays the mail address list from the CPU 202 on the LCD 3 or the like.
[0288]
Thereafter, when the user selects a certain e-mail address from the e-mail address list displayed on the LCD 3 or the like, the process proceeds to step S43, and the CPU 202 recognizes the e-mail address selected by the user as an e-mail destination. To do.
[0289]
Here, the display of the mail address list on the LCD 3 or the like or the selection of the e-mail address from the mail address list is the same as when the telephone number list described in FIG. 29 is displayed or when the telephone number is selected. To be done. Also, the user can directly input the e-mail address in the same manner as the telephone number described in FIG.
[0290]
The mail address list and the telephone number list can be integrated into one list, that is, a list in which the user's name is associated with the electronic mail address and the telephone number.
[0291]
When the e-mail address as the e-mail destination is recognized in step S43, the CPU 202 waits for the input of text as the body of the e-mail, proceeds to step S44, and controls the LCD driver 206 via the bus 201. By doing so, the input text is displayed on the LCD 3 or the like.
[0292]
Here, the text that becomes the body of the e-mail is input by, for example, the user performing a rotation operation and a pressing operation of the jog dial 6 (FIG. 7 or the like) as the operation unit 224. That is, when the jog dial 6 is rotated, the CPU 202 controls the LDC driver 206 to display characters to be input together with the cursor on the LCD 3 or the like in response to the rotation operation. Further, when the cursor designates a character and the jog dial 6 is pressed, the CPU 202 determines the character as a text input.
[0293]
Further, for example, as will be described later, the user can input text as the body of the e-mail by operating a button or the like displayed on the LCD 3 or the like. That is, in this case, the CPU 202 controls the LCD driver 206 to display a button for inputting characters on the LCD 3 or the like. When the user touches a button displayed on the LCD 3 or the like, the touched button is detected by the touch panel 3A or the like and the touch panel driver 205, and a character corresponding to the button is determined as a text input.
[0294]
For e-mail, it is possible to specify an e-mail address that is a so-called carbon copy destination, a file attached to the e-mail, etc., and when such specification is made In step S44, display reflecting the designation is performed on the LCD 3 or the like.
[0295]
In addition, for e-mail, it is possible to edit text that has already been input in the same manner as text input, and when the operation unit 224 or the like is operated to perform such editing. In step S44, display reflecting the edited content is performed on the LCD 3 or the like.
[0296]
Thereafter, the process proceeds to step S45, where the CPU 202 determines whether or not the user has operated the operation unit 224 or the like so that the input of the text that is the body of the e-mail has been completed, and such an operation is not performed. When it determines, it returns to step S44.
[0297]
If it is determined in step S45 that the user has operated the operation unit 224 or the like as if the input of the text as the body of the e-mail has been completed, the process proceeds to step S46, and the CPU 202 ends the input of the body It is determined whether the user has operated the operation unit 224 or the like so as to instruct transmission of an e-mail.
[0298]
If it is determined in step S46 that transmission of an e-mail has not been instructed, the process returns to step S41 after waiting for the user to operate the operation unit 224 or the like so as to display a mail address list.
[0299]
In this case, the e-mail is stored in the HDD 215, for example, and then transmitted at an arbitrary timing or when an instruction is given from the user.
[0300]
On the other hand, if it is determined in step S46 that the transmission of an e-mail has been instructed, that is, if an operation signal requesting e-mail transmission is supplied from the operation unit 224 or the like to the CPU 202, the process proceeds to step S47. The CPU 202 supplies a mail transmission event to the communication I / F 218 via the bus 201, waits for the user to operate the operation unit 224 or the like so as to display the mail address list, and returns to step S41. .
[0301]
As described with reference to the flowchart of FIG. 30A, the communication I / F 218 detects that this mail transmission event has occurred in step S32, and transmits an electronic mail.
[0302]
Here, if it is determined in step S45 that the input of the text that is the body of the e-mail has been completed, the CPU 202 includes the e-mail address of the user of the PDA 101 as the sender in the e-mail. Yes. It is assumed that the e-mail address of the user of the PDA 101 is stored in the HDD 215 when the user operates the operation unit 224 or the like.
[0303]
Next, the PDA 101 shown in FIG. 23 can transmit and receive data such as binary data. Here, the PDA 101 can perform data transmission / reception via the base station, or directly with the other PDA 103 and the communication terminal 106 (FIG. 22) capable of communication without going through the base station. You can also do that. When the PDA 101 is set to a data mode in which the function mode performs transmission / reception of various types of data with other PDAs 103 and the communication terminal 106 capable of communication, the data transmission / reception process and the data transmission event process illustrated in the flowchart of FIG. I do.
[0304]
First, data transmission / reception processing will be described with reference to the flowchart of FIG.
[0305]
In the data transmission / reception process, in step S51, the communication I / F 218 (FIG. 23) stands by with the communication mode as the control channel mode.
[0306]
Thereafter, when any event occurs, the process proceeds to step S52, and the communication I / F 218 determines whether the event is an event related to data transmission or reception (hereinafter, referred to as a data transmission / reception event as appropriate).
[0307]
Here, the communication I / F 218 monitors the data of the control channel that is constantly received by the antenna 64. For example, control data notifying that data has been transmitted is transmitted via the control channel. And a data reception event occurs. Further, when an operation signal indicating that an operation for requesting data transmission has been performed is supplied from the operation unit 224 or the like, the CPU 202 generates a data transmission event. The data transmission event and the data reception event are collectively referred to as a data transmission / reception event.
[0308]
If it is determined in step S52 that the event that has occurred is not a data transmission / reception event, the process returns to step S51.
[0309]
If it is determined in step S52 that the generated event is a data transmission / reception event, the process proceeds to step S53, and the communication I / F 218 changes the communication mode from the control channel mode to the data channel mode using the data channel. Switch to step S54. In step S54, the communication I / F 218 transmits / receives data to / from another PDA 103 or the like.
[0310]
Thereby, for example, when the data transmission / reception event indicates that the control data notifying that the data has been transmitted is transmitted via the control channel (data reception event), the communication I / F 218. Receives data transmitted from another PDA 103 or the like via a data channel via the antenna 64.
[0311]
Further, for example, when the mail transmission / reception event is an operation signal indicating that an operation for requesting data transmission has been performed (data transmission event) indicating that the operation signal is supplied from the operation unit 224 or the like, communication is performed. The I / F 218 transmits data supplied via the bus 201 to another PDA 103 or the like via the antenna 64.
[0312]
Here, the data transmission / reception between the PDA 101 and the other PDA 103 is performed by radio waves via the antenna 64, but the data transmission / reception between the PDA 101 and the other PDA 103, etc. For example, it can be performed by infrared rays through the wireless communication unit 63. In addition, as described above, communication between the PDA 101 and the PDA 103 is performed in a non-contact manner by radio waves or infrared rays (other electromagnetic waves or the like), or in a wired manner (in a contact state) via the connector unit 8 (FIG. 23). It is also possible to do this.
[0313]
In step S54, when data is received, the process proceeds to steps S55 and S56 sequentially. When data is transmitted, steps S55 and S56 are skipped and the process proceeds to step S57.
[0314]
In step S55, the CPU 202 determines the data type of the data received by the communication I / F 218. That is, in the present embodiment, the data includes which data type is image data, audio data, a program, or the like, and when the data is encoded, A data identifier representing information such as what encoding method is used is added. In step S55, the CPU 202 refers to the data identifier added to the data received by the communication I / F 218. Thus, the data type is determined.
[0315]
In step S56, the CPU 202 transfers the data received by the communication I / F 218 to the HDD 215 via the bus 201, stores the data according to the data type, and proceeds to step S57.
[0316]
That is, in the present embodiment, data is stored separately for each type, for example, in separate directories or folders.
[0317]
In the present embodiment, data is stored as a file.
[0318]
In addition, here, the received data is stored in the HDD 215, but the data can also be stored in the flash memory 216 or the DRAM 217.
[0319]
Furthermore, when the received data is, for example, unencoded image data or audio data, the image data or audio data can be encoded and stored in the HDD 215. Here, the encoding of the image data can be performed by the MPEG encoder / decoder 213. The audio data can be encoded by the ATRAC encoder / decoder 214. Furthermore, encoding of image data and audio data can be performed by other encoding methods by the CPU 202 executing a program.
[0320]
In step S57, the CPU 202 determines whether transmission / reception of data has been completed, that is, whether all data transmitted from the other PDA 103 has been received, or whether all data requested to be transmitted has been transmitted. judge.
[0321]
If it is determined in step S57 that data transmission / reception has not ended, the process returns to step S54, and data transmission / reception that has not yet been performed is continued.
[0322]
If it is determined in step S57 that the data transmission / reception has ended, the process returns to step S51, and the same processing is repeated thereafter.
[0323]
Next, data transmission event processing will be described with reference to the flowchart in FIG.
[0324]
In the data mode, for example, when the user operates the operation unit 224 or the like so as to display the mail address list, the data transmission event process is started.
[0325]
That is, in the data transmission event process, the same processes as in steps S41 to S43 in FIG. 30B are performed in steps S61 to S63, whereby the CPU 202 sends an e-mail address as a data transmission destination. Recognize
[0326]
Here, the e-mail address is adopted as the information indicating the data transmission destination, but the transmission destination is, for example, an IP (Internet Protocol) address, a MAC (Media Access Control) address, or a user ID. When it can be specified by (Identification) or the like, such information can be adopted as information indicating a data transmission destination.
[0327]
Thereafter, the process proceeds to step S64, where the CPU 202 configures a data list and supplies the data list to the LCD driver 206 via the bus 201, thereby displaying the data list on the LCD 3 or the like.
[0328]
That is, the CPU 202 accesses the HDD 215 and obtains file names of files such as image data, audio data, and programs stored therein, for example. Further, the CPU 202 forms a data list in which the file names are in a list format, and displays this data list on the LCD 3 or the like.
[0329]
After the data list is displayed on the LCD 3 or the like, the process proceeds to step S65, and the CPU 202 determines whether any data (here, the file name) is selected from the data list by the user. .
[0330]
That is, for the data list, the user selects the jog dial 6 (FIG. 7) as the operation unit 224 in the same manner as when selecting a telephone number from the telephone number list or selecting an e-mail address from the mail address list. Etc.), the data (file name) can be selected from the data list. In step S65, the user can select any data from the data list. It is determined whether or not any data has been selected.
[0331]
If it is determined in step S65 that no data has been selected, the process returns to step S64, and the same processing is repeated thereafter.
[0332]
If it is determined in step S65 that data has been selected, the process proceeds to step S66, and the CPU 202 instructs to transmit data determined to be selected in step S65 (hereinafter referred to as selection data as appropriate). It is determined whether the user has operated the operation unit 224 or the like.
[0333]
If it is determined in step S66 that transmission of selection data is not instructed, the process returns to step S61 after waiting for the user to operate the operation unit 224 or the like so as to display the mail address list.
[0334]
If it is determined in step S66 that transmission of selection data is instructed, that is, if an operation signal for requesting transmission of selection data is supplied from the operation unit 224 or the like to the CPU 202, the process proceeds to step S67. The CPU 202 supplies the data transmission event to the communication I / F 218 via the bus 201, waits for the user to operate the operation unit 224 or the like so as to display the mail address list, and returns to step S61. .
[0335]
As described with reference to the flowchart of FIG. 31A, the communication I / F 218 detects that this data transmission event has occurred in step S52 and transmits the selected data.
[0336]
The selection data is transmitted with the e-mail address recognized in step S63 of FIG. 31B as the destination.
[0337]
Next, the PDA 101 shown in FIG. 23 can reproduce image data and audio data stored (recorded) in the HDD 215. That is, when the function mode is set to a data reproduction mode in which image data or audio data is reproduced, the PDA 101 performs data reproduction processing shown in the flowchart of FIG.
[0338]
The HDD 215 records the image data captured by the CCD camera 65 (FIG. 23) encoded by the MPEG encoder / decoder 213 or an encoder realized by the CPU 202 executing a program. be able to. Further, the HDD 215 records audio data input from the earphone microphone jack 7 or the microphone 9 encoded by the ATRAC encoder / decoder 214 or an encoder realized by the CPU 202 executing a program. be able to. The HDD 215 can record the encoded image data and audio data received by the data transmission / reception process described with reference to FIG.
[0339]
In the data reproduction process of FIG. 32, the data recorded in the HDD 215 is reproduced in this way.
[0340]
That is, in the data reproduction mode, in step S71, the CPU 202 determines whether the user has operated the operation unit 224 or the like so as to request display of the data list, and determines that such an operation has not been performed. If so, the process returns to step S71.
[0341]
If it is determined in step S71 that the user has operated the operation unit 224 or the like so as to request the display of the data list, the process proceeds to step S72, and the CPU 202 configures the data list via the bus 201. By supplying the data to the LCD driver 206, the data list is displayed on the LCD 3 or the like.
[0342]
That is, the CPU 202 accesses the HDD 215 and acquires the file names of the image data and audio data files stored therein. Further, the CPU 202 configures a data list in which the file names are in a list format, and displays this data list on the LCD 3 or the like.
[0343]
After the data list is displayed on the LCD 3 or the like, the process proceeds to step S73, and the CPU 202 determines whether any data (here, the file name) has been selected from the data list by the user. The determination is made in the same manner as in step S65 of 31 (B).
[0344]
If it is determined in step S73 that no data is selected, the process returns to step S72, and thereafter the same processing is repeated.
[0345]
If it is determined in step S73 that the data has been selected, the CPU 202 further determines whether the selected data is image data or audio data (audio data). If the data is image data, the process proceeds to step S74. If the data is audio data, the process proceeds to step S78.
[0346]
In step S74, the CPU 202 reads the image data (file) determined to be selected in step S73 from the HDD 215, and proceeds to step S75. In step S <b> 75, the CPU 202 performs a process for decoding the image data read from the HDD 215.
[0347]
That is, the image data stored in the HDD 215 is encoded as described above, and in step S75, the encoded image data is decoded.
[0348]
Here, decoding of the image data is performed by an MPEG encoder / decoder 213 or a decoder realized by the CPU 202 executing a program.
[0349]
When the image data is decoded in step S75, the process proceeds to step S76, and the CPU 202 supplies the image data to the LCD driver 206 via the bus 201 and displays it on the LCD 3 or the like.
[0350]
Thereafter, the process proceeds to step S77, where the CPU 202 determines whether or not the reproduction of all the image data determined to have been selected in step S73 has been completed. Repeat the same process. That is, the reproduction of the image data is thereby continued.
[0351]
If it is determined in step S77 that the reproduction of all the image data determined to have been selected in step S73 is completed, the process returns to step S71, and the same processing is repeated thereafter.
[0352]
On the other hand, in step S78, the CPU 202 reads out the audio data (file) determined to have been selected in step S73 from the HDD 215, and proceeds to step S79. In step S79, the CPU 202 performs a process of decoding the audio data read from the HDD 215.
[0353]
That is, the audio data stored in the HDD 215 is encoded as described above, and in step S79, the encoded audio data is decoded.
[0354]
Here, decoding of the audio data is performed by an ATRAC encoder / decoder 214 or a decoder realized by the CPU 202 executing a program.
[0355]
When the audio data is decoded in step S79, the process proceeds to step S80, and the CPU 202 transfers the audio data to the earphone microphone jack via the bus 201, the ATRAC encoder / decoder 214, the D / A converter 212, and the amplifier 208. 7 or speaker 10 for output.
[0356]
Thereafter, the process proceeds to step S81, where the CPU 202 determines whether or not the reproduction of all the audio data determined to have been selected in step S73 has been completed. Repeat the same process. That is, the reproduction of the audio data is thereby continued.
[0357]
If it is determined in step S81 that the reproduction of all the audio data determined to have been selected in step S73 is completed, the process returns to step S71, and the same processing is repeated thereafter.
[0358]
Here, the image data or audio data stored in the HDD 215 is reproduced, but in addition, there is a program downloaded and installed in the HDD 215 from a server (not shown) on the Internet 105 (FIG. 22). If it is recorded, a data list including the file name of the program is displayed in step S72, and when the program is selected by the user, the program can be executed by the CPU 202.
[0359]
If the data determined to be selected in step S73 includes both image data and audio data, the processes in steps S74 to S77 and steps S78 to S81 are performed in parallel.
[0360]
Next, the PDA 101 shown in FIG. 23 receives image data and audio data transmitted via the public network 104 by, for example, so-called push distribution from a server (not shown) on the Internet 105 (FIG. 22). It is possible to perform streaming playback such as. That is, when the function mode is set to the streaming playback mode for performing streaming playback, the PDA 101 performs the streaming playback processing shown in the flowchart of FIG.
[0361]
In the streaming playback process, in step S91, the communication I / F 218 (FIG. 23) stands by using the communication mode as the control channel mode.
[0362]
Thereafter, when control data indicating that some data is to be transmitted is transmitted through the control channel, the control data is received by the communication I / F 218 via the antenna 64, and the process proceeds to step S92, where the communication I / F 218 is transmitted. Determines whether data for streaming reproduction (hereinafter referred to as streaming data as appropriate) is transmitted based on the control data.
[0363]
If it is determined in step S92 that streaming data is not transmitted, the process returns to step S91, and the same processing is repeated thereafter.
[0364]
If it is determined in step S92 that streaming data is transmitted, the process proceeds to step S93, and the communication I / F 218 switches the communication mode from the control channel mode to the data channel mode, and then proceeds to step S94. In step S94, the communication I / F 218 starts receiving streaming data transmitted via the data channel.
[0365]
The streaming data received by the communication I / F 218 is supplied to the DRAM 217 via the bus 201 and temporarily stored.
[0366]
When the storage of streaming data is started in the DRAM 217, the CPU 202 proceeds to step S95, and the CPU 202 determines the data type of the streaming data stored in the DRAM 217. That is, as in the case of the data transmission / reception process of FIG. 31A, a data identifier indicating the type of the streaming data is added to the streaming data. In step S95, the CPU 202 stores the data in the DRAM 217. The data type is discriminated by referring to the data identifier of the streaming data.
[0367]
In step S96, the CPU 202 determines whether the streaming data stored in the DRAM 217 is image data or audio data (audio data) based on the data type determined in step S95.
[0368]
If it is determined in step S96 that the streaming data stored in the DRAM 217 is image data, the process proceeds to step S97, where the CPU 202 reads out the image data stored in the DRAM 217 and decodes the image data. .
[0369]
That is, the streaming data is encoded by, for example, MPEG or another encoding method, and in step S95, the encoded image data is decoded.
[0370]
Here, decoding of the image data is performed by an MPEG encoder / decoder 213 or a decoder realized by the CPU 202 executing a program.
[0371]
When the image data is decoded in step S97, the process proceeds to step S98, and the CPU 202 supplies the image data to the LCD driver 206 via the bus 201 and displays it on the LCD 3 or the like.
[0372]
Thereafter, the process proceeds to step S99, where the CPU 202 determines whether or not the reproduction of all the streaming data (here, image data) stored in the DRAM 217 has been completed. Then, the decoding and display (reproduction) of the streaming data stored in the DRAM 217 is continued.
[0373]
If it is determined in step S99 that all playback of the streaming data stored in the DRAM 217 has been completed, the process returns to step S91, and the same processing is repeated thereafter.
[0374]
On the other hand, if it is determined in step S96 that the streaming data is audio data, the process advances to step S100, and the CPU 202 reads the audio data stored in the DRAM 217 and performs a process of decoding the audio data.
[0375]
That is, the streaming data is encoded by, for example, ATRAC or another encoding method, and in step S100, the encoded audio data is decoded.
[0376]
Here, decoding of the audio data is performed by an ATRAC encoder / decoder 214 or a decoder realized by the CPU 202 executing a program.
[0377]
When the audio data is decoded in step S100, the process proceeds to step S101, and the CPU 202 transmits the audio data to the earphone microphone jack via the bus 201, the ATRAC encoder / decoder 214, the D / A converter 212, and the amplifier 208. 7 or speaker 10 for output.
[0378]
Thereafter, the process proceeds to step S102, where the CPU 202 determines whether or not all the reproduction of the streaming data (here, audio data) stored in the DRAM 217 has been completed. Returning to FIG. 4, the decoding and output (reproduction) of the streaming data stored in the DRAM 217 are continued.
[0379]
If it is determined in step S102 that all the streaming data stored in the DRAM 217 has been reproduced, the process returns to step S91, and the same processing is repeated thereafter.
[0380]
If the streaming data includes both image data and audio data, the processes in steps S97 to S99 and steps S100 to S102 are performed in parallel.
[0381]
Next, the PDA 101 in FIG. 23 can exchange various data (files) with the base base computer 102 in FIG.
[0382]
A procedure for exchanging files between the PDA 101 and the base base computer 102 will be described with reference to FIGS. 34 and 35. FIG.
[0383]
Note that file exchange between the PDA 101 (FIG. 23) and the base base computer 102 (FIG. 26) is communication by radio waves via the antennas 64 and 343, and wired communication via the connector units 8 and 337. And communication using infrared rays via the wireless communication units 63 and 304 are possible. However, here, for example, files are exchanged between the PDA 101 and the base base computer 102 by performing communication using radio waves via the antennas 64 and 343.
[0384]
First, a procedure when a file is transmitted from the PDA 101 to the base base computer 102 will be described with reference to FIG.
[0385]
The CPU 312 (corresponding to the control unit 351 in FIG. 27) of the base base computer 102 (FIG. 26) receives from the PDA 101 when the user operates the keyboard 301 (corresponding to the operation unit 353 in FIG. 27) or the like. Waiting for the designation of a file (A1), the designated file is selected as a file to be received from the PDA 101 (A2). At this time, the CPU 202 (corresponding to the control unit 239 in FIG. 24) of the PDA 101 (FIG. 23) stands by in the control channel mode (B1).
[0386]
34, for example, it is assumed that the base base computer 102 has already acquired a list of file names of files stored in the HDD 215 of the PDA 101 by communicating with the PDA 101, and the user The file designation (A1) is performed from the list of file names.
[0387]
After selecting a file to be received from the PDA 101, the CPU 312 of the base base computer 102 controls the communication I / F 336 to thereby request the PDA 101 to receive the selected file via the antenna 343. A signal is transmitted that identifies the selected file, eg, along with the file name (A3).
[0388]
A reception request signal from the base base computer 102 is received by the communication I / F 218 via the antenna 64 in the PDA 101 (B2) and supplied to the CPU 202.
[0389]
When receiving the reception request signal, the CPU 202 of the PDA 101 controls the communication I / F 218 to request an authentication process from the base base computer 102, whereby the CPU 202 of the PDA 101 and the CPU 312 of the base base computer 102 are requested. Authentication processing for confirming whether or not the devices are mutually valid (A4, B3).
[0390]
When the authentication is successful in the CPU 202 of the PDA 101 and the CPU 312 of the base base computer 102, the CPU 202 of the PDA 101 controls the communication I / F 218 to send a reception permission signal indicating that the reception of the file is permitted. 102 (B4).
[0390]
If the authentication fails, the subsequent processing is not performed, and accordingly, the file transmission from the PDA 101 to the base base computer 102 is not performed.
[0392]
The reception permission signal transmitted from the PDA 101 is received by the communication I / F 336 via the antenna 343 in the base base computer 102 (A5) and supplied to the CPU 312 via the bus 311. Thereby, the CPU 312 recognizes that a file is transmitted from the PDA 101.
[0393]
Thereafter, the CPU 202 of the PDA 101 reads out the file with the file name transmitted from the base base computer 102 together with the reception request signal, for example, from the HDD 215, and supplies the file to the communication I / F 218 via the bus 201. Is transmitted to the base base computer 102 via the antenna 64 (B5).
[0394]
In the base base computer 102, the file transmitted from the PDA 101 is received by the communication I / F 336 via the antenna 343 (A6). The communication I / F 336 transfers the received file to the DRAM 335 via the bus 311 and temporarily stores it. The CPU 312 stores the file based on the file identification signal of the file thus stored in the DRAM 335. For example, it is determined which file type is image data, audio data, program, text data, or the like (A7).
[0395]
That is, a file identification signal indicating the file type is added to the file, and the file type is discriminated by referring to the file identification signal.
[0396]
Then, the CPU 312 transfers the file stored in the DRAM 335 to the HDD 333 via the bus 311 and stores it according to the file type (A8).
[0397]
That is, in this embodiment, the base base computer 102 divides the file into different directories or folders for each file type and stores them in the HDD 333.
[0398]
Next, a procedure when a file is transmitted from the base base computer 102 to the PDA 101 will be described with reference to FIG.
[0399]
The CPU 312 (corresponding to the control unit 351 in FIG. 27) of the base base computer 102 (FIG. 26) is stored in the HDD 333 when the user operates the keyboard 301 (corresponding to the operation unit 353 in FIG. 27) or the like. After waiting for designation of a file to be transmitted to the PDA 101 (A11), the designated file is selected as a file to be transmitted to the PDA 101 (A12). At this time, the CPU 202 (corresponding to the control unit 239 in FIG. 24) of the PDA 101 (FIG. 23) stands by in the control channel mode (B11).
[0400]
After selecting a file to be transmitted to the PDA 101, the CPU 312 of the base base computer 102 controls the communication I / F 336 to thereby request the PDA 101 to transmit the selected file via the antenna 343. A signal is transmitted to identify the selected file, for example, along with the file name (A13).
[0401]
The transmission request signal from the base base computer 102 is received by the communication I / F 218 via the antenna 64 in the PDA 101 (B12) and supplied to the CPU 202.
[0402]
When receiving the transmission request signal, the CPU 202 of the PDA 101 controls the communication I / F 218 to request an authentication process from the base base computer 102, whereby the CPU 202 of the PDA 101 and the CPU 312 of the base base computer 102 are requested. Authentication processing is performed to confirm whether or not the devices are valid with each other (A14, B13).
[0403]
When the authentication is successful in the CPU 202 of the PDA 101 and the CPU 312 of the base base computer 102, the CPU 202 of the PDA 101 controls the communication I / F 218 to send a transmission permission signal indicating permission of file transmission to the base base computer. 102 (B14).
[0404]
If the authentication fails, the subsequent processing is not performed as in the case of FIG. 34. Therefore, the file transmission from the base base computer 102 to the PDA 101 is not performed.
[0405]
The transmission permission signal transmitted from the PDA 101 is received by the communication I / F 336 via the antenna 343 in the base base computer 102 (A15) and supplied to the CPU 312 via the bus 311. As a result, the CPU 312 recognizes that the file transmission from the PDA 101 is permitted.
[0406]
Thereafter, the base base computer 102 reads the file with the file name transmitted to the PDA 101 together with the transmission request signal from the HDD 333, for example, and supplies the file to the communication I / F 336 via the bus 311. The data is transmitted to the PDA 101 via the antenna 343 (A16).
[0407]
In the PDA 101, the file transmitted from the base base computer 102 is received by the communication I / F 218 via the antenna 64 (B15). The communication I / F 218 transfers the received file to the DRAM 217 via the bus 201 for temporary storage, and the CPU 202 stores the file based on the file identification signal of the file stored in the DRAM 217 in this way. For example, it is determined which file type is image data, audio data, program, text data, or the like (B16).
[0408]
That is, as described with reference to FIG. 34, a file identification signal indicating the file type is added to the file, and the file type is determined by referring to the file identification signal.
[0409]
Then, the CPU 202 transfers the file stored in the DRAM 217 to the flash memory 216 via the bus 201 and stores it in accordance with the file type (B17).
[0410]
In the PDA 101, the file can be stored in the HDD 215 instead of being stored in the flash memory 216.
[0411]
Next, a method for displaying information on the PDA 101 will be described.
[0412]
In the PDA 101, for example, as shown in FIG. 10, when the lid 20 is opened and the subpanels 15 and 16 are opened, six LCDs 4 are arranged on substantially the same plane. ₁ , 5 ₁ , 12, 21, 22, 23 are arranged next to each other.
[0413]
Therefore, the PDA 101 includes, for example, six LCDs 4 as shown in FIG. ₁ , 5 ₁ , 12, 21 to 23 are regarded as one screen, and one information can be displayed on the one screen.
[0414]
That is, FIG. 36 shows six LCDs 4 when the function mode of the PDA 101 is the telephone mode. ₁ , 5 ₁ , 12, 21 to 23 are shown.
[0415]
In the embodiment of FIG. 36, six LCDs 4 ₁ , 5 ₁ , 12, 21 to 23 as one screen, a group of buttons for causing the PDA 101 to function as a telephone is displayed on the one screen.
[0416]
Specifically, in FIG. ₁ Shows the entire off-hook button for setting the off-hook state, and the LCD 5 ₁ Includes a dial button for inputting a telephone number, an entire button for inputting a number “7”, a part of a button for inputting a number “8”, an entire button for inputting a symbol “*”, and the number “ A part of buttons for inputting “0” are displayed. Further, the LCD 12 includes an entire button for inputting a number “1”, a part of a button for inputting a number “2”, an entire button for inputting a number “4”, and a button for inputting a number “5”. A part of the button is displayed on the LCD 21. The remaining part of the button for inputting the number “2”, the entire button for inputting the number “3”, the remaining part of the button for inputting the number “5”, and The entire button for inputting the number “6” is displayed. The LCD 22 displays an entire on-hook button for setting the on-hook state, and the LCD 23 displays the remaining part of the button for inputting the number “8”, the entire button for inputting the number “9”, The remaining part of the button for inputting the number “0” and the whole button for inputting the symbol “#” are displayed.
[0417]
Of the dial buttons, a button for inputting a number can also input hiragana and alphabets used when describing the body of an e-mail.
[0418]
LCD4 ₁ , 5 ₁ , 12, 21, 22, 23, as described above, the operation of the button is performed on the LCD 4. ₁ , 5 ₁ , 12, 21, 22, and 23, the touch panel 4A is integrally formed. ₁ , 5A ₁ , 12A, 21A, 22A, 23A.
[0419]
Accordingly, an operation on a button that is entirely displayed on a certain LCD is detected by a touch panel integrally formed with the LCD, but an operation on a button that is displayed across a plurality of LCDs is Any of the plurality of touch panels configured integrally with each of the plurality of LCDs is detected.
[0420]
In other words, in FIG. 36, for example, the button for inputting the number “1” is entirely displayed on the LCD 12, and therefore, an operation on the button is detected by the touch panel 12 </ b> A configured integrally with the LCD 12. . Further, for example, since the button for inputting the number “2” is displayed across the LCDs 12 and 21, the operation on the button can be performed according to the operation position of the touch panel 12 </ b> A configured integrally with the LCD 12, or It is detected by any one of the touch panels 21 </ b> A configured integrally with the LCD 21.
[0421]
Therefore, the user touches the button for inputting the number “2” displayed on the LCD 12 or touches the display portion of the button for inputting the number “2” displayed on the LCD 21. Also, the number “2” can be input.
[0422]
As described above, the six LCDs 4 ₁ , 5 ₁ , 12, 21 to 23, when the information is displayed as one screen, the information can be displayed larger than when displayed on one LCD, and the operability is improved. Can be made.
[0423]
Here, displaying information on one screen on a plurality of LCDs as described above is hereinafter referred to as multi-screen display as appropriate.
[0424]
Next, six LCD4 ₁ , 5 ₁ , 12, 21 to 23, as described above, the information is displayed on the entire screen as one screen, and six LCDs 4 ₁ , 5 ₁ , 12, 21 to 23 as one screen, it is also possible to display one information for each screen.
[0425]
That is, FIG. 37 shows six LCDs 4 ₁ , 5 ₁ , 12, 21 to 23 are shown as one screen, and a display example is shown in which one information is displayed for each screen. In the embodiment of FIG. 37 (the same applies to FIGS. 38 and 39 described later), an LCD is schematically drawn.
[0426]
In the embodiment of FIG. 37A, six LCDs 4 ₁ , 5 ₁ , 12, 21 to 23, menu items are displayed. That is, LCD4 ₁ Includes the item “News” on the LCD 5 ₁ The item “music” is displayed on the LCD 12, the item “weather forecast” is displayed on the LCD 12, the item “mail” is displayed on the LCD 21, the item “movie” is displayed on the LCD 22, and the item “phone” is displayed on the LCD 23. Each is displayed.
[0427]
In the state shown in FIG. 37A, when the user touches, for example, the LCD 22 on which the item “movie” is displayed, the touch is detected by the touch panel 22A integrally formed with the LCD 22, for example, As shown in FIG. 37 (B), each of a plurality of pieces of information linked to the item “movie” again has six LCDs 4. ₁ , 5 ₁ , 12, 21 to 23, respectively.
[0428]
That is, in the embodiment of FIG. ₁ , 5 ₁ , 12, 21 to 23, movie titles # 1 to # 6 are displayed.
[0429]
In the state shown in FIG. 37 (B), the user displays, for example, the LCD 4 on which the movie title # 1 is displayed. ₁ When touching, the touch is LCD4 ₁ Touch panel 4A configured integrally with ₁ For example, as shown in FIG. 37 (C), each of a plurality of pieces of information linked to the movie title # 1 has six LCDs 4. ₁ , 5 ₁ , 12, 21 to 23, respectively.
[0430]
That is, in the embodiment of FIG. ₁ Is displayed with a button (hereinafter referred to as “start” button as appropriate) in which characters of “start” are represented. LCD5 ₁ , 12, 21 to 23 display a plurality of images related to the movie, such as the director of the title # 1, the screenwriter, the leading actor, the supporting actor, and the interview.
[0431]
In the state shown in FIG. 37 (C), the user displays, for example, the LCD 4 on which a “start” button is displayed. ₁ When touching, the PDA 101 starts to reproduce the image data of the movie of title # 1, for example, and the reproduced image is, for example, six LCDs 4 ₁ , 5 ₁ , 12, 21 to 23 are displayed on a single screen.
[0432]
Also, in the state shown in FIG. 37C, when the user touches the LCD on which, for example, an image of a movie director is displayed, the PDA 101 introduces the director, such as the director's career and other works. For example, six LCDs 4 ₁ , 5 ₁ , 12, 21 to 23 are displayed on a single screen.
[0433]
As described above, the six LCDs 4 ₁ , 5 ₁ , 12, 21 to 23 as one screen, and when displaying one piece of information for each screen, each piece of information is displayed on a different LCD for each piece of information. When one piece of information is selected from the information, the information can be selected at an interval such that the LCD on which the information is displayed is selected.
[0434]
That is, in the case of multi-screen display, as shown in FIG. 36, a button may be displayed across two LCDs. In this case, any of the portions where the buttons of the two LCDs are displayed. Even if is touched, the button can be operated. However, when one button is displayed over two LCDs in this way, one button corresponds to two LCDs, so the correspondence between buttons and LCDs. Is not one-to-one, the user may be confused by the operation.
[0435]
On the other hand, as shown in FIG. ₁ , 5 ₁ , 12, 21 to 23 as one screen, and when one information is displayed for each screen, the correspondence between one information and one LCD is one-to-one. Information displayed on the LCD can be easily distinguished from information displayed on other LCDs. Therefore, the user can easily and accurately select desired information.
[0436]
Furthermore, when displaying one piece of information on one LCD, information can be displayed larger than when displaying a plurality of pieces of information on one LCD. Can be done.
[0437]
When one LCD screen is divided into a plurality of areas and one piece of information is displayed in each area, it is possible to display the same display as that shown in FIG. However, in order to display information in the same size as in the case of FIG. 37 by dividing one LCD into a plurality of areas, it is necessary to adopt a large LCD as the single LCD, which increases the size of the entire apparatus. . Furthermore, when one LCD is divided into a plurality of areas and a button is displayed in each area, when the user touches the boundary between the two areas, it is displayed in any of the two areas. It becomes difficult to determine whether the button has been operated.
[0438]
On the other hand, when one piece of information is displayed for each of a plurality of rotatable LCDs as in the embodiment of FIG. 37, the size of the device as described above is increased and the user's operation is determined. Can be prevented from becoming difficult.
[0439]
Here, as described above, displaying one piece of information on each of a plurality of LCDs is hereinafter referred to as directory display as appropriate.
[0440]
6 LCD4 ₁ , 5 ₁ , 12, 21 to 23, the user can select whether to display information in multi-screen display or directory display, and to switch between multi-screen display and directory display. Is also possible.
[0441]
That is, FIG. 38 shows a display example in which the directory display shown in FIG.
[0442]
38A shows the directory display of FIG. 37A, FIG. 38B shows the directory display of FIG. 37B, and FIG. 38C shows the directory display of FIG. Each display is a multi-screen display.
[0443]
Next, for example, as shown in the embodiment of FIG. 16 and FIG. ₁ , 4 ₂ , 4 _Three , 4 _Four And LCD5 ₁ , 5 ₂ , 5 _Three , 5 _Four It is also possible to carry out even when provided.
[0444]
FIG. 39 shows 12 LCDs 4 ₁ Thru 4 _Four , 5 ₁ 5 _Four , 12, 21 to 23 show display examples of directory display.
[0445]
That is, FIG. 39 shows a display example when inputting the text of the e-mail, etc. In the embodiment of FIG. 39A, the character string “Kino” already input is displayed on the LCD 12. Has been. And LCD4 _Four The character “A” representing that line is displayed on the LCD4. _Three The character “ka” representing the ka line is displayed on the LCD 4. ₂ The character “sa” representing the line is displayed on the LCD 4. ₁ , The character “TA” representing the line, the character “NA” representing the line on the LCD 22, the character “HA” representing the line on the LCD 21, and the character representing the line on the LCD 23. “Ma” is the LCD5 ₁ The character “ya” representing the “ya” line is displayed on the LCD 5. ₂ The character “ra” representing the line is displayed on the LCD 5. _Three In the LCD5, the character “WA” representing the line is displayed on the LCD5. _Four The characters “n” are displayed respectively.
[0446]
For example, when the user wants to input a character on the line, the LCD 4 on which the character “a” representing the line is displayed. _Four Touch. In this case, LCD4 _Four Touch panel 4A configured integrally with _Four In FIG. 17, the LCD 4 _Four The touch to the CPU 202 is detected, and the CPU 202 (FIG. 23) determines the LCD 4 based on the detection result. ₁ Thru 4 _Four , 5 ₁ 5 _Four , 12, 21 to 23 are changed as shown in FIG.
[0447]
That is, the display on the LCD 12 remains unchanged as shown in FIG. ₁ Thru 4 _Four , 5 ₁ 5 _Four , 21 to 23 change to a display for inputting the character in the line.
[0448]
Specifically, LCD4 _Four In the LCD4 _Three In the LCD4, “maru” representing a semi-turbid sound is displayed. ₂ In the LCD4, "-" representing a long sound is displayed on the LCD4. ₁ The character “A” in line A, the character “I” in line A on the LCD 22, the character “U” in line A on the LCD 21, and the character “E” in line A on the LCD 23. , LCD5 ₁ The line “O” is displayed on the LCD5. ₂ The “reading marks” for inputting the reading marks are displayed on the LCD 5. _Three In the LCD 5, a “punctuation” for inputting a punctuation is displayed on the LCD 5. _Four In the display, “return” indicating returning to the state of FIG. 39A is displayed.
[0449]
For example, when the user wants to input the character “U”, the user touches the LCD 21 on which the character “U” is displayed. In this case, the touch is detected on the touch panel 21A configured integrally with the LCD 21, and based on the detection result, the CPU 202 (FIG. 23) ₁ Thru 4 _Four , 5 ₁ 5 _Four , 12, 21 to 23 are changed as shown in FIG.
[0450]
In FIG. 39C, only the display on the LCD 12 is changed as compared with the case in FIG. 39B. The character string “Kino” already displayed on the LCD 12 is changed to FIG. A character string “Kino” is displayed by adding the character “U” displayed on the LCD 21 touched in the state of B).
[0451]
However, on the LCD 12, the newly displayed character “U” is displayed in a form different from the already displayed character string “Kino”, and the input is still in an unconfirmed state. . In FIG. 39C, the character string “Kino” whose input has been confirmed is indicated by a solid line, and the character “U” whose input has not been determined is indicated by a dotted line.
[0452]
When confirming the input of the character “U” whose input is not yet determined, the user touches the LCD 12 on which the character string “Kino” is displayed in the state of FIG. This touch is detected by the touch panel 12A configured integrally with the LCD 12A, thereby confirming the input of the character “U”. That is, in this case, on the LCD 12, the character “U” is in the same display state as the character string “Kino” already displayed on the LCD 12.
[0453]
As described above, the LCD 12 can have a function as a confirmation button and can also have a function of instructing kana-kanji conversion. Kana-kanji conversion and input confirmation can be performed.
[0454]
Next, for example, the PDA described with reference to FIGS. ₁ , 5 ₁ , 12, 21 to 23, when the power switch 62 (FIG. 7 etc.) is turned on, the seven LCDs 3, 4 ₁ , 5 ₁ , 12, 21 to 23 are not preferable from the viewpoint of power consumption of the battery 222 (FIG. 23).
[0455]
That is, seven LCDs 3, 4 ₁ , 5 ₁ , 12, 21 to 23, in particular, the five LCDs LCD3, 12, 21 to 23 are not always in an exposed state (a state in which the user can see the display). It is wasteful power consumption to keep it always on.
[0456]
Therefore, in PDA 101 (FIG. 23), the state of lid 20 and the state of sub-panels 115 and 16 are detected by switch unit 225, and based on the detection result, CPU 202 turns on LCDs 3, 12, 21 to 23. / Off control is performed.
[0457]
LCD 4 is always exposed ₁ And 5 ₁ (In the embodiment of FIG. 16 and FIG. ₁ Thru 4 _Four , 5 ₁ 5 _Four ), For example, the user can manually switch on / off thereof, or only when the lid 20 and the sub-panels 15 and 16 are all open. In such a case, it is possible to set it to an off state. LCD4 ₁ And 5 ₁ Can be turned on / off in conjunction with the LCD 3, for example.
[0458]
Next, the on / off control processing of the LCDs 3, 12, 21 to 23 performed by the CPU 202 based on the state of the lid 20 and the states of the sub panels 115 and 16 will be described with reference to the flowchart of FIG.
[0459]
First, in step S111, the CPU 202 determines whether the power switch 62 (FIG. 7) constituting the operation unit 224 (FIG. 23) is in an on state or an off state.
[0460]
If it is determined in step S111 that the power switch 62 is off, the process proceeds to step S112, where the CPU 202 controls the LCD driver 206 (FIG. 23) to turn on one of the LCDs 3, 12, 21 to 23. Are turned off (for example, the backlight constituting the LCD is turned off), and the process returns to step S111.
[0461]
Here, the CPU 202 controls the touch panel driver 205 in conjunction with the turning on / off of the LCD, and turns on / off the corresponding touch panel (the touch panel integrated with the turned on / off LCD) (power supply). Is started / stopped).
[0462]
On the other hand, if it is determined in step S111 that the power switch 62 is in the on state, the process proceeds to step S113, and the CPU 202 controls the LCD driver 206 to turn on the LCD 3 when the LCD 3 is not in the on state. The state is set (for example, the backlight constituting the LCD is turned on), and the process proceeds to step S114.
[0463]
In step S114, the CPU 202 determines whether or not the lid 20 is open based on the detection result of the open / close state of the lid 20 (FIG. 7) by the switch 225. In step S114, when it is determined that the lid 20 is not open, that is, the lid 20 is closed, only the LCD 3 is in an exposed state, and the LCDs 12 and 21 to 23 are in a stowed state (a state in which they are not facing upward) ), The process advances to step S115, and the CPU 202 controls the LCD driver 206 to turn on the LCD 3 when it is off, and to turn on the LCDs 12 and 21 to 23. What is present is turned off, and the process returns to step S111.
[0464]
If it is determined in step S114 that the lid 20 is open, that is, the LCD 3 is in the storage state (the user cannot see it), and the LCD 12 (FIG. 9) is in the exposed state (the upper side is the upper side). If the LCD 3 is in the on state, the CPU 202 controls the LCD driver 206 to turn it off, and the LCD 12 turns off. If it is, turn it on and go to step S117.
[0465]
In step S117, the CPU 202 determines whether the sub panel 15 is open based on the detection result of the open / close state of the sub panel 15 (FIGS. 9 and 10) by the switch unit 225. If it is determined in step S117 that the sub-panel 15 is open, that is, if the sub-panel 15 is open and the LCD 22 (FIG. 10) provided therein is in an exposed state, the process proceeds to step S118, and the CPU 202 By controlling the LCD driver 206, when the LCD 22 is off, the LCD 22 is turned on, and the process proceeds to step S120.
[0466]
If it is determined in step S117 that the sub-panel 15 is closed, that is, the side panel 15 is closed, the LCD 22 provided therein is in the storage state, and the side panel in the closed state is further closed. 15, when a part (upper half) of the LCD 21 (FIG. 10) provided on the main panel 14 is hidden, the process proceeds to step S <b> 119, and the CPU 202 controls the LCD driver 206. The LCDs 21 and 22 that are in the on state are turned off, and the process proceeds to step S120.
[0467]
In step S120, the CPU 202 determines whether or not the sub panel 16 is open based on the detection result of the open / close state of the sub panel 16 (FIGS. 9 and 10) by the switch unit 225. If it is determined in step S120 that the sub-panel 16 is closed, that is, the side panel 16 is closed, and the LCD 23 provided therein is in the stowed state. If a part (lower half) of the LCD 21 (FIG. 10) provided on the main panel 14 is hidden, the process proceeds to step S121, and the CPU 202 controls the LCD driver 206 to Of 23, the on state is turned off, and the process returns to step S111.
[0468]
If it is determined in step S120 that the sub-panel 16 is open, that is, if the sub-panel 16 is open and the LCD 23 (FIG. 10) provided in the sub-panel 16 is exposed, the process proceeds to step S122, and the CPU 202 Controls the LCD driver 206 to turn it on when the LCD 23 is turned off, and proceeds to step S123.
[0469]
In step S123, the CPU 202 determines whether both the sub panels 15 and 16 are open based on the detection result of the open / close state of the sub panels 15 and 16 (FIGS. 9 and 10) by the switch unit 225. If it is determined in step S123 that both or one of the sub-panels 15 and 16 is closed, that is, at least one of the side panels 15 and 16 is closed, and the side panel 15 in the closed state or When at least a part (upper half or lower half) of the LCD 21 (FIG. 10) provided on the main panel 14 is hidden by the process 16, the process proceeds to step S124, and the CPU 202 controls the LCD driver 206. Thus, when the LCD 21 is on, the LCD 21 is turned off and the process returns to step S111.
[0470]
If it is determined in step S123 that both the subpanels 15 and 16 are open, that is, the subpanels 15 and 16 are open, and the LCDs 22 and 23 (FIG. 10) provided in the subpanels 15 and 16 respectively. When each of the LCDs 21 is in the exposed state and the LCD 21 provided on the main panel 14 is also in the exposed state, the process proceeds to step S125, and the CPU 202 controls the LCD driver 206 to turn off the LCDs 21 to 23. Is turned on, and the process returns to step S111.
[0471]
Next, in the PDA 101, as described with reference to FIG. ₁ , 5 ₁ , 12, 21 to 23, one information is displayed for each of the six LCDs 4. ₁ , 5 ₁ , 12, 21 to 23, when any one of the displayed information is selected, a plurality of pieces of information linked to the selected information are displayed on the six LCDs 4 respectively. ₁ , 5 ₁ , 12, 21 to 23 can be displayed repeatedly.
[0472]
Accordingly, in the PDA 101, when a series of information is structured in a hierarchical structure, a plurality of information in the upper hierarchy is displayed, and when any of the information is selected by the user, the information is linked to the information. It is possible to sequentially repeat the display of the lower layer information that is associated (associated), and in this case, the user can reach the desired information relatively easily (search for the desired information). Can do.
[0473]
Therefore, a hierarchical display control process for controlling the display of information structured in such a hierarchical structure will be described with reference to the flowchart of FIG.
[0474]
For example, when a user requests display of information by operating the operation unit 224 or the like (FIG. 23), a request for displaying the information (hereinafter, referred to as a display request as appropriate) is sent to the input I / F 223 and the bus 201. Via the CPU 202. Upon receiving the display request, the CPU 202 accesses the HDD 215 via the bus 201, and searches for information requested by the display request in step S131.
[0475]
In this example, information is searched from the HDD 215. However, the information may be stored on an external device (for example, the base base computer 102, another PDA 103, or the Internet 105 via the communication I / F 218). It is also possible to perform a search from a storage medium (recording medium) of an external device by communicating with a server (not shown), communication terminal 106, etc.).
[0476]
When the CPU 202 retrieves information requested by the display request from the HDD 215, the CPU 202 proceeds to step S132 and determines whether the information has a hierarchical structure. If it is determined in step S132 that the information retrieved from the HDD 215 does not have a hierarchical structure, steps S133 to S136 are skipped and the process proceeds to step S137. In step S137, the CPU 202 supplies the information retrieved from the HDD 215 to the LCD driver 206, and the LCD 4 ₁ , 5 ₁ , 12, 21 to 23, the information retrieved from the HDD 215 is displayed, for example, on the multi-screen described with reference to FIG. 36, and the process ends.
[0477]
On the other hand, if it is determined in step S132 that the information retrieved from the HDD 215 has a hierarchical structure, the process proceeds to step S133, and the CPU 202 determines a plurality of pieces of information in the highest hierarchy among the information retrieved from the HDD 215. Is supplied to the LCD driver 206, whereby the LCD 4 ₁ , 5 ₁ , 12, 21 to 23, a plurality of information of the highest hierarchy is displayed as a directory as described with reference to FIG. 37, and the process proceeds to step S134.
[0478]
In step S134, the CPU 202 determines that the user has the LCD 4 ₁ , 5 ₁ , 12, 21 to 23 to touch the LCD 4 ₁ , 5 ₁ , 12, 21 to 23, it is determined whether any information displayed is selected.
[0479]
LCD4 ₁ , 5 ₁ , 12, 21 to 23 is determined by the CPU 202 monitoring the output of the touch panel driver 205.
[0480]
In step S134, the LCD 4 ₁ , 5 ₁ , 12, 21 to 23, when it is determined that any information is not selected, the process returns to step S134.
[0481]
In step S134, the LCD 4 ₁ , 5 ₁ , 12, 21 to 23, the CPU 202 proceeds to step S135, and the CPU 202 determines whether there is a lower hierarchy linked to the hierarchy of the selected information. Determine.
[0482]
If it is determined in step S135 that there is a lower hierarchy linked to the selected information hierarchy, the process advances to step S136, and the CPU 202 supplies a plurality of pieces of information in the lower hierarchy to the LCD driver 206. LCD4 ₁ , 5 ₁ , 12, 21 to 23, a plurality of information in the lower hierarchy is displayed in a directory, the process returns to step S134, and the same processing is repeated thereafter.
[0483]
Here, by repeating the processes of steps S134 to S136, the LCD 4 ₁ , 5 ₁ , 12, 21 to 23, information of lower layers is sequentially displayed.
[0484]
On the other hand, if it is determined in step S135 that there is no lower hierarchy linked to the hierarchy of the information selected by the user, the process proceeds to step S137, and the CPU 202 controls the LCD driver 206 to control the LCD 4. ₁ , 5 ₁ , 12, 21 to 23, the contents of the information selected by the user are displayed on the multi-screen, and the process is terminated.
[0485]
Next, FIG. 42 shows a format of information structured in a hierarchical structure.
[0486]
That is, FIG. 42 (A) shows the information of the highest hierarchy, FIG. 42 (B) shows the information of the next lower hierarchy (second hierarchy), and FIG. The information of the next lower hierarchy (third hierarchy) is shown.
[0487]
Information of each hierarchy is composed of hierarchy information, link information, and display information.
[0488]
The hierarchy information represents the number of information layers. In the embodiment of FIG. 42, the hierarchical information is such that the hierarchical information of the highest hierarchy is 0, and the value increases by 1 as the hierarchy becomes lower.
[0489]
The link information represents link destination information linked to the information.
[0490]
Display information is LCD4 ₁ , 5 ₁ , 12, 21 to 23 are image data to be displayed.
[0491]
Therefore, for example, when the image data of the menu screen is arranged in the display information of the top-level information Info # 0 in FIG. 42A, and the user requests the display of the menu screen, For example, the menu screen as shown in FIG. ₁ , 5 ₁ , 12, 21 to 23.
[0492]
Further, when the user selects by touching the LCD 22 on which the item “movie” is displayed in FIG. 37A, the item “movie” is displayed in the link information of the information Info # 0 in the highest hierarchy in FIG. The display information arranged in the information Info # 1 of the second hierarchy in FIG. ₁ , 5 ₁ , 12, 21 to 23. That is, in the information “Info # 1” linked to the item “movie”, title information of the movie (for example, title image data) is arranged as display information. ₁ , 5 ₁ , 12, 21 to 23. As a result, as shown in FIG. 37B, movie titles # 1 to # 6 are displayed in a directory.
[0493]
Then, in the state of FIG. 37 (B), the user displays, for example, the LCD 4 on which the movie title # 1 is displayed. ₁ When the selection is made by touching, the information of the third hierarchy in FIG. 42C linked to the movie title # 1 in the link information of the information Info # 1 of the second hierarchy of FIG. The display information arranged in Info # 2 is the LCD 4 ₁ , 5 ₁ , 12, 21 to 23. That is, in the information Info # 2 linked to the title # 1 of the movie, movie introduction information (for example, image data of a still image such as a director or a leading actor) is arranged as display information. LCD4 ₁ , 5 ₁ , 12, 21 to 23. As a result, as shown in FIG. 37C, images such as movie directors and leading actors are displayed in a directory.
[0494]
In the information information # 2 of the third hierarchy, the link information is linked with video clips and other data such as directors and leading actors displayed in the image as the introduction information. When the user touches the LCD on which the movie director is displayed, the LCD 4 ₁ , 5 ₁ , 12, 21 to 23, images obtained by reproducing the video clip of the director are displayed on a multi-screen.
[0495]
As described above, the information can be easily searched by configuring the information in a hierarchical structure.
[0496]
Note that image data, text data, and audio data can be employed as information configured in a hierarchical structure. When audio data is adopted as information having a hierarchical structure, the audio data itself is displayed on the LCD 4. ₁ , 5 ₁ , 12, 21 to 23, the LCD 4 ₁ , 5 ₁ , 12, 21 to 23, the text corresponding to the audio data (for example, if the audio data is music data, the music title, etc.) is displayed. However, for audio, LCD4 ₁ , 5 ₁ , 12, 21 to 23 are each provided with a speaker, and sound is output from each speaker, so that sound can be output by a method corresponding to directory display.
[0497]
Next, FIG. 43 shows how the PDA 101 communicates with another PDA 103. As shown in the figure, the PDA 101 can communicate with one PDA 103 or with a plurality of PDAs obtained by adding other PDAs to the PDA 103. The same applies to the other PDAs 103.
[0498]
Thus, the PDAs 101 and 103 can obtain data of higher quality based on the data received by communicating with one or more other PDAs.
[0499]
That is, FIG. 44 shows a functional configuration example of the PDAs 101 and 103 when data is transmitted and received between the PDAs 101 and 103. Here, the functional configuration of the PDA 101 shown in FIG. 44 is realized by the CPU 202 (FIG. 23) executing a program, for example. The functional configuration of the PDA 103 is the same.
[0500]
In the embodiment of FIG. 44, data is transmitted from the PDA 101 to the PD 103. Therefore, the PDA 101 functions as a transmitting device that transmits data, and the PDA 103 functions as a receiving device that receives the data.
[0501]
The PDA 101 as a transmission device includes a transmission processing unit 401. The transmission processing unit 401 performs, for example, predetermined transmission processing on image data as data to be transmitted to the PDA 103 as a reception device, and serves as a reception device. To the PDA 103.
[0502]
The PDA 103 as a receiving device receives data transmitted from the PDA 101 as a transmitting device, performs a predetermined receiving process, obtains and outputs image data.
[0503]
Here, data is assumed to be transmitted from the PDA 101 to the PDA 103, but data can also be transmitted from the PDA 103 to the 101. In this case, the PDA 103 functions as a transmission device that transmits data. The PDA 101 functions as a receiving device that receives the data.
[0504]
Accordingly, since the PDAs 101 and 103 may function as a transmission device or may function as a reception device, both have both a transmission processing unit 401 and a reception processing unit 402. In FIG. For the PDA 101, the reception processing unit 402 is not shown, and for the PDA 103, the transmission processing unit 401 is not shown. Therefore, in the following description, unless otherwise specified, the PDA 101 is a transmission device and the PDA 103 is a reception device.
[0505]
FIG. 45 illustrates a first configuration example of the transmission processing unit 401 in FIG.
[0506]
In the embodiment of FIG. 45, the transmission processing unit 401 includes an encoding unit 411 and an event detection unit 412. The encoding unit 411 is supplied with image data to be transmitted to the PDA 103, and the event detection unit 412 is supplied with a request signal to be described later.
[0507]
Here, the image data supplied to the encoding unit 411 is, for example, one captured by the CCD camera 65 of the PDA 101 (FIG. 23) or stored in the HDD 215. Furthermore, it is assumed here that the image data supplied to the encoding unit 411 is a high-resolution image with high image quality (hereinafter, referred to as an HD (High Definition) image as appropriate).
[0508]
Further, the request signal supplied to the event detection unit 412 is transmitted from the PDA 103 as a receiving device, is received by the communication I / F 218 (FIG. 23), and is supplied to the event detection unit 412. It has become.
[0509]
The encoding unit 411 performs encoding or the like on the HD image data supplied thereto, and obtains and outputs transmission data to be transmitted to the PDA 103 as a receiving apparatus. This transmission data is transmitted to the PDA 103 as a receiving apparatus via the communication I / F 218 (FIG. 23).
[0510]
When the event detection unit 412 receives the request signal, the event detection unit 412 detects reception of the request signal as an event, and supplies a message indicating that the event has occurred (hereinafter, appropriately referred to as an event message) to the encoding unit 411.
[0511]
Here, the encoding unit 411 includes a data compression unit 421, a transmission control unit 422, a class tap extraction unit 423, a class classification unit 424, and a class code database 425, as shown in FIG.
[0512]
The HD image data supplied to the encoding unit 411 is supplied to the data compression unit 421 and the class tap extraction unit 423, and the data compression unit 421 converts the HD image data supplied thereto into, for example, Then, the data is compressed by thinning out the number of pixels in the spatial direction, and converted into standard resolution or low resolution image data (hereinafter referred to as SD (Standard Definition) image as appropriate). The SD image data is supplied from the data compression unit 421 to the transmission control unit 422.
[0513]
Here, if communication between the PDAs 101 and 103 is performed wirelessly, it may not be possible to secure a very wide band as the transmission band. For this reason, the data compression unit 421 converts HD image data with a large amount of data into an SD image with a small amount of data (data compression) so that high-speed data transmission can be performed even in a narrow band. It has become.
[0514]
In addition to the SD image data supplied from the data compression unit 421, the transmission control unit 422 is supplied with a class code from the class code database 425. Further, the transmission control unit 422 is also supplied with an event message from the event detection unit 412. The transmission control unit 422 basically selects and outputs the SD image data from the data compression unit 421 as transmission data, but when receiving an event message from the event detection unit 412, the transmission control unit 422 receives the event message from the class code database 425. Select and output the class code as transmission data.
[0515]
The class tap extraction unit 423 sequentially extracts each pixel of the HD image data supplied thereto (hereinafter referred to as “HD pixel” as appropriate) as a target pixel, and extracts HD pixels used for classifying the target pixel. , Output as a class tap.
[0516]
That is, the class tap extraction unit 423 uses several HD pixels (for example, spatially or temporally close to the target pixel) as HD pixels used for clustering (classifying) the target pixel into any one of a plurality of the target pixels. HD pixels with the number of pixels in the horizontal direction × vertical direction 3 × 3 centered on the target pixel are extracted from the HD image data supplied thereto and output as class taps. The class tap output from the class tap extraction unit 423 is supplied to the class classification unit 424.
[0517]
The class classification unit 424 classifies the target pixel based on the class tap for the target pixel supplied from the class tap extraction unit 423, and obtains a class code representing the class to which the target pixel belongs, obtained as a result of the class classification. Output.
[0518]
Here, as a method of classifying, for example, ADRC (Adaptive Dynamic Range Coding) or the like can be employed.
[0519]
In the method using ADRC, pixel values of pixels constituting the class tap are subjected to K-bit ADRC processing, and the class of the pixel of interest is determined according to the ADRC code obtained as a result.
[0520]
In the K-bit ADRC, for example, the maximum value MAX and the minimum value MIN of the pixel values constituting the class tap are detected, and DR = MAX-MIN is set as the local dynamic range of the set, and the dynamic range DR Based on this, the pixels making up the class tap are requantized to K bits. That is, the minimum value MIN is subtracted from the pixel value of each pixel constituting the class tap, and the subtracted value is DR / 2. ^K Divide by (quantize). A bit string obtained by arranging the pixel values of the K-bit pixels constituting the class tap in a predetermined order is output as an ADRC code. Therefore, when a class tap is subjected to, for example, 1-bit ADRC processing, the pixel value of each pixel constituting the class tap is an average of the maximum value MAX and the minimum value MIN after the minimum value MIN is subtracted. By dividing by the value, the pixel value of each pixel is made 1 bit (binarized). Then, a bit string in which the 1-bit pixel values are arranged in a predetermined order is output as an ADRC code.
[0521]
In class classification, for example, a level distribution pattern (pixel value distribution) of pixels constituting a class tap can be output as a class code as it is, but in this case, the class tap has N pixels. And K bits are assigned to each pixel, the number of class codes obtained by class classification is (2 ^N ) ^K As a result, the number is exponentially proportional to the number K of pixels.
[0522]
Therefore, it is preferable to use a method for compressing the class tap information amount by the above-described ADRC processing or vector quantization for class classification.
[0523]
The class code database 425 stores the class code output from the class classification unit 424.
[0524]
Here, in the class classification unit 424, class classification is performed using each HD pixel of the HD image data as a pixel of interest, so a class code is obtained for each HD pixel. Therefore, it can be considered that the class code database 425 stores image data whose pixel value is the class code of each HD pixel of the HD image data. It is called a class code image.
[0525]
In the transmission processing unit 401 configured as described above, image data transmission processing for transmitting SD image data obtained from HD image data, class code generation processing for generating a class code image from the HD image data, and class code A class code transmission process for transmitting the class code image stored in the database 425 is performed.
[0526]
46, the image data transmission process, class code generation process, and class code transmission process performed by the transmission processing unit 401 in FIG. 45 will be described.
[0527]
First, the image data transmission processing performed by the transmission processing unit 401 in FIG. 45 will be described with reference to the flowchart in FIG.
[0528]
The image data transmission process is started, for example, when the operation unit 224 (FIG. 23) is operated by the user so as to request transmission of image data.
[0529]
That is, the HD image data to be transmitted is supplied to the encoding unit 411, for example, in units of frames. In the image data transmission process, first, in step S201, the encoding unit 411 The data compression unit 421 compresses the HD image data by converting it into SD image data, and supplies the SD image data to the transmission control unit 422.
[0530]
Note that one frame here can be considered as one screen of a moving image or one screen of a still image.
[0531]
In step S202, the transmission control unit 422 selects the SD image data from the data compression unit 421 and outputs it as transmission data. This transmission data is supplied to the communication I / F 218 (FIG. 23) and transmitted from the antenna 64, for example.
[0532]
In step S202, when there is a margin between the transmission of the SD image data and the transmission of the next SD image data, the SD image data of the same frame is repeatedly transmitted. Accordingly, in step S202, SD image data of the same frame is transmitted one or more times.
[0533]
Thereafter, the process proceeds to step S203, in which the data compression unit 421 determines whether there is HD image data of the next frame (the frame to be processed next). If it is determined that there is, the process returns to step S201, and Similar processing is repeated.
[0534]
If it is determined in step S203 that there is no HD image data of the next frame, the image data transmission process ends.
[0535]
In the above-described case, in step S202, SD image data of the same frame may be continuously transmitted a plurality of times. However, transmission (retransmission) of the same SD image data can be performed later. It is. That is, the retransmission of the SD image data can be performed by transmitting the series of SD image data again after the transmission of the series of SD image data is completed.
[0536]
Next, class code generation processing performed by the transmission processing unit 401 in FIG. 45 will be described with reference to the flowchart in FIG.
[0537]
The class code generation process is started, for example, when the image data transmission process shown in FIG.
[0538]
That is, the same HD image data supplied to the data compression unit 421 is supplied to the class tap extraction unit 423, for example, in units of one frame. First, in step S211, class taps are extracted for each pixel of interest using each HD pixel constituting the HD image data of one frame supplied thereto as the pixel of interest. The class tap is supplied to the class classification unit 424.
[0539]
When the class classification unit 424 receives from the class tap extraction unit 423 a class tap having each HD pixel of one frame as the pixel of interest, in step S212, the class classification unit 424 converts the class code of each HD pixel into the class tap for that HD pixel. Based on the classification, it is obtained and supplied to the class code database 425.
[0540]
In step S213, the class code database 425 stores an image composed of the class code for each HD pixel constituting one frame of HD image data supplied from the class classification unit 424, that is, a class code image, and in step S214. move on.
[0541]
In step S214, the class tap extraction unit 423 determines whether there is HD image data of the next frame. If it is determined that there is, the process returns to step S211, and the same processing is repeated thereafter.
[0542]
If it is determined in step S214 that there is no HD image data of the next frame, the class code generation process ends.
[0543]
Next, the class code transmission processing performed by the transmission processing unit 401 in FIG. 45 will be described with reference to the flowchart in FIG.
[0544]
In the class code transmission process, first, in step S221, the transmission control unit 422 determines whether a predetermined event has occurred.
[0545]
That is, here, for example, the fact that a request signal has been transmitted from the PDA 103 as a receiving device is adopted as a predetermined event, and in step S221, the event detection unit 412 receives the request signal, whereby the PDA 103 It is determined whether or not a predetermined event has occurred depending on whether or not an event message indicating that an event that a request signal has been transmitted from has been supplied to the transmission control unit 422.
[0546]
If it is determined in step S221 that a predetermined event has not occurred, the process returns to step S221 and waits until a predetermined event occurs.
[0547]
If it is determined in step S221 that a predetermined event has occurred, the process proceeds to step S222, and the transmission control unit 422 determines whether or not the class code image is stored in the class code database 425.
[0548]
If it is determined in step S222 that the class code image is not stored, the process returns to step S221, and the same processing is repeated thereafter.
[0549]
If it is determined in step S222 that a class code image is stored, the process proceeds to step S223, and the transmission control unit 422 reads out the class code stored in the class code database 425 and selects it as transmission data. The transmission data is supplied from the transmission control unit 422 to the communication I / F 218 (FIG. 23) and transmitted from the antenna 64.
[0550]
In step S223, the transmission control unit 422 terminates transmission of all class code images stored in the class code database 425 from the class code database 425 at the start of the process. Is deleted, and the process returns to step S222. Then, as described above, in step S222, the transmission control unit 422 determines whether or not a class code image is stored in the class code database 425.
[0551]
That is, a new class code image obtained in the class classification unit 424 is stored in the class code database 425 between the start of the process of the previous step S223 and the start of the process of the current step S222. Therefore, the transmission control unit 422 determines whether or not such a new class code image is stored in the class code database 425. Then, based on the determination result, the same processing is repeated hereinafter.
[0552]
Therefore, in the class code transmission process, when a predetermined event occurs, the class code image is transmitted until there are no more class code images stored in the class code database 425.
[0553]
Next, FIG. 47 illustrates a first configuration example of the reception processing unit 402 of FIG.
[0554]
In the embodiment of FIG. 47, the reception processing unit 402 includes a decoding unit 431, an image quality determination unit 432, and a request signal transmission unit 433.
[0555]
The decoding unit 431 decodes received data supplied thereto to obtain SD image data or HD image data.
[0556]
That is, in the PDA 103, transmission data transmitted from the transmission processing unit 401 of the PDA 101 as described with reference to FIGS. 45 and 46 is received by the communication I / F 218 via the antenna 64 (FIG. 23). The reception data obtained as a result is supplied to the reception processing unit 402.
[0557]
In the reception processing unit 402, reception data from the communication I / F 218 is supplied to the decoding unit 431, and the decoding unit 431 obtains SD image data or HD image data by processing the reception data.
[0558]
The image quality determination unit 432 determines the image quality of the SD image data obtained by the decoding unit 431 and supplies the determination result to the request signal transmission unit 433.
[0559]
The request signal transmission unit 433 generates and outputs a request signal for requesting a class code image based on the determination result of the image quality of the SD image data by the image quality determination unit 432. This request signal is supplied to the communication I / F 218 (FIG. 23), and is transmitted to the PDA 101 as a transmission device via the antenna 64.
[0560]
Here, as shown in FIG. 47, the decoding unit 431 includes a reception control unit 441, a reception buffer 442, a registration unit 443, a storage 444, a selection unit 445, a class code database 446, and an adaptive processing unit 447. .
[0561]
The reception control unit 441 receives the received data, and supplies the received data to the reception buffer 442 if the received data is SD image data, and supplies the received data to the class code database 446 if the received data is a class code image. To do.
[0562]
The reception buffer 442 temporarily stores SD image data supplied from the reception control unit 441. The registration unit 443 performs control to store (register) each frame of the SD image data stored in the reception buffer 442 in the storage 444. The storage 444 stores SD image data under the control of the registration unit 443. This SD image data is read from the storage 444 and supplied to the selection unit 445.
[0563]
The selection unit 445 selects the SD image data supplied from the storage 444, supplies it to the LCD driver 206 (FIG. 23), and displays it on the LCD 3 or the like. However, when the HD image data is output from the adaptive processing unit 447 as described later, the selection unit 445 selects the HD image data and supplies the HD image data to the LCD driver 206 (FIG. 23). Etc.
[0564]
The class code database 446 stores the class code image supplied from the reception control unit 441.
[0565]
The adaptation processing unit 447 performs adaptation processing on the SD image data stored in the storage 444 using each class code constituting the class code image stored in the class code database 446, and thereby, the SD image data HD image data with improved image quality is obtained and supplied to the selection unit 445.
[0566]
Here, in the adaptive processing, the pixels of the HD image in which the spatial resolution of the SD image is improved by linear combination of the pixels constituting the SD image (hereinafter, appropriately referred to as SD pixels) and a predetermined tap coefficient. By obtaining the predicted value, an image in which the resolution of the SD image is increased can be obtained.
[0567]
Specifically, for example, a certain HD image is used as teacher data, and an SD image with degraded resolution of the HD image is used as student data, and pixels constituting the HD image (hereinafter, referred to as HD pixels as appropriate) The predicted value E [y] of the pixel value y of the pixel value x is the pixel value x of several SD pixels (pixels constituting an SD image). ₁ , X ₂ , ... and a predetermined tap coefficient w ₁ , W ₂ Consider a linear primary combination model defined by the linear combination of. In this case, the predicted value E [y] can be expressed by the following equation.
[0568]
E [y] = w ₁ x ₁ + W ₂ x ₂ + ...
... (1)
[0569]
To generalize equation (1), tap coefficient w _j A matrix W consisting of _ij And a predicted value E [y _j ] A matrix Y ′ consisting of a set of
[Expression 1]

Then, the following observation equation holds.
[0570]
XW = Y ′ (2)
Here, the component x of the matrix X _ij Is a set of i-th student data (i-th teacher data y _i The j-th student data in the set of student data used for the prediction of _j Represents a tap coefficient by which a product with the jth student data in the student data set is calculated. Y _i Represents the i-th teacher data, and thus E [y _i ] Represents the predicted value of the i-th teacher data. Note that y on the left side of Equation (1) is the component y of the matrix Y. _i The suffix i is omitted, and x on the right side of Equation (1) ₁ , X ₂ ,... Are also components x of the matrix X _ij The suffix i is omitted.
[0571]
Consider that the least square method is applied to the observation equation of Equation (2) to obtain a predicted value E [y] close to the pixel value y of the HD pixel. In this case, a matrix Y composed of a set of true pixel values y of HD pixels serving as teacher data and a matrix E composed of a set of residuals e of predicted values E [y] with respect to the pixel values y of HD pixels,
[Expression 2]

From the equation (2), the following residual equation is established.
[0572]
XW = Y + E (3)
[0573]
In this case, the tap coefficient w for obtaining the predicted value E [y] close to the pixel value y of the HD pixel _j Is the square error
[Equation 3]

Can be obtained by minimizing.
[0574]
Therefore, the above square error is converted to the tap coefficient w. _j When the value differentiated by 0 is 0, that is, the tap coefficient w satisfying the following equation: _j However, this is the optimum value for obtaining the predicted value E [y] close to the pixel value y of the HD pixel.
[0575]
[Expression 4]

[0576]
Therefore, first, the equation (3) is changed to the tap coefficient w _j Is differentiated by the following equation.
[0577]
[Equation 5]

[0578]
From equations (4) and (5), equation (6) is obtained.
[0579]
[Formula 6]

[0580]
Furthermore, the student data x in the residual equation of equation (3) _ij , Tap coefficient w _j , Teacher data y _i And residual e _i Considering this relationship, the following normal equation can be obtained from the equation (6).
[0581]
[Expression 7]

[0582]
In addition, the normal equation shown in Expression (7) has a matrix (covariance matrix) A and a vector v,
[Equation 8]

And the vector W is defined as shown in Equation 1,
AW = v (8)
It can be expressed as
[0583]
Each normal equation in equation (7) is the student data x _ij And teacher data y _i By preparing a certain number of sets, a tap coefficient w to be obtained _j Therefore, by solving equation (8) with respect to vector W (however, to solve equation (8), matrix A in equation (8) is regular). Necessary), the optimal tap coefficient w _j Can be requested. In solving the equation (8), for example, a sweeping method (Gauss-Jordan elimination method) or the like can be used.
[0584]
As described above, using the student data and the teacher data, the tap coefficient w that minimizes a statistical error (here, square error, for example) is obtained from the student data and the tap coefficient. _j And learning the tap coefficient w _j The adaptive process is to obtain the predicted value E [y] close to the teacher data y by using the equation (1).
[0585]
The adaptive processing is not included in the SD image, but is different from simple interpolation in that a component included in the HD image is reproduced. That is, in the adaptive processing, as long as only the expression (1) is seen, it looks the same as simple interpolation using a so-called interpolation filter, but the tap coefficient w corresponding to the tap coefficient of the interpolation filter uses the teacher data y. In other words, since it is obtained by learning, the components included in the HD image can be reproduced. From this, it can be said that the adaptive process is a process having an image creation (resolution creation) effect.
[0586]
Although the adaptive processing based on linear primary prediction (Equation (1)) has been described here, it is possible to use second-order or higher-order prediction calculations in the adaptive processing.
[0587]
FIG. 48 shows a configuration example of the adaptive processing unit 447 of FIG. 47 that performs the adaptive processing as described above.
[0588]
For example, the SD image data stored in the storage 444 is supplied to the buffer 451 in units of one frame, and the buffer 451 temporarily stores the SD image data.
[0589]
The prediction tap extraction unit 452 sets the oldest (temporally oldest frame) frame that has not yet been processed among the frames of the SD image data stored in the buffer 451 as the target frame, and further, the SD of the target frame. The HD pixels constituting the HD image data with improved image quality are sequentially used as the target pixel, and the SD pixel used for predicting the pixel value of the target pixel is determined from the SD image data stored in the buffer 451. Extracted and output as a prediction tap used in the product-sum operation unit 453 in the subsequent stage.
[0590]
That is, the prediction tap extraction unit 452 has several SD pixels (for example, the number of pixels in the horizontal direction × vertical direction of 5) that are spatially or temporally close to the position of the SD image data corresponding to the position of the target pixel. X5 SD pixels) are extracted and output as prediction taps.
[0591]
The product-sum operation unit 453 uses the prediction tap for the pixel of interest output from the prediction tap extraction unit 452 and the tap coefficient of the class of the pixel of interest supplied from the coefficient memory 455, which will be described later. A primary prediction calculation is performed, thereby obtaining and outputting the pixel value of the pixel of interest (predicted value of the pixel value of the HD pixel).
[0592]
The class code reading unit 454 reads a class code image stored from the class code database 446 (FIG. 47), which is generated from the HD image data of the frame of interest, as the class code image of interest. Further, the class code reading unit 454 gives the coefficient code corresponding to the target pixel among the class codes of the target class code image to the coefficient memory 455 as an address.
[0593]
Here, frame identification information for identifying a frame of an HD image corresponding to the class code image is added to the class code image generated by the transmission processing unit 401 in FIG. The unit 454 recognizes the class code image corresponding to the frame of interest (class code image generated from the HD image data of the frame of interest) based on the frame identification information.
[0594]
The coefficient memory 455 uses the learning HD image data as teacher data, and the learning SD image data as student data, and a class in which HD pixels of the HD image data can be classified (hereinafter referred to as HD class as appropriate). For each, a tap coefficient for each HD class obtained by building and solving the normal equation shown in Expression (8) is stored. That is, the coefficient memory 455 stores the tap coefficient of the HD class at the address corresponding to each HD class. The coefficient memory 455 reads the tap coefficient of the class from the address corresponding to the class of the pixel of interest supplied from the class code reading unit 454 and supplies the tap coefficient to the product-sum operation unit 453.
[0595]
A method for performing learning for obtaining the tap coefficient for each class will be described later.
[0596]
Next, in the reception processing unit 402 of FIG. 47, image data reception processing for processing image data transmitted from the PDA 101, request signal transmission processing for transmitting a request signal to the PDA 101, and adaptive processing for improving the image quality of the image data Is to be done.
[0597]
Therefore, the image data reception process, request signal transmission process, and adaptation process performed by the reception processing unit 402 in FIG. 47 will be described with reference to the flowchart in FIG.
[0598]
First, image data reception processing will be described with reference to the flowchart in FIG.
[0599]
Transmission data transmitted from the PDA 101 is received by the antenna 64 (FIG. 23) and supplied to the reception control unit 441 as reception data via the communication I / F 218.
[0600]
When the received data is a class code image, the reception control unit 441 supplies the class code image to the class code database 446 and stores it.
[0601]
In addition, when the reception data is SD image data, the reception control unit 441 supplies the SD image data to the reception buffer 442 for storage.
[0602]
When the storage of the SD image data in the reception buffer 442 is started, the image data reception process is started. First, in step S231, the registration unit 443 selects the most of the SD image data stored in the reception buffer 442. The old (most past in time) frame is used as a noticed frame, the SD image data of the noticed frame is read, and the read SD image data is deleted from the reception buffer 442. In step S231, the registration unit 443 determines whether the SD image data of the same frame as the SD image data of the frame of interest read from the reception buffer 442 is already stored in the storage 444.
[0603]
Here, in the transmission processing unit 401 of FIG. 45, identification information for identifying the frame is added to each frame of the SD image data output as the transmission data in the transmission control unit 422. The unit 443 determines whether or not SD image data of the same frame as the SD image data read from the reception buffer 442 is stored in the storage 444 by referring to the frame identification information.
[0604]
As described above with reference to the flowchart of FIG. 46A, the reception processing unit 402 can include a plurality of SD image data of the same frame. This is because SD image data may be transmitted.
[0605]
If it is determined in step S231 that the SD image data of the same frame as the target frame is not stored in the storage 444, the process proceeds to step S232, and the registration unit 443 writes the SD image data of the target frame in the storage 444. The process proceeds to step S236.
[0606]
On the other hand, when it is determined in step S231 that SD image data of the same frame as the target frame is stored in the storage 444, the process proceeds to step S233, and the registration unit 443 stores the SD image data stored in the storage 444. Among them, the SD image data of the same frame as the target frame is read, and the process proceeds to step S234.
[0607]
In step S234, the registration unit 443 weights and adds the SD image data of the frame of interest and the SD image data of the same frame as the frame of interest read in step S234, and uses the weighted addition value as a pixel value. SD image data of the frame is generated.
[0608]
That is, the registration unit 443 aligns the SD image data of the frame of interest and the SD image data of the same frame as the frame of interest read from the storage 444, and then weights and adds the pixel values at the same position, A new pixel value at that position is obtained.
[0609]
Here, in the weighted addition in step S234, the weight used for the weighted addition can be determined based on, for example, the weighted addition count of the SD image data read from the storage 444.
[0610]
That is, when SD image data of the same frame as the target frame is stored in the storage 444, the registration unit 443 weights the SD image data of the target frame and the SD image data stored in the storage 444. The added SD image data having the weighted added value as the pixel value is newly stored in the storage 444 as the SD image data of the frame of interest.
[0611]
Therefore, when the SD image data of the same frame as the target frame stored in the storage 444 is a weighted addition value of N frames of SD image data, in step S234, the weight of the SD image data of the target frame is set. 1 and the weight of the SD image data of the same frame as the frame of interest stored in the storage 444 is N, and the weighted addition of the two SD image data can be performed.
[0612]
Note that the weighting method is not limited to the method described above.
[0613]
When the registration unit 443 obtains new SD image data as a weighted addition value of the SD image data read from the reception buffer 442 and the SD image data read from the storage 444 for the frame of interest, the process proceeds to step S235. The new SD image data of the frame of interest is supplied to the storage 444 and stored in the form of overwriting the SD image data of the same frame as the frame of interest stored therein.
[0614]
In step S236, the registration unit 443 determines whether the SD image data is still stored in the reception buffer 442. If it is determined that the SD image data is stored, the registration unit 443 returns to step S231 and returns to the next frame ( The next processing is repeated with the next frame to be processed as a new frame of interest.
[0615]
On the other hand, if it is determined in step S236 that the SD image data is not stored in the reception buffer 442, the process ends.
[0616]
As described above, in the image data reception process, SD image data of the same frame is weighted and added, so that it is possible to obtain SD image data with improved image quality.
[0617]
That is, when SD image data is transmitted to the PDA 101 to 103, noise may be superimposed on the SD image data or some data may be lost during the transmission. Such superimposition of noise, lack of data, and the like greatly deteriorate the image quality of SD image data. Therefore, as described above, the registration unit 443 improves the image quality degradation by performing weighted addition between the SD image data of the same frame (the image quality is improved if an image with degraded image quality is used as a reference). )
[0618]
Next, request signal transmission processing will be described with reference to the flowchart in FIG.
[0619]
For example, the request signal transmission process is started at an arbitrary timing. First, in step S241, the image quality determination unit 432 reads the SD image data of each frame stored in the storage 444, and the SD image of each frame. Calculate the autocorrelation of the data.
[0620]
Then, the process proceeds to step S242, the image quality of the SD image data of each frame is evaluated based on the autocorrelation calculated in step S241, and the process proceeds to step S243.
[0621]
In step S243, the image quality determination unit 432 determines whether the SD image data of each frame stored in the storage 444 has a certain level of image quality, based on the image quality evaluation result in step S242. If it is determined in step S243 that the SD image data of each frame stored in the storage 444 is not very good in image quality, step S244 is skipped and the process is terminated.
[0622]
If it is determined in step S243 that the SD image data of each frame stored in the storage 444 has good image quality to some extent, the process proceeds to step S244, where the image quality determination unit 432 The request signal is output to 433, and the process ends.
[0623]
This request signal is transmitted to the PDA 101 as described above, and the transmission processing unit 401 (FIG. 45) of the PDA 101 that has received the request signal transmits a class code image using the request signal as an event as described above. The The class code image transmitted from the PDA 101 is supplied and stored in the class code database 446 via the reception control unit 441 as described above.
[0624]
Here, in step S242, for example, whether or not the autocorrelation of the SD image data is equal to or greater than (or greater than) a predetermined threshold, or whether or not the autocorrelation of the SD image data is equal to or greater than the predetermined threshold. Whether or not the autocorrelation changes substantially before and after the weighted addition of the SD image data in the image data reception process of FIG. 49A can be adopted as an evaluation standard for the image quality of the SD image.
[0625]
In step S243, when the above evaluation criteria are satisfied, that is, when the autocorrelation of the SD image data is greater than or equal to a predetermined threshold, or when the autocorrelation of the SD image data is greater than or equal to the predetermined threshold, When the autocorrelation equal to or higher than the threshold value hardly changes before and after the weighted addition of the SD image data in the image data receiving process of FIG. 49A, the SD image data of each frame stored in the storage 444 is to some extent. It can be determined that the image quality is good.
[0626]
Although the autocorrelation of the SD image data is used here as the evaluation value for evaluating the image quality of the SD image data, other evaluation values can be used.
[0627]
That is, as the evaluation value for evaluating the image quality of the SD image data, for example, the SD image data stored in the storage 444 before and after the weighted addition in the image data receiving process of FIG. It is also possible to adopt the cross correlation of the above. In this case, the image quality determination unit 432 needs the SD image data before weighted addition and the SD image data after weighted addition. The SD image data before weighted addition is acquired by reading from the storage 444, The SD image data after weighted addition is acquired by requesting the registration unit 443.
[0628]
Note that when the cross-correlation as described above is used as the evaluation value for evaluating the image quality of the SD image data, for example, as in the case of using the auto-correlation of the SD image data, the cross-correlation of the SD image data is predetermined. Whether or not the threshold value is equal to or greater than the threshold value can be adopted as an evaluation standard for the image quality of the SD image data.
[0629]
Next, the adaptation process will be described with reference to the flowchart in FIG.
[0630]
The adaptation process is started when a class code image is stored in the class code database 446, for example.
[0631]
That is, when the class code image is stored in the class code database 446, the adaptive processing unit 447 (FIG. 48) sequentially reads the SD image data stored in the storage 444 and stores it in the buffer 451.
[0632]
Then, the prediction tap extraction unit 452 sets the oldest (temporally past frame) frame that has not yet been processed among the frames of the SD image data stored in the buffer 451 as the attention frame. Further, in step S251, the prediction tap extraction unit 452 improves the image quality of the SD image data of the frame of interest (this HD image data does not actually exist but is assumed to exist). SD pixels used for predicting the pixel value of the target pixel are extracted from the SD image data stored in the buffer 451, with one of the HD pixels constituting the pixel having not yet obtained a predicted value as the target pixel. And output as a prediction tap.
[0633]
Thereafter, the process proceeds to step S252, and the class code reading unit 454 sets, as the class code image of interest, the one generated from the HD image data of the frame of interest among the class code images stored in the class code database 446 (FIG. 47). The class code corresponding to the target pixel is read out from the class code of the target class code image and given to the coefficient memory 455 as an address.
[0634]
The class code reading unit 454 deletes the class code read from the class code database 446 from the class code database 446 after the reading, for example.
[0635]
When the coefficient memory 455 receives the class code as the address, the process proceeds to step S253, reads the prediction tap of the class corresponding to the class code, supplies it to the product-sum operation unit 453, and proceeds to step S254.
[0636]
In step S254, the product-sum operation unit 453 uses the prediction tap for the pixel of interest output from the prediction tap extraction unit 452 and the tap coefficient of the class of the pixel of interest supplied from the coefficient memory 455, and the expression (1) Thus, the pixel value of the target pixel (predicted value of the pixel value of the HD pixel) is obtained and output, and the process proceeds to step S255.
[0637]
In step S255, the class code reading unit 454 determines whether or not the class code is still stored in the class code database 446. If it is determined that the class code is still stored, the process returns to step S251 to return to the class code database. Hereinafter, the same processing is repeated with the HD pixel corresponding to one of the class codes stored in 446 as a new target pixel.
[0638]
If it is determined in step S255 that no class code is stored in the class code database 446, the process is terminated.
[0639]
As described above, in the reception processing unit 402, the SD image data transmitted from the transmission processing unit 401 is accumulated, and the image quality is improved by weighted addition.
[0640]
Further, when the image quality of the SD image data is improved to some extent, a request signal is transmitted to the transmission processing unit 401, whereby the class code image transmitted from the transmission processing unit 401 is stored in the class code database 446. Is done. Then, the SD image data is converted into HD image data using the class code image.
[0641]
Accordingly, in the reception processing unit 402, when the image quality of the SD image data is improved to some extent, this triggers a class code image, and the image quality of the SD image data is further increased based on the class code image. Improved HD image data can be obtained. That is, the improvement of the image quality of the SD image data is a trigger, and the SD image data is converted into HD image data with better image quality.
[0642]
Therefore, for the user, the image quality of the image is suddenly improved.
[0643]
In this case, the image quality of the SD image data (S / N (Signal / Noise), etc.) is improved by weighted addition, and the SD image data is converted into an HD image after the image quality of the SD image data is improved to some extent. Although conversion to data is performed, conversion to HD image data can be performed without improving the image quality of the SD image data.
[0644]
However, the tap coefficient stored in the coefficient memory 455 used in the adaptive processing unit 447 (FIG. 48) that performs the adaptive process as the conversion process from the SD image to the HD image generally uses the learning HD image data as the teacher data. In addition, since the SD image data obtained by simply degrading the resolution of the HD image data is used as student data, the normal equation of equation (8) is obtained, that is, SD as student data. Since the image data is not the one with missing data or noise superimposed, it is suitable for SD image data with such data missing or with greatly deteriorated image quality with noise superimposed When processing is performed, it may be difficult to obtain HD image data with sufficiently improved image quality.
[0645]
For this reason, as described above, the adaptive processing is desirably performed using SD image data with improved image quality, that is, here accumulated SD image data received in the past while being weighted and added.
[0646]
In the above case, the image quality determination unit 432 determines the image quality of the SD image data stored in the storage 444 based on the autocorrelation or cross-correlation of the SD image data. The determination can be made to be made by the user.
[0647]
That is, when the SD image data stored in the storage 444 is displayed on the LCD 3 or the like (FIG. 23) and the user feels that the image quality of the SD image data has improved, the operation unit 224 or the like (FIG. 23) is displayed. , It can be operated to indicate that the image quality has improved. In this case, the operation unit 224 outputs an operation signal indicating that such an operation has been performed (hereinafter, appropriately referred to as an image quality improvement operation signal) to the request signal transmission unit 433. When the request signal transmission unit 433 receives the image quality improvement operation signal as indicated by a dotted line in FIG. 47, the request signal transmission unit 433 transmits the request signal.
[0648]
Also in this case, since the class code image is transmitted from the transmission processing unit 401 (FIG. 45) to the reception processing unit 402, the reception processing unit 402 also improved the image quality of the SD image data. HD image data can be obtained. That is, in this case, an SD image is converted into an HD image with a user operation as an external input as a trigger.
[0649]
In addition, the request signal transmission unit 433 determines whether, for example, the PDA 103 as a receiving apparatus is in a standby state where data is not transmitted / received, and transmits a request signal when the data is in a standby state. It is possible to In this case, the class code is transmitted from the transmission processing unit 401 to the reception processing unit 402 when the PDA 103 (reception processing unit 402) is in a standby state.
[0650]
In addition, in the transmission processing unit 401 of FIG. 45, the event detection unit 412 detects a request signal and determines whether the PDA 101 as a transmission device is in a standby state in which data transmission / reception is not performed. It is possible to output an event message when In this case, the class code is transmitted from the transmission processing unit 401 to the reception processing unit 402 when the PDA 101 (transmission processing unit 401) is in a standby state.
[0651]
In addition, transmission of a class code from the transmission processing unit 401 to the reception processing unit 402 is performed as a predetermined event that both the PDA 101 as a transmission device and the PDA 103 as a reception device are in a standby state. Is also possible.
[0652]
Next, FIG. 50 illustrates a second configuration example of the transmission processing unit 401 in FIG. 44. In the figure, portions corresponding to those in FIG. 45 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, the transmission processing unit 401 in FIG. 50 is basically configured in the same manner as in FIG. 45 except that a storage 413, a learning unit 414, a tap coefficient buffer 415, and a storage 416 are newly provided. Yes.
[0653]
The storage 413 is supplied with the same HD image data as that supplied to the encoding unit 411, and the storage 413 temporarily stores the HD image data.
[0654]
The learning unit 414 uses the HD image data newly stored in the storage 413 as learning image data by performing learning to solve the normal equation of Expression (8) described above, and is used for adaptive processing. Get the tap coefficient. This tap coefficient is supplied to the tap coefficient buffer 415.
[0655]
The tap coefficient buffer 415 temporarily stores the tap coefficient output from the learning unit 414. The tap coefficient stored in the tap coefficient buffer 415 is transmitted together with the class code at the same timing as the class code stored in the class code buffer 425 is transmitted in the transmission control unit 422, for example.
[0656]
The storage 416 temporarily stores information obtained during the learning when the learning unit 414 learns the tap coefficient.
[0657]
Next, FIG. 51 illustrates a configuration example of the learning unit 414 of FIG.
[0658]
The teacher data memory 461 stores, for example, HD image data newly stored in the storage 413 as teacher data.
[0659]
The data compression unit 462 compresses the HD image data in the same manner as the data compression unit 421 in FIG. 50, and generates and outputs SD image data.
[0660]
The student data memory 463 stores the SD image data output from the data compression unit 462 as student data.
[0661]
The prediction tap extraction unit 464 sequentially uses the HD pixels constituting the HD image data as the teacher data stored in the teacher data memory 461 as the attention pixel, and the prediction tap extraction unit 452 (described later) of the attention pixel. 54 is generated from SD pixels constituting SD image data as student data stored in the student data memory 463, and is added to the adding unit 467. Supply.
[0662]
The class tap extraction unit 465 uses the same class tap as the class tap extraction unit 423 of FIG. 50 for the target pixel, and the HD pixels that constitute the HD image data as the teacher data stored in the teacher data memory 461 And output to the class classification unit 466.
[0663]
Based on the class tap for the target pixel supplied from the class tap extraction unit 465, the class classification unit 466 classifies the target pixel in the same manner as in the class classification unit 424 in FIG. 50, and represents the class. The class code is output to the adding unit 467.
[0664]
The adding unit 467 reads out the teacher data (HD pixel) serving as the target pixel from the teacher data memory 461, and receives the student data constituting the prediction tap from the prediction tap generation unit 464 and the teacher data as the target pixel. The target addition is performed for each class supplied from the class classification unit 466 while using the storage contents of the storage 416 (FIG. 50).
[0665]
That is, the adding unit 467 first uses the prediction tap (student data) for each class corresponding to the class code supplied from the class classification unit 466, and becomes each component in the matrix A of Expression (8). , Multiplication of student data (x _in x _im ) And a calculation corresponding to summation (Σ).
[0666]
Further, the adding unit 467 also uses the prediction tap (student data) and the target pixel (teacher data) for each class corresponding to the class code supplied from the class classification unit 466, and uses the vector v of Expression (8). Multiplying student data and teacher data (x _in y _i ) And a calculation corresponding to summation (Σ).
[0667]
On the other hand, the storage 416 stores, for each class, the components of the matrix A and the components of the vector v in Expression (8) obtained by the adding unit 467 until the previous learning.
[0668]
In the case of performing learning using new learning image data, the adding unit 467 stores the component of the matrix A in the expression (8) obtained by the previous learning stored in the storage 416 and the vector v. A component x is read out, and the corresponding component x calculated using the teacher data and student data obtained from the new learning image data for the component of the matrix A or vector v _in x _im Or x _in y _i Are added (by performing addition represented by summation in matrix A and vector v), a new normal equation shown in Equation (8) is established for each class.
[0669]
Therefore, in the addition unit 467, the normal equation of Expression (8) is established based not only on new learning image data but also on image data used for past learning. That is, the storage 416 accumulates the components of the matrix A and the vector v obtained in the past learning, and the adding unit 467 also uses the accumulated components of the matrix A and the vector v to obtain the formula ( The normal equation of 8) is established.
[0670]
For example, when learning is performed for the first time in the learning unit 414, the storage 416 does not store the components of the matrix A and the vector v obtained up to the previous learning, so the equation (8) The normal equation is established using only the existing learning image data.
[0671]
In this case, there may be a class in which the number of normal equations necessary for obtaining the tap coefficient cannot be obtained due to an insufficient number of samples of learning image data.
[0672]
Therefore, the storage 416 stores, as initial values, the components of the matrix A for each class and the components of the vector v obtained by performing learning using a large number of HD image data prepared in advance as learning data. You can let it go. In this case, it is possible to prevent the occurrence of a class in which the number of normal equations necessary for obtaining the tap coefficient cannot be obtained.
[0673]
The adding unit 467 uses the matrix A and vector v components obtained from the new learning image data and the matrix A and vector v components stored in the storage 416 to newly add a matrix for each class. When the components of A and vector v are determined, these components are supplied to the storage 416 and stored in an overwritten form.
[0674]
Further, the adding unit 467 supplies the tap coefficient determination unit 468 with the normal equation of the formula (8) configured by the newly obtained components of the matrix A and the vector v for each class.
[0675]
The tap coefficient determination unit 468 obtains the tap coefficient for each class by solving the normal equation for each class supplied from the adding unit 467, and the tap coefficient for each class is obtained as the tap coefficient buffer 415 (FIG. 50). And store it in the form of overwriting.
[0676]
In the learning unit 414 in FIG. 51, the class tap extraction unit 465 generates a class tap from the HD pixels constituting the HD image data in the same manner as the class tap extraction unit 423 in FIG. In FIG. 5, since the class classification is performed based on the class tap constituted by the HD pixels, the tap coefficient for each class obtained by the tap coefficient determination unit 468 is a tap coefficient for each HD class.
[0677]
In the transmission processing unit 401 configured as described above, in addition to the image data transmission process, the class code generation process, and the class code transmission process described with reference to FIG. 46, a learning process for obtaining a tap coefficient for each HD class, and its HD Tap coefficient transmission processing for transmitting tap coefficients for each class is performed.
[0678]
Therefore, the learning process and the tap coefficient transmission process performed by the transmission processing unit 401 in FIG. 50 will be described with reference to the flowchart in FIG.
[0679]
First, the learning process performed by the transmission processing unit 401 in FIG. 50 will be described with reference to the flowchart in FIG.
[0680]
The learning process is started in the learning unit 414 (FIG. 51) at a predetermined timing.
[0681]
That is, the learning unit 414 starts the learning process, for example, periodically or when new HD image data of a predetermined number of frames or more is stored in the storage 413 (FIG. 50).
[0682]
When the learning process is started, first, HD image data newly stored in the storage 413 (FIG. 50) between the previous learning and the current learning is read as new learning image data. Is supplied to the teacher data memory 461 and stored as teacher data.
[0683]
In step S261, the adding unit 467 reads the component of the matrix A and the component of the vector v in the equation (8) from the storage 416, and proceeds to step S262.
[0684]
In step S262, the data compression unit 462 reads the teacher data stored in the teacher data memory 461 and converts it into SD image data. The SD image data is supplied to the student data memory 463 and stored as student data.
[0685]
In step S263, the prediction tap extraction unit 461 selects one of the HD pixels as the teacher data stored in the teacher data memory 416 that has not yet been set as the target pixel, and sets the target pixel. The prediction tap is generated by reading several SD pixels as student data from the student data memory 463.
[0686]
Further, in step S263, the class tap extraction unit 465 generates a class tap by reading several HD pixels as teacher data from the teacher data memory 461 for the target pixel.
[0687]
The prediction tap generated by the prediction tap extraction unit 464 is supplied to the addition unit 467, and the class tap generated by the class tap extraction unit 465 is supplied to the class classification unit 466.
[0688]
Thereafter, the process proceeds to step S264, where the class classification unit 466 classifies the target pixel based on the class tap supplied from the class tap extraction unit 465, and adds a class code representing the class of the target pixel to the addition unit 467. Supply.
[0689]
In step S265, the adding unit 467 reads the target pixel from the teacher data memory 461, and calculates the components of the matrix A and the vector v using the target pixel and the prediction tap from the prediction tap extraction unit 464. To do. Further, the adding unit 467 predicts a pixel of interest for a component corresponding to the class code from the class classification unit 466 among the components of the matrix A and the vector v read from the storage 416 (FIG. 50) in step S261. The components of the matrix A and the vector v obtained from the tap are added, and the process proceeds to step S266.
[0690]
In step S266, the prediction tap extraction unit 464 determines whether there is still teacher data that is not a pixel of interest in the teacher data memory 461. If it is determined that there is a pixel of interest, the process returns to step S263, The same processing is repeated thereafter with the unsupervised teacher data as a new target pixel.
[0691]
If it is determined in step S266 that there is no teacher data that is not a pixel of interest in the teacher data memory 461, the adding unit 467 adds the components of the matrix A and the vector v obtained for each class so far. Is supplied to the tap coefficient determination unit 468, and the process proceeds to step S267.
[0692]
In step S267, the addition unit 467 also supplies the components of the matrix A and the vector v for each class supplied to the tap coefficient determination unit 468 to the storage 416 and stores them in an overwritten form, and the process proceeds to step S268.
[0693]
In step S268, the tap coefficient determination unit 468 obtains a tap coefficient for each class by solving the normal equation for each class supplied from the adding unit 467. Further, in step S268, the tap coefficient determination unit 468 supplies the tap coefficient for each class to the tap coefficient buffer 415 to be overwritten, and ends the learning process.
[0694]
Next, the tap coefficient transmission processing performed by the transmission processing unit 401 in FIG. 50 will be described with reference to the flowchart in FIG.
[0695]
In the tap coefficient transmission process, first, in step S271, the transmission control unit 401 determines whether or not a predetermined event has occurred in the same manner as in step S221 in FIG. If it is determined that no occurrence has occurred, the process returns to step S271 and waits until a predetermined event occurs.
[0696]
If it is determined in step S271 that a predetermined event has occurred, the process proceeds to step S272, and the transmission control unit 422 determines whether the tap coefficient for each class (HD class) is stored in the tap coefficient buffer 415. Determine.
[0697]
If it is determined in step S272 that the tap coefficient for each HD class is not stored, the process returns to step S271, and the same processing is repeated thereafter.
[0698]
If it is determined in step S272 that the tap coefficient for each HD class is stored, the process proceeds to step S273, and the transmission control unit 422 reads the class code stored in the tap coefficient buffer 415 and transmits it as transmission data. select. The transmission data is supplied from the transmission control unit 422 to the communication I / F 218 (FIG. 23) and transmitted from the antenna 64.
[0699]
When the transmission of the tap coefficients for all HD classes stored in the tap coefficient buffer 415 is completed in step S223, the transmission control unit 422 returns to step S271, and thereafter repeats the same processing.
[0700]
Next, FIG. 53 shows a second configuration example of the reception processing unit 402 in FIG. That is, FIG. 53 shows a configuration example of the reception processing unit 402 when the transmission processing unit 401 is configured as shown in FIG.
[0701]
In the figure, portions corresponding to those in FIG. 47 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, the reception processing unit 402 in FIG. 53 is basically configured in the same manner as in FIG. 47 except that an adaptive processing unit 448 is provided instead of the adaptive processing unit 447.
[0702]
However, in the embodiment of FIG. 53, the tap coefficient for each HD class may be transmitted in addition to the SD image data and the class code representing the HD class from the transmission processing unit 401 of FIG. When each tap coefficient is transmitted, the reception control unit 441 supplies the adaptive processing unit 448 with the tap coefficient for each HD class.
[0703]
The adaptive processing unit 448 performs the same processing (adaptive processing) as that in the adaptive processing unit 447 of FIG. 48 using the tap coefficient supplied from the reception control unit 441, thereby generating HD image data ( Predict).
[0704]
That is, FIG. 54 shows a configuration example of the adaptive processing unit 448 of FIG. In the figure, portions corresponding to those in the case of the adaptive processing unit 447 in FIG. 48 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, the adaptive processing unit 448 in FIG. 54 is configured in the same manner as the adaptive processing unit 447 in FIG. 48 except that a registration unit 456 is newly provided.
[0705]
The registration unit 456 receives the tap coefficient for each HD class supplied from the reception control unit 441 (FIG. 53), and stores it in the form of overwriting the coefficient memory 455.
[0706]
Therefore, in the embodiment of FIG. 48, adaptive processing is performed using a fixed tap coefficient for each HD class. However, in the embodiment of FIG. 54, the transmission processing unit 401 of FIG. When a tap coefficient for each new HD class is transmitted, the content stored in the coefficient memory 455 is updated with the tap coefficient for each new HD class, and is adapted using the tap coefficient for each new HD class. Processing is performed.
[0707]
In the transmission processing unit 401 of FIG. 50, as described above, the tap coefficient is updated using the new HD image data as new learning image data. It will be converted to an image close to the image. Therefore, the reception processing unit 402 in FIG. 53 performs adaptive processing using such a tap coefficient, so that it is possible to obtain an HD image with better image quality.
[0708]
The data compression unit 421 in FIGS. 45 and 50 and the data compression unit 462 in FIG. 51 generate SD image data by thinning out the number of pixels in the spatial direction of the HD image data. In addition, the SD image data can be generated, for example, by thinning out the number of pixels in the time direction of the HD image data.
[0709]
Next, FIG. 55 illustrates a third configuration example of the transmission processing unit 401 in FIG. 44.
[0710]
In the embodiment of FIG. 55, the transmission processing unit 401 encodes and transmits image data by vector quantization.
[0711]
That is, the vectorization unit 501 is supplied with image data captured by the CCD camera 65 (FIG. 23) and image data stored in the HDD 215. The vectorization unit 501 vectorizes the image data supplied thereto. That is, for example, the vectorization unit 501 blocks image data of each frame supplied thereto into blocks each having 3 × 3 pixels in the horizontal direction × vertical direction, for example, and 9 pixels ( A vector (hereinafter, referred to as an image vector as appropriate) is generated with a vector component obtained by arranging pixel values of (3 × 3 pixels) in a predetermined order. The image vector obtained by the vectorization unit 501 is supplied to the vector quantization unit 502 and the difference calculation unit 504.
[0712]
The vector quantization unit 502 refers to the code book stored in the code book storage unit 509 and performs vector quantization on the image vector supplied from the vectorization unit 501.
[0713]
That is, the vector quantization unit 502 calculates the distance between all code vectors registered in the code book stored in the code book storage unit 509 and the image vector from the vectorization unit 501, and minimizes the distance. A code corresponding to the code vector to be output is output as a vector quantization result. The code output from the vector quantization unit 502 is supplied to the local decoding unit 503, the entropy encoding unit 505, and the updating unit 506.
[0714]
The local decoding unit 503 converts the code supplied from the vector quantization unit 502 into the code book stored in the code book storage unit 509 (the same code book used by the vector quantization unit 502 for vector quantization). Use vector inverse quantization. That is, the local decoding unit 503 outputs a code vector corresponding to the code output from the vector quantization unit 502 in the code book stored in the code book storage unit 509 as a vector inverse quantization result. The code vector as the vector dequantization result is supplied from the local decoding unit 503 to the difference calculation unit 504.
[0715]
The difference calculation unit 504 calculates a difference between the image vector supplied from the vectorization unit 501 and the code vector obtained by vector quantization and further vector inverse quantization supplied from the local decoding unit 503. Then, a vector obtained as a result (hereinafter referred to as a difference vector as appropriate) is supplied to the entropy encoding unit 505 and the updating unit 506.
[0716]
The entropy encoding unit 505 entropy the code as the vector quantization result of the image vector supplied from the vector quantization unit 502 and the difference vector obtained from the difference calculation unit 504 and obtained for the image vector. Encode and output as transmission data. This transmission data is transmitted to the reception processing unit 402 (PDA 103 as a receiving device) via the communication I / F 218 (FIG. 23).
[0717]
Since the difference vector is entropy-encoded in the entropy encoding unit 505 as described above, the data amount of the transmission data amount can be reduced when the frequency of the difference vector being high is high. That is, when the vector quantization unit 502 has a high frequency in which the quantization error due to vector quantization (encoding) performed using the code book stored in the code book storage unit 509 is high, the data of the transmission data The amount can be reduced. This means that the data amount of transmission data for obtaining an image of the same image quality is reduced. Therefore, assuming that the data amount of transmission data is constant by changing the way of viewing, the amount of data described above is reduced. This is equivalent to improving the image quality by the amount of reduction.
[0718]
The update unit 506 updates the code book stored in the code book database 507 based on the code supplied from the vector quantization unit 502 and the difference vector supplied from the difference calculation unit 504. Note that the update unit 506 receives from the selection unit 508 information that specifies the code book stored in the code book storage unit 509, that is, the code book currently used for vector quantization (a code book number to be described later). ) Is supplied, and the update unit 506 identifies the code book to be updated based on the code book number.
[0719]
The code book database 507 stores one or more code books used for vector quantization of image vectors.
[0720]
The code book database 507 stores, as an initial value of the code book, for example, a code book obtained by an LBG (Linde Buzo Gray) algorithm using a large amount of image data for learning prepared in advance. The update unit 506 updates the initial value code book as appropriate, as will be described later.
[0721]
The code book database 507 stores one or more code books. For example, the code book database 507 may initially store one or more code books of initial values, Initially, only one initial value codebook may be stored, and then the initial value codebook may be copied as necessary.
[0722]
Selection information is supplied to the selection unit 508, and the selection unit 508 performs vector quantization from one or more codebooks stored in the codebook database 507 according to the selection information. Select what to use. Then, the selection unit 508 reads the selected code book from the code book database 507, supplies it to the code book storage unit 509, and stores it in an overwritten form.
[0723]
Here, as the selection information, for example, user input, communication partner information, the same image data supplied to the vectorization unit 501 and the like can be adopted. The user input is given by the user operating the operation unit 224 (FIG. 23) or the like. The communication partner information is transmitted from the PDA 103 as the receiving device when the PDA 101 as the transmitting device starts communication with the PDA 103 as the receiving device, so that the antenna 64 and the communication I / F 218 (FIG. 23) are transmitted. ). Further, the same image data supplied to the vectorization unit 501 is given from the CCD camera 65 or HDD 215 (FIG. 23).
[0724]
When a user input is given as selection information, the selection unit 508 selects one code book from one or more code books stored in the code book database 507 according to the user input. Therefore, in this case, the vector quantization unit 502 performs vector quantization of the image data using the code book designated by the user.
[0725]
In addition, when information on a communication partner is given as selection information, the selection unit 508 selects one code book from one or more code books stored in the code book database 507 according to the information on the communication partner. . Therefore, in this case, the vector quantization unit 502 uses a codebook that differs depending on the communication partner (different for each communication partner or a group of a plurality of users and different for each group), and uses vector code of image data. Will be performed.
[0726]
Furthermore, when image data is given as selection information, the selection unit 508 selects one code book from one or more code books stored in the code book database 507 according to the image data. Accordingly, in this case, the vector quantization unit 502 uses a different code book for each characteristic of the image data (for example, activity representing the pattern of the image data, brightness, movement, etc.), and vector quantization of the image data. Will be done.
[0727]
Note that the selection unit 508 selects one code book from one or more code books stored in the code book database 507 and stores it in the code book storage unit 509, that is, the vector quantization unit 502 performs vector quantization. When the code book used for conversion is changed, a code book number specifying the code book after the change (the code book selected from the code book database 507) is supplied to the update unit 506.
[0728]
In the transmission processing unit 401 configured as described above, image data transmission processing for vector-quantizing and transmitting image data, codebook selection processing for selecting a codebook used for the vector quantization, and storage of the codebook database 507 An update process for updating the contents is performed.
[0729]
First, image data transmission processing and codebook selection processing performed by the transmission processing unit 401 in FIG. 55 will be described with reference to the flowchart in FIG.
[0730]
First, the image data transmission process will be described with reference to the flowchart of FIG.
[0731]
In the image data transmission process, image data to be transmitted is supplied to the vectorization unit 501 in units of frames, for example, and the vectorization unit 501 receives the image data.
[0732]
In step S301, the vectorization unit 501 vectorizes one frame of image data supplied thereto. That is, the vectorization unit 501 blocks one frame of image data into, for example, 3 × 3 pixel blocks, and arranges the pixel values of 9 pixels (3 × 3 pixels) of the block in a predetermined order. An image vector as a component is generated. In the vectorization unit 501, an image vector obtained for one frame of image data is supplied to the vector quantization unit 502 and the difference calculation unit 504.
[0733]
Note that the processing in steps S302 to S305 described below is performed for each image vector obtained for one frame of image data.
[0734]
When the vector quantization unit 502 receives an image vector from the vectorization unit 501, in step S302, the vector quantization unit 502 vector-quantizes the image vector using the code book stored in the code book storage unit 509, and obtains a code obtained as a result. Are supplied to the local decoding unit 503, the entropy encoding unit 505, and the updating unit 506, and the process proceeds to step S303.
[0735]
In step S303, the local decoding unit 503 performs vector inverse quantization on the code supplied from the vector quantization unit 502 using the code book stored in the code book storage unit 509, and converts the code vector obtained as a result into a difference. The data is supplied to the calculation unit 504, and the process proceeds to step S304.
[0736]
In step S304, the difference calculation unit 504 calculates the difference between the image vector supplied from the vectorization unit 501 and the code vector supplied from the local decoding unit 503, and the resulting difference vector is entropy encoded. The data is supplied to the unit 505 and the update unit 506, and the process proceeds to step S305.
[0737]
In step S305, the entropy encoding unit 505 supplies the code as the vector quantization result of the image vector supplied from the vector quantization unit 502 and the difference obtained for the image vector supplied from the difference calculation unit 504. Entropy-encode the vector and output as transmission data. This transmission data is transmitted to the reception processing unit 402 (PDA 103 as a receiving device) via the communication I / F 218 (FIG. 23).
[0738]
Thereafter, the process proceeds to step S306, where the vectorization unit 501 determines whether there is image data of the next frame. If it is determined that there is, the process returns to step S301, and the same processing is repeated thereafter.
[0739]
If it is determined in step S306 that there is no image data for the next frame, the process ends.
[0740]
Next, the code book selection process will be described with reference to the flowchart in FIG.
[0741]
Note that the code book selection process is started immediately before the image data transmission process (FIG. 56A) is started, for example.
[0741]
In the code book selection process, first, in step S311, the selection unit 508 selects a default code book from one or more code books stored in the code book database 507, and stores it in the code book storage unit 509. Supply and store.
[0743]
Here, as the default codebook, for example, the above-described initial value codebook can be adopted.
[0744]
Thereafter, the process proceeds to step S312, and the selection unit 508 determines whether the selection information is supplied. If the selection unit 508 determines that the selection information is not supplied, the selection unit 508 skips step S313 and proceeds to step S314.
[0745]
If it is determined in step S312 that selection information has been supplied, the process proceeds to step S313, and the selection unit 508 selects one or more codebooks stored in the codebook database 507 according to the selection information. Those used for vector quantization are selected and supplied to the codebook storage unit 509 for storage.
[0746]
In step S314, the selection unit 508 determines whether the transmission of the image data by the image data transmission process (FIG. 56A) started immediately after starting the current codebook selection process is completed. If it is determined that the process has not been completed, the process returns to step S312 to repeat the same processing.
[0747]
Therefore, in this case, if new selection information is supplied to the selection unit 508 while image data is being transmitted by the image data transmission process, the code book stored in the code book storage unit 509 is stored. That is, the code book used for vector quantization in the vector quantization unit 502 can be changed based on the new selection information.
[0748]
On the other hand, if it is determined in step S314 that the transmission of the image data has ended, the process ends.
[0749]
In the embodiment of FIG. 56B, as described above, new selection information is supplied to the selection unit 508 while image data is being transmitted by the image data transmission process. In this case, the code book used in vector quantization can be changed at any time based on the new selection information, but the code book while the image data is being transmitted by the image data transmission process is changed. Changes can be limited to one time only. That is, the change of the code book can be limited to only the change from the default code book to the code book based on the selection information supplied first.
[0750]
Next, update processing for updating the storage contents of the code book database 507 performed in the transmission processing unit 401 of FIG. 55 will be described. Before that, the code book stored in the code book database 507 and the update unit are updated. The configuration of 506 will be described.
[0751]
FIG. 57 shows an example of a code book stored in the code book database 507 of FIG.
[0752]
The code book has a code book number and a code book version.
[0753]
The code book number is a unique number that identifies the code book. Therefore, according to the code book number, the code book is uniquely identified. The code book version is information indicating the version of the code book, and is represented by, for example, the date and time when the code book is updated, the number of times of update, and the like.
[0754]
In the embodiment of FIG. 57, the code book is a code n and a code vector V in the same way as a general code book. _n (= (A _n , B _n ,...)) Are associated with each other. In the embodiment of FIG. 57, the number of codes (code vectors) is n + 1, and integer values from 0 to N are used as codes.
[0755]
Further, in the code book of the embodiment of FIG. 57, each code #n includes a code vector V _n And frequency A until the last update _n , Addition value of difference vector Σ △ _n (= (A ' _n , B ' _n , ...)), and frequency B from the last update to the present _n Are also associated.
[0756]
Here, frequency A until the last update A _n Represents the frequency with which code #n is output as a vector quantization result in vector quantization until the previous codebook update.
[0757]
Also, the addition value Σ △ of the difference vector _n Is the difference vector Δ when the vector quantization result of code #n is obtained in the vector quantization from immediately after the previous codebook update to the present. _n Represents the sum of
[0758]
Furthermore, the frequency B from the last update to the present _n Represents the frequency with which code #n is output as the vector quantization result in the vector quantization from immediately after the previous codebook update to the present.
[0759]
Frequency A until the last update A _n , Addition value of difference vector Σ △ _n , And frequency B from the last update to the present _n Is for vector quantization performed using that codebook, and the frequency A until the previous update of vector quantization performed using another codebook _n , Addition value of difference vector Σ △ _n , And frequency B from the last update to the present _n Is registered in the other codebook.
[0760]
For the default codebook, the frequency A until the last update _n , Addition value of difference vector Σ △ _n , And frequency B from the last update to the present _n Are all set to 0, for example.
[0761]
Next, FIG. 58 shows a configuration example of the update unit 506 of FIG.
[0762]
As shown in FIG. 58, the update unit 506 includes a data update unit 521 and a code book update unit 522.
[0763]
The data update unit 521 is supplied from the selection unit 508 with a code book number as information specifying the code book stored in the code book storage unit 509, that is, the code book used in the vector quantization unit 502. It has come to be. Further, the data update unit 521 is also supplied with a code as a vector quantization result of the image vector output from the vector quantization unit 502. The data update unit 521 is also supplied with the difference vector obtained from the difference calculation unit 504 and obtained for the image vector subjected to vector quantization by the vector quantization unit 502.
[0764]
Then, the data updating unit 521 selects the code book currently used for vector quantization from one or more code books stored in the code book database 507, and the code supplied from the selection unit 508 (FIG. 55). Specify by book number. Furthermore, the data update unit 521 uses the identified code book as the target code book, and adds the difference vector ΣΔ of the target code book. _n , And frequency B from the last update to the present _n Is updated with the code from the vector quantization unit 502 and the difference vector from the difference calculation unit 504.
[0765]
The code book updating unit 522 converts each of the one or more code books stored in the code book database 507 into a frequency A up to the previous update time stored in each code book. _n , Addition value of difference vector Σ △ _n , And frequency B from the last update to the present _n And the updated codebook is stored in the form of overwriting the codebook database 507.
[0766]
The update process performed by the update unit 506 configured as described above is the addition value ΣΔ of the difference vector stored in the codebook database 507. _n , And frequency B from the last update to the present _n Data update processing for updating the code vector, and the code vector of the code book, the frequency A up to the previous update time registered in the code book _n , Addition value of difference vector Σ △ _n , And frequency B from the last update to the present _n Codebook update processing to be updated based on the above.
[0767]
Therefore, the data update process and the code book update process will be described with reference to the flowchart of FIG.
[0768]
First, the data update process will be described with reference to the flowchart in FIG.
[0769]
In the data update process, first, in step S321, the data update unit 521 determines whether a code book number is received from the selection unit 508 (FIG. 55).
[0770]
If it is determined in step S321 that the code book number has been received from the selection unit 508, that is, if the code book used for vector quantization in the vector quantization unit 502 has been changed, the process proceeds to step S322 to update the data The unit 521 sets the code book specified by the code book number supplied from the selection unit 508 among the one or more code books stored in the code book database 507 as the target code book, and proceeds to step S323.
[0771]
If it is determined in step S321 that the code book number has not been received from the selection unit 508, that is, the code book used for vector quantization in the vector quantization unit 502 is not changed. If the code book used in the vector quantization is directly used as the target codebook, the process skips step S322 and proceeds to step S323.
[0772]
In step S323, the data update unit 521 is supplied with a code as a vector quantization result of the image vector from the vector quantization unit 502 (FIG. 55) and is obtained from the difference calculation unit 504 for the image vector. It is determined whether a difference vector has been supplied.
[0773]
If it is determined in step S323 that the code and the difference vector are not supplied, the process returns to step S321, and the same processing is repeated thereafter.
[0774]
If it is determined in step S323 that the code and the difference vector have been supplied, the process proceeds to step S324, and the data update unit 521 adds the difference vector ΣΔ registered in the target codebook of the codebook database 507. _n , And frequency B from the last update to the present _n Update.
[0775]
That is, the data update unit 521 uses the entry of the code #n supplied from the vector quantization unit 502 of the target codebook as the target entry, and the frequency B from the previous update time to the present in the target entry. _n Is incremented by one. Furthermore, the data update unit 521 adds the difference vector ΣΔ in the entry of interest. _n Is added to the difference vector supplied from the difference calculation unit 504, and the addition value is added to the addition value ΣΔ of the new difference vector. _n And overwriting the attention entry of the attention codebook.
[0776]
Then, the process returns to step S321, and the same processing is repeated thereafter.
[0777]
Next, the code book update process will be described with reference to the flowchart in FIG.
[0778]
The code book update process is started at an arbitrary timing periodically or irregularly.
[0779]
In the code book update process, the target code book that is currently the target of update is excluded from the target of the data update process of FIG. 59A (so-called exclusive control) in order to maintain the consistency of the contents. To be).
[0780]
In the code book update process, first, in step S331, the code book update unit 522 initializes a variable i representing a code book number to 1, for example, and proceeds to step S332. In step S332, the code book updating unit 522 sets the i-th code book among the one or more code books stored in the code book database 507 as the target code book, and proceeds to step S333. In step S333, the code book updating unit 522 initializes a variable n representing a code in the target code book to 0, for example, and proceeds to step S334.
[0781]
In step S334, the code book updating unit 522 sets the entry of the code #n in the target code book as the target entry, for example, the frequency A until the previous update of the target entry. _n And frequency B from the last update to the present _n As the weight, and the code vector V of the entry of interest _n And the addition value Σ △ of the difference vector _n Is weighted and added to the code vector V of the entry of interest. _n Update.
[0782]
That is, the code book updating unit 522, for example, according to the following equation, the code vector V _n Update.
[0783]
V _n = V _n + B _n × Σ △ _n / (A _n + B _n )
[0784]
Then, the code book updating unit 522 proceeds to step S335, and the frequency A up to the previous update time in the entry of interest _n , Addition value of difference vector Σ △ _n , And frequency B from the last update to the present _n Update.
[0785]
That is, the code book updating unit 522 uses the frequency A up to the time of the previous update. _n And the frequency B from the last update to the present _n And add the value to the new frequency A _n And Furthermore, the code book update unit 522 adds the difference vector ΣΔ. _n , And frequency B from the last update to the present _n Are initialized to 0, and the process proceeds to step S336.
[0786]
In step S336, the code book updating unit 522 determines whether or not the variable n representing the code in the target code book is equal to N which is the maximum value. If it is determined in step S336 that the variable n is not equal to N, the process proceeds to step S337, and the code book updating unit 522 increments the variable n by 1. Then, the process returns to step S334, and the same processing is repeated thereafter.
[0787]
If it is determined in step S336 that the variable n is equal to N, that is, if all entries of the target codebook are updated, the process proceeds to step S338, and the variable i is stored in the codebook database 507. It is determined whether it is equal to I representing the number of books.
[0788]
If it is determined in step S338 that the variable i is not equal to I, the process proceeds to step S339, and the code book updating unit 522 increments the variable i by 1. And it returns to step S332 and repeats the same process hereafter.
[0789]
On the other hand, if it is determined in step S338 that the variable i is equal to I, that is, if the update of all the codebooks stored in the codebook database 507 is completed, the process ends.
[0790]
As described above, in the code book database 507, only one initial value code book can be stored at first, and then the initial value code book can be copied as necessary. However, in this case, if the code book that is the copy source is updated, a copy of the code book of the initial value cannot be generated thereafter. Therefore, the code book serving as the copy source is excluded from the data update process and code book update process shown in FIG.
[0791]
As described above, the code book stored in the code book database 507 includes a code as a result of vector quantization performed using the code book and a difference vector corresponding to the code (the image from which the code is obtained). Therefore, it is possible to improve the image quality.
[0792]
That is, as described above, the code book used for vector quantization in the vector quantization unit 502 is selected by, for example, a communication partner. In this case, the code book used for vector quantization differs for each communication partner. Therefore, if attention is paid to a certain code book, the target code book is a code obtained from image data transmitted to a specific communication partner. It will be updated with the difference vector.
[0793]
As a result, the codebook of interest is updated so that the frequency at which the difference vector becomes 0 is increased according to the characteristics (for example, frequency characteristics) of the image that is often transmitted to a specific communication partner. Thus, the data amount of the transmission data is reduced. As described above, the image quality can be improved by reducing the data amount of the transmission data.
[0794]
In addition, as described above, the code book used for vector quantization in the vector quantization unit 502 is selected by, for example, an activity representing a pattern of image data to be vector quantized. In this case, the code book used for vector quantization is different for each picture of the image data. Therefore, if attention is paid to a certain code book, the noticed code book is obtained from the code obtained from the image data of the specific picture and the difference vector. Will be updated.
[0795]
As a result, the codebook of interest is updated so that the frequency at which the difference vector becomes 0 is increased for an image of a specific pattern, and the data amount of transmission data is reduced. As described above, the image quality can be improved by reducing the data amount of the transmission data.
[0796]
Next, FIG. 60 illustrates a third configuration example of the reception processing unit 402 of FIG. That is, FIG. 60 illustrates a configuration example of the reception processing unit 402 when the transmission processing unit 401 is configured as illustrated in FIG.
[0797]
The transmission data output from the entropy coding unit 505 in FIG. 55 is received by the antenna 64 (FIG. 23) of the PDA 103 as a receiving device, and is sent to the entropy decoding unit 531 and the error detection unit 540 via the communication I / F 218. It comes to be supplied.
[0798]
The entropy decoding unit 531 entropy-decodes the received data supplied thereto into a code and a difference vector, supplies the code to the vector inverse quantization unit 532 and the update unit 539, and supplies the difference vector to the addition unit 533 and The data is supplied to the update unit 539.
[0799]
The vector inverse quantization unit 532 performs vector inverse quantization on the code supplied from the entropy decoding unit 531 using the code book stored in the code book storage unit 538. That is, the vector inverse quantization unit 532 outputs a code vector corresponding to the code from the entropy decoding unit 531 among the code vectors in the code book stored in the code book storage unit 538 as a vector inverse quantization result. The code vector as the vector inverse quantization result output from the vector inverse quantization unit 532 is supplied to the addition unit 533.
[0800]
The adding unit 533 adds the difference vector supplied from the entropy decoding unit 531 and the code vector supplied from the vector inverse quantization unit 432, thereby decoding the original image vector. This image vector is supplied to the error correction unit 534.
[0801]
When the error correction unit 534 receives from the error detection unit 540 an error message indicating that there is an error such as a loss in the code or difference vector as the encoded data of the image vector, the error correction unit 534 stores the error in the code book storage unit 538. Refer to the stored codebook and correct it. Then, the error correction unit 534 supplies the image vector after the error correction to the scalarization unit 535.
[0802]
The scalarization unit 535 arranges each component of the image vector as the pixel value of the image at the original position, performs the scalarization that constitutes the original image data of one frame, and outputs the resulting image data .
[0803]
The code book database 536 stores one or more code books used for vector inverse quantization of a code obtained by vector quantization of an image vector. Note that the code book stored in the code book database 536 has the same format as that shown in FIG. 57, for example.
[0804]
Selection information is supplied to the selection unit 537, and the selection unit 537 performs vector quantization from one or more codebooks stored in the codebook database 536 according to the selection information. Select what to use. Then, the selection unit 537 reads the selected code book from the code book database 536, supplies it to the code book storage unit 538, and stores it in an overwritten form.
[0805]
Here, as the selection information, for example, user input, communication partner information, image data decoded from the code output from the entropy decoding unit 531 and the difference vector, or the like can be employed. The user input is given by the user operating the operation unit 224 (FIG. 23) or the like. The communication partner information is transmitted from the PDA 101 as the transmission device when the PDA 103 as the reception device starts communication with the PDA 101 as the transmission device, so that the antenna 64 and the communication I / F 218 (FIG. 23) are transmitted. ). Further, the code output from the entropy decoding unit 531 and the image data decoded from the difference vector are supplied from the scalarizing unit 535.
[0806]
When a user input is given as selection information, the selection unit 537 selects one code book from one or more code books stored in the code book database 536 according to the user input. Therefore, in this case, the vector inverse quantization unit 532 performs vector inverse quantization using the codebook designated by the user.
[0807]
Further, when information on the communication partner is given as selection information, the selection unit 537 selects one code book from one or more code books stored in the code book database 536 according to the information on the communication partner. . Therefore, in this case, the vector inverse quantization unit 532 performs vector inverse quantization using a codebook that differs depending on the communication partner.
[0808]
Furthermore, when image data is given as selection information, the selection unit 537 selects one code book from one or more code books stored in the code book database 536 according to the image data. Therefore, in this case, the vector dequantization unit 532 performs vector dequantization using a different codebook for each characteristic of the image data to be decoded.
[0809]
Note that the selection unit 537 selects one code book from one or more code books stored in the code book database 536 and stores it in the code book storage unit 538, that is, the vector dequantization unit 532 uses the vector. When the code book used for quantization is changed, a code book number specifying the code book after the change (the code book selected from the code book database 536) is supplied to the updating unit 539.
[0810]
The update unit 539 is configured in the same manner as the update unit 539 in FIG. 58, and updates the code book stored in the code book database 536 based on the code and the difference vector supplied from the entropy decoding unit 531. As described above, the update unit 539 receives from the selection unit 537 information that identifies the code book stored in the code book storage unit 538, that is, the code book used for the current vector inverse quantization. The update unit 539 identifies the code book to be updated based on the code book number.
[0811]
Similarly to the code book database 507 in FIG. 55, the code book database 536 uses, for example, a large amount of learning image data prepared in advance as a code book initial value, and a code obtained by the LBG algorithm or the like. The update unit 539 sequentially updates the code book of the initial value.
[0812]
Further, as a storage method for storing one or more codebooks in the codebook database 536, as in the case of the codebook database 507 in FIG. 55, first, one or more initial value codebooks are stored. Alternatively, only one initial value codebook may be stored at first, and then the initial value codebook may be copied as necessary.
[0813]
The error detection unit 540 detects the presence / absence of an error such as missing data that has occurred in the received data. If an error is detected, the error detection unit 540 outputs an error message to that effect to the error correction unit 534.
[0814]
For example, the entropy encoding unit 505 in FIG. 55 adds an error detection code (error detection code) to the transmission data, and the error detection unit 540 is based on the error detection code. Detect the presence of errors.
[0815]
In the reception processing unit 402 configured as described above, a code for selecting a codebook used for image data reception processing for receiving received data and decoding it into image data, and for vector dequantization in the vector dequantization unit 532 A book selection process and an update process for updating the stored contents of the code book database 536 are performed.
[0816]
However, the update process (data update process and code book update process) is performed in the update unit 539 using the code and the difference vector supplied from the entropy decoding unit 531 in the same manner as described in the flowchart of FIG. Therefore, the description is omitted.
[0817]
The image data reception process and codebook selection process performed by the reception processing unit 402 in FIG. 60 will be described with reference to the flowchart in FIG.
[0818]
First, the image data reception process will be described with reference to the flowchart in FIG.
[0819]
The image data reception process is started when the received data is supplied to the entropy decoding unit 531 and the error detection unit 540.
[0820]
That is, in the image data reception process, first, in step S351, the entropy decoding unit 531 entropy decodes the received data, and outputs a code and a difference vector obtained as a result. The code is supplied to the vector inverse quantization unit 532 and the updating unit 539, and the difference vector is supplied to the adding unit 533 and the updating unit 539.
[0821]
Here, the update unit 539 updates the code book in the code book database 536 as described with reference to FIG. 59 based on the code and the difference vector supplied from the entropy decoding unit 531 in this way.
[0822]
Therefore, in the transmission processing unit 401 in FIG. 55 and the reception processing unit 402 in FIG. 60, for example, a different codebook for each communication partner is selected as a codebook used for vector quantization and vector dequantization. When image data is transmitted from the PDA 101 of a certain user A to the PDA 103 of another user B, the transmission processing unit 401 in the PDA 101 of the user A has a codebook corresponding to the user B In the reception processing unit 402 of the PDA 103 of the user B, the code book corresponding to the user A is used for vector inverse quantization.
[0823]
As a result, the update unit 506 of the transmission processing unit 401 (FIG. 55) and the update unit 539 of the reception processing unit 402 (FIG. 60) each time a communication is performed between the users A and B, Will be updated in the same way. That is, between the users A and B (same between other users), basically, the same codebook is used to perform vector quantization and vector inverse quantization, respectively, The code book used in the vector quantization and the code book used in the vector inverse quantization are updated in the same way.
[0824]
As described with reference to FIG. 59, the code book is updated so as to reduce the data amount or improve the image quality. Therefore, the reception processing unit 402 in FIG. Can be accurately performed.
[0825]
In the entropy decoding unit 531, for example, when entropy decoding of image data for one frame is completed, the process proceeds to step S352, and the vector quantization unit 532 converts the code for one frame supplied from the entropy decoding unit 531 into code Vector dequantization is performed using the code book stored in the book storage unit 538, and a code vector is obtained for each code for one frame. This code vector is supplied to the adder 533.
[0826]
In step S353, the addition unit 533 adds the corresponding one of the difference vectors for one frame supplied from the entropy decoding unit 531 to each of the code vectors for one frame supplied from the vector inverse quantization unit 532. Thus, the image vector for one frame is decoded. The image vector for one frame obtained in the adder 533 is sequentially supplied to the error corrector 534.
[0827]
In step S <b> 354, the error correction unit 534 performs error correction processing, which will be described later, on the image vector supplied from the addition unit 533 and supplies the image vector to the scalarization unit 535.
[0828]
In step S355, the scalarizing unit 535 converts the image vector for one frame supplied from the error correcting unit 534 into a scalar as described above, and outputs one frame of image data obtained as a result, and then proceeds to step S356. move on.
[0829]
In step S356, the entropy decoding unit 531 determines whether or not the reception data of the next frame has been transmitted. If it is determined that the reception data of the next frame has been transmitted, the process returns to step S351 to target the reception data of the next frame. Thereafter, the same processing is repeated.
[0830]
If it is determined in step S356 that the received data of the next frame has not been transmitted, the process ends.
[0831]
Next, the code book selection process will be described with reference to the flowchart in FIG.
[0832]
Note that the code book selection process is started immediately before the image data reception process (FIG. 61A) is started, for example.
[0833]
In the code book selection process, first, in step S361, the selection unit 537 selects a default code book from one or more code books stored in the code book database 536, and stores it in the code book storage unit 538. Supply and store.
[0834]
Here, as the default codebook, for example, the above-described initial value codebook can be adopted.
[0835]
Thereafter, the process proceeds to step S362, and the selection unit 537 determines whether the selection information is supplied. If the selection unit 537 determines that the selection information is not supplied, the process skips step S363 and proceeds to step S364.
[0836]
If it is determined in step S362 that the selection information has been supplied, the process proceeds to step S363, and the selection unit 537 selects one or more codebooks stored in the codebook database 536 according to the selection information. Those used for vector quantization are selected and supplied to the codebook storage unit 538 for storage.
[0837]
In step S364, the selection unit 537 determines whether the reception of the image data by the image data reception process (FIG. 61A) started immediately after starting the current code book selection process is completed. If it is determined that the processing has not ended, the process returns to step S362, and the same processing is repeated thereafter.
[0838]
Therefore, in this case, when new selection information is supplied to the selection unit 537 while image data is being received by the image data reception process, the code book stored in the code book storage unit 538 is stored. That is, the code book used in the vector inverse quantization in the vector inverse quantization unit 532 can be changed based on the new selection information.
[0839]
On the other hand, if it is determined in step S364 that the reception of the image data has been completed, the process ends.
[0840]
In the embodiment of FIG. 61B, as described above, new selection information is supplied to the selection unit 537 while image data is being transmitted by the image data reception process. In this case, the code book used in the vector inverse quantization can be changed at any time based on the new selection information. However, the code book while the image data is being received by the image data receiving process. Can be limited to only one change. That is, the change of the code book can be limited to only the change from the default code book to the code book based on the selection information supplied first.
[0841]
Next, FIG. 62 shows a configuration example of the error correction unit 534 of FIG.
[0841]
The image vector output from the adding unit 533 is supplied to the writing unit 550, and the writing unit 550 supplies the image vector to the memory 551 for writing. The memory 551 stores the image vector supplied from the writing unit 550.
[0843]
Here, for example, the memory 551 has a storage capacity capable of storing an image vector for at least one frame.
[0844]
Further, as described above, the image vector is a vector having, for example, a pixel value of 9 pixels in a 3 × 3 pixel block as a component, but the writing unit 550 converts the image vector into a block as the image vector. The address is written in the corresponding memory 551.
[0845]
For example, when an image vector for one frame is stored in the memory 551, the reading unit 552 reads the image vector for one frame from the memory 551 and supplies the image vector to the scalarizing unit 535 (FIG. 60).
[0846]
Further, an error message is supplied from the error detection unit 540 (FIG. 60) to the reading unit 552, and the reading unit 552 controls the partial vector estimation unit 553 based on the error message.
[0847]
That is, the error message output from the error detection unit 540 includes information indicating the position of the block as an image vector in which an error has occurred, and the reading unit 552 has an image vector in which the error has occurred (hereinafter referred to as an image vector). The partial vector estimation unit 553 is controlled so as to estimate a partial vector to be described later.
[0848]
The partial vector estimation unit 553 estimates a partial vector composed of some components of the error vector as shown in FIG. 63, for example, according to the control of the reading unit 552.
[0849]
That is, in this case, as shown in FIG. 63 (A), the image vector is a vector whose component is a pixel value of 9 pixels constituting a block of 3 × 3 pixels. The fact that the obtained code or difference vector is missing means that the 3 × 3 pixels of the block as the image vector cannot be decoded as shown in FIG.
[0850]
Therefore, in the partial vector estimation unit 553, as shown in FIG. 63C, the 8 pixels adjacent to the pixels of other blocks of the block as the error vector (hereinafter referred to as an error block as appropriate) are adjacent to each other. Complement by pixels.
[0851]
That is, in FIG. 63 (C), out of the 3 × 3 pixels of the error block, the center pixel p ₉ 8 pixels p excluding ₁ Thru p ₈ Are complemented by pixels of other blocks adjacent to each other. The central pixel p ₉ 8 pixels p excluding ₁ Thru p ₈ P adjacent to only one pixel of another block ₂ , P _Four , P ₆ , P ₈ For example, the pixel values of the pixels of other blocks adjacent to each other are copied. In addition, a pixel p adjacent to two pixels of another block ₁ , P _Three , P _Five , P ₇ For example, an average value of two pixels of other blocks adjacent to each other or one of the values is copied.
[0852]
As described above, the partial vector estimation unit 553 performs the pixel p at the center of the block for the error vector. ₉ For a vector having no component corresponding to, that is, a vector originally having a pixel value of 9 pixels, the pixel p at the center of the block ₉ A partial vector having no component corresponding to is generated. Then, the partial vector estimation unit 553 outputs the partial vector as an estimated value of a part (partial vector) of the true image vector corresponding to the error vector.
[0853]
Returning to FIG. 62, the partial vector output from the partial vector estimation unit 553 is supplied to the image vector estimation unit 554.
[0854]
The image vector estimation unit 554 performs error correction by estimating a true image vector corresponding to the error vector from the partial vector from the partial vector estimation unit 553 and the code book stored in the code book storage unit 538. The estimated image vector (hereinafter referred to as an estimated vector as appropriate) is stored at a corresponding address in the memory 551.
[0855]
Next, the error correction processing performed by the error correction unit 534 in FIG. 62 in step S354 in FIG. 61A will be described with reference to the flowchart in FIG.
[0856]
In the error correction process, first, in step S371, the writing unit 550 determines whether an image vector is supplied from the adding unit 533 (FIG. 60). If it is determined in step S371 that an image vector has been supplied, the process proceeds to step S372, and the writing unit 550 supplies the image vector to the memory 551 for storage, and the process proceeds to step S375.
[0857]
If it is determined in step S371 that no image vector is supplied, the process proceeds to step S373, and the reading unit 552 determines whether an error message is supplied from the error detection unit 540.
[0858]
If it is determined in step S373 that no error message is supplied, step S374 is skipped and the process proceeds to step S375.
[0859]
If it is determined in step S373 that an error message has been supplied, the process proceeds to step S374, and the reading unit 552 determines the position of the block (image vector) in which the error has occurred (hereinafter, as appropriate) based on the error message. The error position is recognized and temporarily stored in a built-in memory (not shown).
[0860]
In step S375, the reading unit 552 determines whether an image vector (including an error vector) for one frame is stored in the memory 551.
[0861]
If it is determined in step S375 that the image vector for one frame is not stored in the memory 551, the process returns to step S371, and the same processing is repeated thereafter.
[0862]
If it is determined in step S375 that the image vector for one frame has been stored in the memory 551, the process proceeds to step S376, and the reading unit 552 determines whether the error position is stored in the built-in memory. To do.
[0863]
If it is determined in step S376 that the error position is stored, the process proceeds to step S377, and the reading unit 552 selects one of the one or more error positions stored in the built-in memory as the target error position. Then, the partial vector estimation unit 553 is controlled so as to estimate the partial vector for the error vector at the target error position. Thereby, in step S377, the partial vector estimation unit 553 estimates the partial vector and outputs it to the image vector estimation unit 554 as described with reference to FIG.
[0864]
In step S378, the image vector estimation unit 554 determines the true image vector corresponding to the error vector from the partial vector from the partial vector estimation unit 553 and the code book stored in the code book storage unit 538 (FIG. 60). The process proceeds to step S379, and the estimated vector obtained by the error correction is stored in the address corresponding to the target error position in the memory 551.
[0865]
That is, the image vector estimation unit 554 determines the code vector that the component corresponding to each component of the partial vector (here, each of the eight components as described in FIG. 63) most closely approximates each component of the partial vector, It extracts from the code book memorize | stored in the code book memory | storage part 538, and makes the code vector the estimation vector.
[0866]
Specifically, the image vector estimation unit 554 calculates, for example, the sum of squares of the difference between each component of the partial vector and the corresponding component of the code vector of the codebook, and obtains a code vector that minimizes the sum of the squares. To detect. Then, the image vector estimation unit 554 sets the code vector as an estimated vector of the image vector at the target error position, and writes it in the memory 551.
[0867]
Thereafter, the reading unit 552 erases the target error position from the built-in memory, and returns to step S376. Hereinafter, steps S376 to S379 are performed until the error position is not stored in the built-in memory. Repeat the process.
[0868]
On the other hand, when it is determined in step S376 that the error position is not stored, that is, when the image vector for one frame is stored in the memory 551 in an error-free state (error corrected state). In step S380, the reading unit 551 reads the image vector for one frame from the memory 551, supplies the image vector to the scalarizing unit 535 (FIG. 60), and ends the error correction processing.
[0869]
Note that communication for performing transmission / reception of data by performing encoding / decoding using the code book as described in FIGS. 55 to 64 is not only performed between the PDA 101 and another PDA 103 but also the base base computer 102 (see FIG. 22) and the PDA 103 or another base base computer (not shown).
[0870]
When a certain user A owns the PDA 101 and the base base computer 102 and another user B owns the PDA 103, the user A uses the PDA 101 to exchange data with the PDA 103 of the user B. Sometimes it is done using the base base computer 102.
[0871]
As described above, when data is exchanged between the PDA 101 or the base base computer 102 and the PDA 103 of the user B, a codebook for communication with the user B is generated in both the PDA 101 and the base base computer 102. .
[0872]
However, the data exchanged between the PDAs 101 and 103 and the data exchanged between the base base computer 102 and the PDA 103 are not necessarily the same, but rather are often different data. The PDA 101 and the base base computer 102 generate different codebooks as codebooks for communication with the user B.
[0873]
Therefore, as shown in FIG. 65, the code book in the PDA 101 and the code book in the base base computer 102 can be integrated into the same code book.
[0874]
That is, as shown in FIG. 65, the PDA 101 has the transmission processing unit 401 shown in FIG. 55 and the reception processing unit 402 shown in FIG. 50, and the base base computer 102 transmits the transmission processing shown in FIG. When the transmission processing unit 401 ′ having the same configuration as the unit 401 and the reception processing unit 402 ′ having the same configuration as the reception processing unit 402 in FIG. 50 are included, the PDA 101 and the base base computer 102 are as described above. As a result of the communication, the respective codebooks are exchanged. Thereby, the corresponding codebooks in the transmission processing unit 401 of the PDA 101 and the transmission processing unit 401 ′ of the base base computer 102 are integrated into the same codebook. Furthermore, the corresponding codebooks in the reception processing unit 402 of the PDA 101 and the reception processing unit 402 ′ of the base base computer 102 are also integrated into the same codebook.
[0875]
As an integration method of two code books, for example, a code vector of one code book is adopted as a code vector of an integrated code book, or an average value (average vector) of code vectors of two code books ) As the code vector of the integrated codebook.
[0876]
In the embodiment shown in FIGS. 45 to 65, the case where the image quality is improved has been described. However, the embodiment shown in FIGS. 45 to 65 is also applicable to the case where the sound quality of sound is improved. Applicable.
[0877]
Next, in the PDA 101, the quality-enhanced data for improving the quality of the data is acquired from one or more other PDAs, and based on the quality-enhanced data and the already-enhanced quality-enhanced data, New quality-enhanced data can be generated, and the existing quality-enhanced data can be updated with the new quality-enhanced data. Furthermore, the PDA 101 can process data based on the updated quality-enhanced data (new quality-enhanced data) to obtain higher-quality data.
[0878]
FIG. 66 shows a functional configuration example of such a PDA 101. In the following description, the data targeted for quality improvement will be described as audio data. However, the data targeted for high quality can be other than audio data, for example, image data.
[0879]
In FIG. 66, portions corresponding to the hardware configuration of FIG. 23 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate.
[0880]
For example, the audio decoding unit 600 decodes encoded audio data stored in the HDD 215 (FIG. 23), and supplies the decoded audio data to the sound quality improvement unit 601.
[0881]
The sound quality improvement unit 601 obtains sound quality improvement data obtained by improving the sound quality of the decoded sound data based on the high quality data supplied from the learning unit 602 and the decoded sound data, and supplies the sound quality improvement data to the D / A converter 212. To do.
[0882]
The learning unit 602 obtains quality-enhanced data by performing learning based on the audio data supplied from the amplifier 209. Further, the learning unit 602 receives learning information to be described later transmitted from one or more other PDAs, and performs learning based on the learning information as necessary, thereby improving new quality. Ask for data. Then, the learning unit 602 supplies the newly obtained high quality data to the sound quality improvement unit 601.
[0883]
Further, an operation signal is supplied to the learning unit 602 from the operation unit 224 (FIG. 23). The learning unit 602 generates and outputs a request signal for requesting learning information when the operation signal from the operation unit 224 represents an operation for requesting learning information from another PDA. This request signal is transmitted from, for example, the antenna 64 via the communication I / F 218 (FIG. 23).
[0884]
Furthermore, the learning unit 602 is supplied with request signals from other PDAs. That is, when another PDA transmits a request signal, the request signal is received by the antenna 64 (FIG. 23) and supplied to the learning unit 602 via the communication I / F 218. When the learning unit 602 receives a request signal from another PDA, the learning unit 602 transmits the learning information that the learning unit 602 has to another PDA that has transmitted the request signal via the communication I / F 218 and the antenna 64.
[0885]
Next, FIG. 67 shows a configuration example of the sound quality improvement unit 601 of FIG.
[0886]
In the embodiment of FIG. 67, the sound quality improvement unit 601 improves the sound quality of the decoded speech data from the speech decoding unit 600 by, for example, the adaptive processing described above. A tap coefficient is used as the digitized data.
[0887]
That is, the decoded audio data output from the audio decoding unit 600 is supplied to the buffer 611, and the buffer 611 temporarily stores the decoded audio data supplied thereto.
[0888]
The prediction tap extraction unit 612 sequentially uses the sound quality improvement data obtained by improving the sound quality of the decoded speech data as the attention data, and the prediction used to obtain the prediction value of the attention data by the linear primary prediction calculation of Expression (1). The tap is configured (generated) by extracting some audio samples of the decoded audio data stored in the buffer 611 and supplied to the product-sum operation unit 616.
[0889]
Note that the prediction tap extraction unit 612 configures the same prediction tap as that configured by the prediction tap extraction unit 624 of FIG. 69 described later.
[0890]
The class tap extraction unit 613 configures (generates) class taps for the data of interest by extracting some audio samples from the decoded audio data stored in the buffer 611, and supplies the class taps to the class classification unit 614. .
[0891]
The class tap extraction unit 613 constitutes the same class tap as a class tap extraction unit 625 of FIG. 69 described later.
[0892]
The class classification unit 614 performs class classification using the class tap from the class tap extraction unit 613, and supplies the class code obtained as a result to the coefficient memory 615.
[0893]
The class classification unit 614 performs the same class classification as a class classification unit 626 of FIG. 69 described later.
[0894]
The coefficient memory 615 stores the tap coefficient for each class as the quality enhancement data supplied from the learning unit 602 at an address corresponding to the class. Further, the coefficient memory 615 supplies the tap coefficient stored at the address corresponding to the class code supplied from the class classification unit 614 to the product-sum operation unit 616.
[0895]
The product-sum operation unit 616 obtains the prediction tap output from the prediction tap extraction unit 612 and the tap coefficient output from the coefficient memory 615, and uses the prediction tap and the tap coefficient to obtain the equation (1). Perform linear prediction operations. As a result, the product-sum operation unit 616 obtains (predicted value) of the sound quality improvement data as the attention data, and supplies it to the D / A converter 212 (FIG. 66).
[0896]
Next, processing (sound quality improvement processing) of the sound quality improvement unit 601 in FIG. 67 will be described with reference to the flowchart in FIG.
[0897]
The buffer 611 sequentially stores the decoded audio data output from the audio decoding unit 600 (FIG. 66).
[0898]
First, in step S401, among the sound quality improvement data in which the prediction tap extraction unit 612 has improved the sound quality of the decoded sound data, for example, the most recent sound sample that has not yet been set as the attention data in chronological order. As the attention data, by reading some sound samples of the decoded sound data from the buffer 611 for the attention data, a prediction tap is formed and supplied to the product-sum operation unit 616.
[0899]
Further, in step S401, the class tap extraction unit 613 reads out some audio samples from the decoded audio data stored in the buffer 611, thereby forming a class tap for the data of interest, and sends it to the class classification unit 614. Supply.
[0900]
Upon receiving the class tap from the class tap extraction unit 613, the class classification unit 614 proceeds to step S402, performs class classification using the class tap, and supplies the resulting class code to the coefficient memory 615. Proceed to step S403.
[0901]
In step S403, the coefficient memory 615 reads the tap coefficient stored in the address corresponding to the class code from the class classification unit 614, supplies the tap coefficient to the product-sum operation unit 616, and proceeds to step S404.
[0902]
In step S404, the product-sum operation unit 616 acquires the tap coefficient output from the coefficient memory 615, and uses the tap coefficient and the prediction tap from the prediction tap extraction unit 612 to obtain the product shown in Expression (1). Sum calculation is performed to obtain sound quality improvement data (predicted value thereof).
[0903]
The sound quality improvement data obtained as described above is supplied from the product-sum operation unit 616 to the speaker 10 and the like via the D / A converter 212 (FIG. 66) and the amplifier 208, thereby the speaker 10 and the like. Outputs high-quality sound.
[0904]
After the process of step S404, the process proceeds to step S405, and it is determined whether there is still sound quality improvement data to be processed as the attention data. If it is determined that there is, the process returns to step S401, and the same process is repeated thereafter. It is. If it is determined in step S405 that there is no sound quality improvement data to be processed as attention data, the process ends.
[0905]
Next, FIG. 69 shows a configuration example of the learning unit 602 of FIG.
[0906]
The teacher data memory 621 is supplied with voice data output from the amplifier 209 (FIG. 66) as learning data. The teacher data memory 621 serves as a learning teacher. Temporary storage as teacher data.
[0907]
The student data generation unit 622 generates student data to be a student of learning from voice data as teacher data stored in the teacher data memory 621.
[0908]
That is, the student data generation unit 622 includes a speech encoding unit 622E and a speech decoding unit 622D. The audio encoding unit 622E encodes audio data by an encoding method corresponding to the decoding method in the audio decoding unit 600 (FIG. 66), and is stored in the teacher data memory 621 by such an encoding method. The teacher data is encoded, and encoded audio data is output. The audio decoding unit 622D is configured similarly to the audio decoding unit 600, decodes the encoded audio data output from the audio encoding unit 622E, and outputs the decoded audio data obtained as a result thereof as student data.
[0909]
Here, the student data is generated by encoding the teacher data into the encoded voice data and further decoding the encoded voice data. However, the student data is, for example, the teacher data Can be generated by, for example, filtering the sound data with a low-pass filter to deteriorate the sound quality.
[0910]
The student data memory 623 temporarily stores the student data output from the audio decoding unit 622D of the student data generation unit 622.
[0911]
The prediction tap extraction unit 624 sequentially sets the voice samples of the teacher data stored in the teacher data memory 621 as the attention data, and further, some of the voice samples as the student data used for predicting the attention data. By extracting from the student data stored in the student data memory 623, a prediction tap (a tap for obtaining a predicted value of attention data) is generated. This prediction tap is supplied from the prediction tap extraction unit 624 to the addition unit 627.
[0912]
The class tap extraction unit 625 extracts several audio samples as student data used for class classification for classifying attention data from the student data stored in the student data memory 623, thereby class taps (class classification). Tap to use). This class tap is supplied from the class tap extraction unit 625 to the class classification unit 626.
[0913]
Here, as the audio samples constituting the prediction tap and the class tap, for example, a plurality of students that are close in time to the audio sample of the student data corresponding to the audio sample of the teacher data that is the attention data Audio samples as data can be used.
[0914]
In addition, as the audio samples constituting the prediction tap and the class tap, the same audio sample can be used, or different audio samples can be used.
[0915]
The class classification unit 626 classifies the data of interest based on the class tap from the class tap extraction unit 625, and outputs a class code corresponding to the class obtained as a result to the addition unit 627.
[0916]
As a class classification method in the class classification unit 626, for example, ADRC or the like can be adopted as in the case described above.
[0917]
The adding unit 627 reads out the audio sample of the teacher data as the attention data from the teacher data memory 621, and targets the student data constituting the prediction tap from the prediction tap extraction unit 624 and the teacher data as the attention data. The addition is performed for each class supplied from the class classification unit 626 while using the stored contents of the component database 630 as necessary.
[0918]
That is, the adding unit 627 basically uses a prediction tap (student data) for each class corresponding to the class code supplied from the class classifying unit 626, and uses each component in the matrix A of Equation (8). Multiplication of student data (x _in x _im ) And a calculation corresponding to summation (Σ).
[0919]
Further, the adding unit 627 uses the prediction tap (student data) and the attention data (teacher data) for each class corresponding to the class code supplied from the class classification unit 626, and uses the vector v of the equation (8). Multiplying student data and teacher data (x _in y _i ) And a calculation corresponding to summation (Σ).
[0920]
On the other hand, the component database 630 stores, for each class, the component of the matrix A and the component of the vector v in Expression (8) obtained by the previous learning in the adding unit 627.
[0921]
When learning using newly input speech data, the adding unit 627 stores the component of the matrix A in the equation (8) obtained by the previous learning and the vector v stored in the component database 630. The corresponding component x is calculated by using the teacher data and the student data obtained from the newly input speech data for the component of the matrix A or vector v. _in x _im Or x _in y _i (Addition represented by summation in matrix A and vector v) is performed to obtain new components of matrix A and vector v. For each class, the normal equation shown in equation (8) is obtained. Build up.
[0922]
Therefore, in the addition unit 627, the normal equation of Expression (8) is established based not only on newly input speech data but also on speech data used for past learning.
[0923]
As described above, the adding unit 627 uses the components of the matrix A and the vector v obtained from the newly input speech data and the components of the matrix A and the vector v stored in the component database 630, and When new components of the matrix A and the vector v are obtained for each class, these components are supplied to the component database 630 and stored in an overwritten form.
[0924]
Further, the adding unit 627 reads the components of the matrix A and the components of the vector v stored in the component database 630 as necessary, even when no new audio data is input. Using the matrix A or vector v component, the normal equation shown in equation (8) is established for each class.
[0925]
That is, the components of the matrix A and the vector v of each class in the equation (8) stored in the component database 630 are learned by using the newly input speech data in the adder 627. As described above, in addition to being updated (overwritten) by the matrix A component and the vector v component of each new class obtained during the learning, it may be updated by the integration unit 631 described later. It has become.
[0926]
As described above, when the components of the matrix A and the vector v of each class in Expression (8) stored in the component database 630 are updated by the integrating unit 631, the adding unit 627 displays the component database 630. The components of the matrix A of each class after update and the components of the vector v stored in the table are read out, and the normal equation shown in the equation (8) is obtained for each class using the matrix A or the component of the vector v. It has come to build.
[0927]
As described above, when the addition unit 627 newly creates a normal equation of the formula (8) composed of the matrix A and the vector v for each class as described above, the normal equation for each class is converted into a tap coefficient. It supplies to the determination part 628.
[0928]
The tap coefficient determination unit 628 obtains a tap coefficient for each class by solving the normal equation for each class supplied from the adding unit 627, and uses the tap coefficient for each class as the quality-enhanced data. The data is supplied to the memory 629 and stored in an overwritten form at the address corresponding to each class.
[0929]
The tap coefficient for each class as the quality-enhanced data stored in the tap coefficient memory 629 is supplied to the sound quality improvement unit 601 (FIG. 66).
[0930]
As described above, the component database 630 stores the components of the matrix A and the vector v in the equation (8) for each class.
[0931]
The integration unit 631 transmits the components of the matrix A and the vector v in the equation (8) for each class, which is transmitted from another PDA, via the antenna 64 (FIG. 23) and the communication I / F 218. Receive. The integration unit 631 uses the received components of the matrix A and the vector v of each class and the components of the matrix A and the vector v of each class stored in the component database 630, and creates a new matrix A and a new class A. Find the components of the vector v.
[0932]
That is, when the integration unit 631 receives the matrix A and the component of the vector v from each PDA, the integration unit 631 reads the component of the matrix A and the component of the vector v stored in the component database 630, and For each class matrix A or vector v component, add the corresponding class matrix A or vector v component received from another PDA (represented by summation in matrix A or vector v). For each class, a new matrix A component and a vector v component are obtained.
[0933]
Then, the integration unit 631 stores the components of the new matrix A and the vector v obtained for each class in the form of overwriting the component database 630, whereby the matrix A of each class stored in the component database 630 is stored. And update the components of vector v.
[0934]
In response to a request from the event detection unit 633, the learning information transmission unit 632 reads the components of the matrix A and the vector v of each class from the component database 630, and via the communication I / F 218 (FIG. 23) and the antenna 64, Send as learning information.
[0935]
Here, the integration unit 631 receives the components of the matrix A and the vector v of each class as learning information transmitted from the learning information transmission unit 632 in another PDA.
[0936]
When a request signal requesting transmission of learning information is transmitted from another PDA, the event detection unit 633 detects that the request signal has been transmitted as a predetermined event. That is, a request signal transmitted from another PDA is received by the communication I / F 218 via the antenna 64 (FIG. 23) and supplied to the event detection unit 633. When the event detection unit 633 detects the request signal supplied in this manner, the event detection unit 633 determines that a predetermined event has occurred and determines the components of the matrix A and the vector v of each class as learning information stored in the component database 630. The learning information transmission unit 632 is controlled to transmit.
[0937]
The request signal transmission unit 634 is supplied with an operation signal from the operation unit 224 (FIG. 23) or the like, and the request signal transmission unit 634 operates the operation unit 224 so as to request learning information. When an operation signal indicating that the operation has been performed is received, a request signal for requesting learning information is transmitted via the communication I / F 218 (FIG. 23) and the antenna 64 with the reception of the operation signal as a predetermined event.
[0938]
Here, in one or more other PDAs that have received the request signal transmitted in this way, it is assumed that a predetermined event has occurred due to the reception (detection) of the request signal, and each class of learning information that it has Transmission of the components of the matrix A and the vector v is performed. Then, the components of the matrix A and the vector v of each class as learning information transmitted from one or more other PDAs in this way are received by the integration unit 631.
[0939]
In the learning unit 602 configured as described above, learning processing for obtaining tap coefficients for each class using newly input speech data, a matrix A for each class as learning information stored in the component database 630, and A learning information transmission process for transmitting a component of vector v, and a component for updating learning information stored in the component database 630 by the matrix A of each class and the component of vector v as learning information transmitted from another PDA Data integration processing is performed.
[0940]
Then, with reference to the flowchart of FIG. 70, the learning process, learning information transmission process, and component data integration process which the learning part 602 of FIG. 69 performs are demonstrated.
[0941]
First, the learning process will be described with reference to the flowchart in FIG.
[0942]
For example, the learning process is started when new audio data of a predetermined amount or more is stored in the teacher data memory 621.
[0943]
That is, when new audio data is stored in the teacher data memory 621 in a predetermined amount or more, the adder 627 adds the matrix A and the vector v of each class stored in the component database 630 in step S411. Are read out, and the process proceeds to step S412.
[0944]
In step S412, the student data generation unit 622 reads out the teacher data using the voice data stored in the teacher data memory 621 as teacher data, and generates student data from the teacher data. Then, the student data generation unit 622 supplies the obtained student data to the student data memory 623 for storage, and the process proceeds to step S413.
[0945]
In step S413, the prediction tap extraction unit 624 uses, as attention data, one of the audio samples as teacher data stored in the teacher data memory 621 that has not yet been regarded as attention data, and the student data memory 623 for the attention data. The prediction taps are generated by reading some of the voice samples as the student data stored in, and supplied to the adding unit 627.
[0946]
Further, in step S 413, the class tap extraction unit 625 generates a class tap for the data of interest in the same manner as in the prediction tap extraction unit 624 and supplies the class tap to the class classification unit 626.
[0947]
After the processing of step S413, the process proceeds to step S414, where the class classification unit 626 performs class classification based on the class tap from the class tap extraction unit 625, and supplies the resulting class code to the addition unit 627. To do.
[0948]
In step S415, the adding unit 627 reads the attention data from the teacher data memory 621, and calculates the components of the matrix A and the vector v using the attention data and the prediction tap from the prediction tap extraction unit 624. To do. Further, the adding unit 627 calculates the matrix obtained from the attention data and the prediction tap for the component corresponding to the class code from the class classification unit 626 among the components of the matrix A and the vector v read from the component database 630. The components of A and vector v are added, thereby obtaining a new component of matrix A and vector v, and the process proceeds to step S416.
[0949]
In step S416, the prediction tap extraction unit 624 determines whether there is teacher data that has not yet been set as attention data in the teacher data memory 621. If it is determined that the teacher data is present, the process returns to step S413, and In the following, the same processing is repeated with the unsupervised teacher data as new attention data.
[0950]
If it is determined in step S416 that there is no teacher data that is not the target data in the teacher data memory 621, the process proceeds to step S417, and the adding unit 627 is obtained by repeating the processes of steps S413 to S416. The components of the new matrix A and vector v of each new class are stored in the form of being overwritten in the component database 310. Further, the adding unit 627 supplies the normal equation of the formula (8) including the components of the matrix A and the vector v of each new class to the tap coefficient determining unit 628, and the process proceeds to step S418.
[0951]
In step S418, the tap coefficient determination unit 628 obtains the tap coefficient for each class by solving the normal equation for each class supplied from the addition unit 627. Further, in step S418, the tap coefficient determination unit 628 supplies the tap coefficient for each class to the tap coefficient memory 629, stores it in an overwritten form, and ends the process.
[0952]
Next, the learning information transmission process will be described with reference to the flowchart in FIG.
[0953]
In the learning information transmission process, first, in step S431, the event detection unit 633 determines whether or not a predetermined event has occurred. If it is determined that the predetermined event has not occurred, the process returns to step S431.
[0954]
When it is determined in step S431 that a predetermined event has occurred, that is, when the event detection unit 633 receives a request signal transmitted from another PDA, the event detection unit 633 obtains learning information. The learning information transmitting unit 632 is controlled to transmit, and the process proceeds to step S432.
[0955]
In step S432, the learning information transmission unit 632 reads the components of the matrix A and the vector v of each class as learning information from the component database 630 according to the control of the event detection unit 633, and proceeds to step S433.
[0956]
In step S433, the learning information transmission unit 632 sends the components of the matrix A and the vector v of each class as learning information read from the component database 630 to the other PDAs that have transmitted the request signal received by the event detection unit 633. Send. Then, the process returns to step S431, and the same processing is repeated thereafter.
[0957]
Next, component data integration processing will be described with reference to the flowchart in FIG.
[0958]
In the component data integration process, first, in step S441, whether or not such a predetermined event has occurred, with the request signal transmission unit 634 receiving the operation signal requesting the learning information as a predetermined event. If it is determined that it has not occurred, the process returns to step S441.
[0959]
When it is determined in step S441 that a predetermined event has occurred, that is, the operation unit 224 (FIG. 23) is operated so as to request learning information by the user, If the request signal transmission unit 634 has received, the process proceeds to step S442, where the request signal transmission unit 634 transmits (for example, broadcasts) a request signal requesting learning information to any one or more PDAs.
[0960]
In step S443, the integration unit 631 determines whether learning information has been transmitted from another PDA according to the request signal transmitted in step S442.
[0961]
If it is determined in step S443 that the learning information has been transmitted, the process proceeds to step S444, where the integration unit 631 receives the learning information, temporarily stores it in its built-in memory (not shown), and performs step S445. Proceed to
[0962]
If it is determined in step S443 that the learning information has not been transmitted, the process proceeds to step S445, where the integration unit 631 transmits a request signal after the request signal transmission unit 634 transmits the request signal in step S442. Determine if it has passed.
[0964]
If it is determined in step S445 that the predetermined time has not elapsed since the transmission of the request signal, the process returns to step S443, and the same processing is repeated thereafter.
[0964]
If it is determined in step S445 that the predetermined time has elapsed since the request signal was transmitted, the process proceeds to step S446, where the integrating unit 631 determines the matrix A and the vector of each class as learning information stored in step S444. The component of v, the matrix A of each class stored in the component database 630, and the component of the vector v are added for each class to obtain a new component of the matrix A and the vector v of each class. Further, in step S446, the integration unit 631 updates the component database 630 by overwriting the components of the new matrix A and vector v of each new class, and proceeds to step S447.
[0965]
In step S447, the adding unit 627 reads the components of the matrix A and the vector v of each new class from the component database 630, builds a normal equation of Expression (8) composed of the components, and tap coefficients It supplies to the determination part 628. Further, in step S447, the tap coefficient determination unit 628 obtains a tap coefficient for each class by solving the normal equation for each class supplied from the addition unit 627, and supplies the tap coefficient to the tap coefficient memory 629. , Memorize it in the form of overwriting. Then, the process returns to step S441, and the same processing is repeated thereafter.
[0966]
As described above, the learning unit 602 performs the learning process based on the voice data used for the past learning in addition to the newly input voice data, so that the tap whose learning progressed as the user uttered A coefficient will be obtained. Therefore, the sound quality improvement unit 601 can process the decoded sound data using such a tap coefficient to obtain sound data with improved sound quality (sound quality improvement data).
[0967]
Further, in the learning unit 602, the components of the matrix A and the vector v of each class stored in the component database 630 are obtained by the components of the matrix A and the vector v of each class as learning information collected from one or more other PDAs. It will be updated. Then, the tap coefficient is obtained by solving the normal equation of Expression (8) established from the components of the matrix A and the vector v of each updated class. Therefore, the sound quality improvement unit 601 can process the decoded sound data using such a tap coefficient to obtain sound data with improved sound quality.
[0968]
Further, as described above, the learning unit 602 collects the components of the matrix A and the vector v of each class as learning information from one or more other PDAs, so that voice data with improved sound quality can be obtained. The tap coefficient can be obtained quickly.
[0969]
That is, when the normal equation of equation (8) is established using only the voice data input from the user, the tap coefficient is obtained due to the insufficient number of samples of the input voice data. As described above, the components of the matrix A and the vector v of each class as learning information are collected from one or more other PDAs as described above. In this case, for each class, it is possible to quickly obtain the components of the matrix A and the vector v for establishing the normal equation of the equation (8), and as a result, to obtain voice data with improved sound quality. The tap coefficient can be obtained quickly.
[0970]
In addition, as described above, by exchanging the components of the matrix A and the vector v of each class as learning information that each PDA has, the plurality of PDAs have the same sound quality and high quality. Sound data with sound quality can be obtained.
[0971]
In the embodiment of FIGS. 69 and 70, the request signal transmission unit 634 transmits a request signal when an operation signal is received. It is possible to transmit at an arbitrary timing periodically.
[0972]
In addition, in the PDA 101 of FIG. 66, the components of the matrix A and the vector v of each class obtained in the course of learning are used as learning information and exchanged with one or more other PDAs. It is also possible to exchange the tap coefficient itself as learning information.
[0973]
When the tap coefficient is exchanged with one or more other PDAs in the PDA 101, the tap coefficient is updated by, for example, the tap coefficient owned by itself and the tap coefficient received from another PDA. Can be performed by weighted addition.
[0974]
Note that as the weight in this case, the number of audio samples used to obtain the tap coefficient can be used.
[0975]
That is, in order to obtain the tap coefficient, the teacher data y obtained from the voice data _i And student data x _in X, which is a component of matrix A, computed using _in x _im And x which is a component of the vector v _in y _i However, it is possible to use the number of times of addition (which is equal to the number of voice samples that became teacher data) as a weight.
[0976]
In this case, for example, the tap coefficient of the PDA 101 is set to w _a = {W _a1 , W _a2 , W _a3 ,...}, The number of audio samples used to obtain the tap coefficient, α, and the tap coefficient received from another PDA, w. _b = {W _b1 , W _b2 , W _b3 ,..., And β, which is the number of audio samples used to determine the tap coefficient, the new tap coefficient is expressed by the equation (αw _a + Βw _b ) / (Α + β).
[0977]
Further, in the case where the tap coefficient is exchanged with one or more other PDAs in the PDA 101, the tap coefficient is updated by, for example, replacing the tap coefficient of a class that does not have the tap coefficient with another tap coefficient. This can be done by complementing with the tap coefficients received from the PDA.
[0978]
That is, when the PDA 101 uses only the voice data input from the user as the normal equation of the formula (8), the number of samples of the input voice data is not sufficient. There may be a class that does not obtain the number of normal equations required to determine the tap coefficient, but other users' PDAs may have a sufficient number of normal equations for that class. obtain.
[0979]
Therefore, in the PDA 101, by exchanging tap coefficients with other PDAs, it is possible to complement the tap coefficients of a class that cannot be obtained by itself with those obtained by other PDAs. It is.
[0980]
The case where the present invention is applied to a PDA has been described above, but the present invention can also be applied to an information processing apparatus other than a PDA.
[0981]
The series of processes described above can be performed by hardware or software.
[0982]
When a series of processing is performed by software, a program constituting the software is installed in the PDA 101 or the base base computer 102.
[0983]
The program can be recorded in advance in an HDD 215 (FIG. 23) or an HDD 333 (FIG. 26) as a recording medium built in the computer.
[0984]
Alternatively, the program is temporarily or permanently stored on a removable recording medium such as a flexible disk, CD-ROM (Compact Disc Read Only Memory), MO (Magneto Optical) disk, DVD (Digital Versatile Disc), magnetic disk, or semiconductor memory. Can be stored (recorded). Such a removable recording medium can be provided as so-called package software.
[0985]
The program is installed on the PDA 101 and the base base computer 102 from the above-described removable recording medium, and is also wirelessly transferred from the download site to the PDA 101 and the base base computer 102 via a digital satellite broadcasting artificial satellite. Or transferred to the PDA 101 or the base base computer 102 via a network such as a LAN (Local Area Network) or the Internet. The PDA 101 or the base base computer 102 transmits the program transferred in this way to the communication unit 108. Can be received and installed at.
[0986]
Here, in this specification, the processing steps describing the program for causing the CPU 202 (FIG. 23) of the PDA 101 and the CPU 312 (FIG. 26) of the base base computer 102 to perform various processes are not necessarily described as flowcharts. It is not necessary to perform processing in time series according to the order, and includes processing executed in parallel or individually (for example, parallel processing or object processing).
[0987]
The program may be processed by one CPU, or may be distributedly processed by a plurality of CPUs. Furthermore, the program may be transferred to a remote computer or the like and executed.
[0988]
【The invention's effect】
As described above, according to the display device, the display method, and the program of the present invention, one piece of information is displayed for each of the plurality of display units, and any of the information displayed on each of the plurality of display units is selected. It is detected whether it was done. Each of the plurality of pieces of information associated with the selected information is displayed on each of the plurality of display means. Accordingly, easy-to-understand information display and accurate operation are possible.
[Brief description of the drawings]
FIG. 1 is a perspective view showing an external configuration example of a first embodiment of a PDA 101 to which the present invention is applied.
2 is a perspective view showing a first external configuration example of a main block 2 of a PDA 101. FIG.
FIG. 3 is a perspective view showing an external configuration example of the main block 2 in a state where a lid 20 is opened.
FIG. 4 is a perspective view showing an external configuration example of a PDA 101 in a state where a lid 20 and sub panels 15 and 16 are opened.
5 is a perspective view showing a state of wiring in the PDA 101. FIG.
FIG. 6 is a perspective view illustrating a configuration example of a hinge unit 13;
7 is a perspective view showing a second external configuration example of the main block 2 of the PDA 101. FIG.
8 is a top view and a side view showing a second external configuration example of the main block 2. FIG.
FIG. 9 is a perspective view showing an external configuration example of the main block 2 in a state where a lid portion 20 is opened.
10 is a perspective view showing an external configuration example of a PDA 101 in a state in which a lid 20 and sub-panels 15 and 16 are opened. FIG.
FIG. 11 is a perspective view showing a state in which a hinge part 71 is disassembled.
FIG. 12 is a perspective view showing more details in a state where the hinge part 71 is disassembled.
13 is a perspective view showing a third external configuration example of the main block 2 of the PDA 101. FIG.
14 is a cross-sectional view showing a state of wiring in the main block 2. FIG.
15 is a cross-sectional view showing a state of wiring in the main block 2. FIG.
FIG. 16 is a perspective view showing an example of the external configuration of a second embodiment of a PDA 101 to which the present invention is applied.
17 is a plan view showing a configuration example of a second embodiment of the PDA 101. FIG.
18 is a perspective view showing a fourth external configuration example of the main block 2 of the PDA 101. FIG.
FIG. 19 is a perspective view for explaining a state of wiring in the main block 2;
20 is a perspective view showing a fifth external configuration example of the main block 2 of the PDA 101. FIG.
21 is a perspective view showing a sixth external configuration example of the main block 2 of the PDA 101. FIG.
22 is a diagram illustrating a configuration example of an embodiment of a PDA system using the PDA 101. FIG.
23 is a block diagram illustrating a hardware configuration example of the PDA 101. FIG.
24 is a block diagram showing a first functional configuration example of a PDA 101. FIG.
25 is a perspective view showing an example of an external configuration of a base base computer 102. FIG.
26 is a block diagram illustrating a hardware configuration example of the base base computer 102. FIG.
27 is a block diagram illustrating a functional configuration example of a base base computer 102. FIG.
FIG. 28 is a block diagram showing a state where the PDA 101 and the base base computer 102 are connected.
FIG. 29 is a flowchart illustrating call processing and call processing performed by the PDA 101.
30 is a flowchart for explaining mail transmission / reception processing and mail transmission event processing performed by the PDA 101. FIG.
FIG. 31 is a flowchart for explaining data transmission / reception processing and data transmission event processing performed by the PDA 101;
32 is a flowchart for explaining data reproduction processing performed by the PDA 101. FIG.
FIG. 33 is a flowchart illustrating streaming playback processing performed by the PDA 101.
34 is a diagram illustrating a communication procedure when a file is transmitted from the PDA 101 to the base base computer 102. FIG.
35 is a flowchart illustrating a communication procedure when a file is transmitted from the base base computer 102 to the PDA 101. FIG.
FIG. 36 LCD4 ₁ , 5 ₁ , 12, 21 to 23 are diagrams showing display examples of screens.
FIG. 37 LCD4 ₁ , 5 ₁ , 12, 21 to 23 are diagrams showing display examples of screens.
FIG. 38 LCD4 ₁ , 5 ₁ , 12, 21 to 23 are diagrams showing display examples of screens.
FIG. 39 LCD4 ₁ Thru 4 _Four , 5 ₁ 5 _Four , 12, 21 to 23 are diagrams showing display examples of screens.
FIG. 40 is a flowchart illustrating an on / off control process for LCDs 3, 12, 21 to 23.
FIG. 41 is a flowchart illustrating a hierarchy display control process.
FIG. 42 is a diagram illustrating a format of information structured in a hierarchical structure.
43 is a diagram showing a state in which a PDA 101 is communicating with another PDA 103. FIG.
44 is a block diagram showing a second functional configuration example of the PDA 101. FIG.
45 is a block diagram illustrating a first configuration example of a transmission processing unit 401. FIG.
FIG. 46 is a flowchart for describing image data transmission processing, class code generation processing, and class code transmission processing performed by the transmission processing unit 401;
47 is a block diagram illustrating a first configuration example of a reception processing unit 402. FIG.
48 is a block diagram illustrating a configuration example of an adaptive processing unit 447. FIG.
FIG. 49 is a flowchart for describing image data reception processing, request signal transmission processing, and adaptation processing performed by the reception processing unit 402;
50 is a block diagram illustrating a second configuration example of the transmission processing unit 401. FIG.
51 is a block diagram illustrating a configuration example of a learning unit 414. FIG.
FIG. 52 is a flowchart for describing learning processing and tap coefficient transmission processing performed by the transmission processing unit 401;
53 is a block diagram illustrating a second configuration example of the reception processing unit 402. FIG.
54 is a block diagram illustrating a configuration example of an adaptive processing unit 448. FIG.
55 is a block diagram illustrating a third configuration example of the transmission processing unit 401. FIG.
FIG. 56 is a flowchart for describing image data transmission processing and codebook selection processing performed by the transmission processing unit 401;
Fig. 57 is a diagram illustrating a format of a code book.
58 is a block diagram illustrating a configuration example of an update unit 506 (539). FIG.
FIG. 59 is a flowchart for describing data update processing and codebook update processing performed by the update unit 506;
60 is a block diagram illustrating a third configuration example of the reception processing unit 402. FIG.
FIG. 61 is a flowchart for describing image data reception processing and codebook selection processing performed by the reception processing unit 402;
62 is a block diagram illustrating a configuration example of an error correction unit 534. FIG.
Fig. 63 is a diagram for describing processing of a partial vector estimation unit 553;
FIG. 64 is a flowchart for describing error correction processing;
FIG. 65 is a diagram for explaining codebook integration performed between the PDA 101 and the base computer 102;
66 is a block diagram showing a third functional configuration example of the PDA 101. FIG.
67 is a block diagram illustrating a configuration example of a sound quality improvement unit 601. FIG.
FIG. 68 is a flowchart illustrating sound quality improvement processing.
69 is a block diagram illustrating a configuration example of a learning unit 602. FIG.
FIG. 70 is a flowchart for describing learning processing, learning information transmission processing, and component data integration processing performed in a learning unit 602;
[Explanation of symbols]
1 watch band, 2 main block, 3 LCD, 3A touch panel, 4 side panel, 4 ₁ Thru 4 _Four LCD, 4A ₁ To 4A _Four Touch panel, 5 side panel, 5 ₁ 5 _Four LCD, 5A ₁ To 5A _Four Touch panel, 6 Jog dial, 7 Earphone microphone jack, 8 Connector part, 8 ₁ 8 _Four Connector, 9 microphone, 10 speaker, 11 main body, 12 LCD, 12A touch panel, 13 hinge part, 14 main panel, 15, 16 sub-panel, 17, 18 hinge part, 20 lid part, 21 LCD, 21A touch panel, 22 LCD, 22A Touch panel, 23 LCD, 23A touch panel, 31, 32 axis, 33, 34 bearing part, 35 to 40 through hole, 41 to 43 circuit block, 51 locking part, 52 groove, 61 hold switch, 62 power switch, 63 wireless communication Part, 64 antenna, 65 CCD camera, 71 to 73 hinge part, 81 hinge fitting, 82 axis, 83 hinge fitting, 91, 92 hinge cover, 101 PDA, 102 base base computer, 103 PDA, 104 public network, 105 Internet 111, circuit board, 112, 113 flexible, 114, 115 holes, 131 arm part, 131A, 131B pin part, 132 arm part, 132A, 132B pin part, 133 arm part, 133A, 133B pin part, 143 locking Part, 144 holes, 145 to 147 flexible, 201 bus, 202 CPU, 203 ROM, 204 RAM, 205 (205 ₁ To 205 ₇ Touch panel driver, 206 (206 ₁ To 206 ₇ LCD driver, 207 to 209 amplifier, 210, 211 A / D converter, 212 D / A converter, 213 MPEG encoder / decoder, 214 ATRAC encoder / decoder, 215 HDD, 216 Flash memory, 217 DRAM, 218 Communication I / F, 219 driver, 220 light receiving unit, 221 light emitting unit, 222 battery, 223 input I / F, 224 operation unit, 225 switch unit, 226 bus, 231 RF processing unit, 232 channel decoding unit, 233 channel encoding unit, 234 encoding / decoding unit, 235 image encoding unit, 236 image decoding unit, 237 audio encoding unit, 238 audio decoding unit, 239 control unit, 240 modulation / demodulation unit, 241 memory, 242 alarm unit, 243 open / close detection unit, 244 display Control unit, 245 display unit, 251 to 255 display control unit, 256 to 261 display unit, 301 keyboard, 302 display unit, 303 PDA mounting unit, 304 wireless communication unit, 305 IEEE1394 terminal, 306 USB terminal, 311 bus, 312 CPU, 313 ROM , 314 RAM, 315 flash memory, 316 LCD, 317 LCD driver, 318 keyboard I / F, 319 USB interface, 320 IEEE1394 interface, 321 PCMCIAA driver, 322 PCMCIA slot, 323 ATRAC encoder / decoder, 324 A / D converter, 325 D / A converter, 326,327 amplifier, 328 microphone, 329 speaker, 330 microphone jack, 331 earphone jack, 332 CD-RW drive, 333 HDD, 334 flash Memory, 335 DRAM, 336 communication I / F, 337 connector, 337 ₁ Thru 337 _Four Connector, 338 driver, 339 light receiving unit, 340 light emitting unit, 341 LAN board, 342 modem / TA / DSU, 343 bus, 344 charging circuit, 345 bus, 351 control unit, 352 audio image encoding / decoding unit, 353 operation unit, 354 modulation / demodulation unit, 355 display control unit, 356 display unit, 357 memory, 401, 401 ′ transmission processing unit, 402, 402 ′ reception processing unit, 411 encoding unit, 412 event detection unit, 413 storage, 414 learning unit, 415 Tap coefficient buffer, 416 storage, 421 data compression unit, 422 transmission control unit, 423 class tap extraction unit, 424 class classification unit, 425 class code database, 431 decoding unit, 432 image quality determination unit, 433 request signal transmission unit, 441 reception Control unit, 44 2 reception buffer, 443 registration unit, 444 storage, 445 selection unit, 446 class code database, 447, 448 adaptive processing unit, 451 buffer, 452 prediction tap extraction unit, 453 product-sum operation unit, 454 class code reading unit, 455 coefficient Memory, 456 registration unit, 461 teacher data memory, 462 data compression unit, 463 student data memory, 464 prediction tap extraction unit, 465 class tap extraction unit, 466 class classification unit, 467 addition unit, 468 tap coefficient determination unit, 501 Vectorization unit, 502 Vector quantization unit, 503 Local decoding unit, 504 Difference operation unit, 505 Entropy coding unit, 506 Update unit, 507 Codebook database, 508 selection unit, 509 Codebook storage unit, 521 Data update unit, 522 code book update unit, 531 entropy decoding unit, 532 vector inverse quantization unit, 533 addition unit, 534 error correction unit, 535 scalarization unit, 536 code book database, 537 selection unit, 538 code book storage unit , 539 updating unit, 540 error detection unit, 550 writing unit, 551 memory, 552 reading unit, 553 partial vector estimation unit, 554 image vector estimation unit, 600 speech decoding unit, 601 sound quality improvement unit, 602 learning unit, 611 buffer, 612 prediction tap extraction unit, 613 class tap extraction unit, 614 class classification unit, 615 coefficient memory, 616 product-sum operation unit, 621 teacher data memory, 622 student data generation unit, 622D speech decoding unit, 622E speech encoding unit, 623 Student day Data memory, 624 prediction tap extraction unit, 625 class tap extraction unit, 626 class classification unit, 627 addition unit, 628 tap coefficient determination unit, 629 tap coefficient memory, 630 component database, 631 integration unit, 632 learning information transmission unit, 633 Event detection unit, 634 Request signal transmission unit

Claims (38)

A display device having a plurality of display means for displaying information,
The body,
A lid portion movably attached to the main body;
A panel movably attached to the body or lid,
First as one of the plurality of display means provided in the main body, stored when the lid portion is moved to the first position, and exposed when the lid portion is moved to the second position. Display means,
A second display means as another one of the plurality of display means provided on the lid and exposed on the upper surface when the lid is moved to the second position;
A third display means as another one of the plurality of display means provided on the panel and exposed on the upper surface when the panel is moved to a predetermined position;
Display control means for performing control to display one information on each of the plurality of display means;
Detecting means for detecting which of the information displayed on each of the plurality of display means is selected;
Lid state detection means for detecting the state of the lid;
Panel state detecting means for detecting the state of the panel, and
The display control unit displays each of a plurality of pieces of information associated with the selected information on each of the plurality of display units, and the state of the lid unit detected by the lid unit state detection unit and Depending on the state of the panel detected by the panel state detection means, the on / off of the plurality of display means is controlled, and when a plurality of menu items are displayed on the plurality of display means, according to user selection A directory display for displaying the display unit and the menu item in a one-to-one correspondence, and a multi- display for displaying the menu item over the plurality of display units without a one-to-one correspondence between the display unit and the menu item. A display device characterized by controlling switching to a screen display. The other panel which has a 4th display means which is attached to the said main body or a cover part so that movement is possible, and is exposed to an upper surface when it moves to a predetermined | prescribed position is characterized by the above-mentioned. Display device. The lid is
It is pivotally attached to one end of the main body,
When the first display device is closed when moved to the first position, the first and second display means are accommodated so that the first and second display means face each other,
2. The display device according to claim 1, wherein the display device is opened when moved to the second position and exposes the first and second display means. 3. The display device according to claim 3, wherein the first display unit is provided on an upper surface of the main body so as to be exposed when the lid portion is in an open state. The display device according to claim 3, wherein the second display unit is provided on a back surface of the lid portion when the lid portion is in a closed state. The panel is
It is pivotally attached to one end of the main body or lid,
The display device according to claim 1, wherein the display device is rotatable in a state where the lid portion is moved to the second position. The display device according to claim 6, wherein the third display unit is provided on a back surface of the panel when the panel is in a closed state. The panel is
It is pivotally attached to one end of the lid,
By rotating to the open state, the third display means is exposed,
The display device according to claim 6, wherein the third display unit is accommodated by being rotated to a closed state. The display device according to claim 8, wherein the second display means is exposed when the lid portion is moved to a second position and the panel is opened. The lid is attached to the main body through a hinge mechanism so as to be freely rotatable.
The hinge mechanism is
A first hinge fitting fixed to the body;
A second hinge fitting fixed to the lid,
The first and second hinge brackets have shafts on which both ends are press-fitted,
The display device according to claim 1, wherein the first hinge fitting and the shaft and the second hinge fitting and the shaft are rotatable. The display device according to claim 10, wherein the first and second hinge fittings are formed of curled spring steel plates. The panel is rotatably attached to the lid portion via a hinge mechanism.
The hinge mechanism is
A first hinge fitting fixed to the body;
A second hinge fitting fixed to the lid,
The first and second hinge brackets have shafts on which both ends are press-fitted,
The display device according to claim 1, wherein the first hinge fitting and the shaft and the second hinge fitting and the shaft are rotatable. The display device according to claim 12, wherein the first and second hinge fittings are formed of a curled spring steel plate. The display device according to claim 1, wherein the display control unit also performs control to display one piece of information on the whole of the plurality of display units. A portable device,
It is further equipped with a band for wearing on the arm,
The display device according to claim 1, wherein the band is provided with one or more display means. The display device according to claim 1, further comprising a communication unit that communicates with a communicable communication device. The display device according to claim 16, further comprising destination input means for inputting a destination of a communication partner. The display device according to claim 16, wherein the communication unit functions as a telephone for performing a voice call. Voice input means for inputting voice to be transmitted in the voice call;
The display device according to claim 18, further comprising voice output means for outputting voice received in the voice call. The display device according to claim 16, wherein the communication unit transmits and receives data. The display device according to claim 20, further comprising transmission / reception data selection means for selecting data to be transmitted or received. The display device according to claim 20, wherein the communication unit transmits and receives image data or audio data. The display device according to claim 20, wherein the communication unit transmits and receives an electronic mail. The display device according to claim 23, further comprising message input means for inputting the e-mail message. The display device according to claim 1, further comprising data storage means for storing data. The display device according to claim 25, wherein the data storage means stores image data or audio data. 26. The display device according to claim 25, further comprising reproduction means for reproducing data stored in the data storage means. The display device according to claim 27, further comprising reproduction data selection means for selecting data to be reproduced. The data storage means stores a program to be executed by a computer,
The display device according to claim 25, further comprising execution means for executing the program. The display device according to claim 16, wherein the communication unit performs communication in contact with the communication device or in a non-contact manner. The battery is further provided as a power source,
The display device according to claim 16, wherein the communication device charges the battery when the communication unit and the communication device communicate with each other. The display device according to claim 16, further comprising an authentication unit configured to perform authentication with the communication device. It is configured to be detachable from the communication device,
The display device according to claim 16, wherein the display device functions as a part of the communication device when attached to the communication device. A memory for storing data;
The display device according to claim 33, wherein the memory functions as a part of a memory of the communication device when the display device is attached to the communication device. The display device according to claim 1, wherein the display device is a portable terminal. A display device having a plurality of display means for displaying information, a main body, a lid portion movably attached to the main body, and a panel movably attached to the main body or the lid portion. , Provided as one of the plurality of display means, which is provided in the main body, is accommodated when the lid portion is moved to the first position, and is exposed when the lid portion is moved to the second position. One display means and a second display as another one of the plurality of display means provided on the lid portion and exposed on an upper surface when the lid portion moves to the second position And a third display means as another one of the plurality of display means provided on the panel and exposed on the upper surface when the panel moves to a predetermined position Display method,
A display control step for performing control to display one piece of information on each of the plurality of display means;
A detection step of detecting which of the information displayed on each of the plurality of display means is selected;
A lid state detecting step for detecting the state of the lid;
A panel state detecting step for detecting the state of the panel, and
In the display control step, each of a plurality of pieces of information associated with the selected information is displayed on each of the plurality of display units, and the plurality of pieces of information are displayed according to the state of the lid and the state of the panel. Directory for controlling on / off of display means, and further displaying the display means and the menu items in a one-to-one correspondence according to user selection when displaying a plurality of menu items on the plurality of display means Switching between display and multi-screen display in which the menu items are displayed across the plurality of display means without causing the display means and the menu items to correspond one-to-one . A display device having a plurality of display means for displaying information, a main body, a lid portion movably attached to the main body, and a panel movably attached to the main body or the lid portion. , Provided as one of the plurality of display means, which is provided in the main body, is accommodated when the lid portion is moved to the first position, and is exposed when the lid portion is moved to the second position. One display means and a second display as another one of the plurality of display means provided on the lid portion and exposed on an upper surface when the lid portion moves to the second position And a third display means as another one of the plurality of display means provided on the panel and exposed on the upper surface when the panel moves to a predetermined position A computer that controls the computer A lamb,
A display control step for performing control to display one piece of information on each of the plurality of display means;
A detection step of detecting which of the information displayed on each of the plurality of display means is selected;
A lid state detecting step for detecting the state of the lid;
A panel state detecting step for detecting the state of the panel, and
In the display control step, each of a plurality of pieces of information associated with the selected information is displayed on each of the plurality of display units, and the plurality of pieces of information are displayed according to the state of the lid and the state of the panel. Directory for controlling on / off of display means, and further displaying the display means and the menu items in a one-to-one correspondence according to user selection when displaying a plurality of menu items on the plurality of display means A program for controlling switching between display and multi-screen display in which the menu items are displayed across the plurality of display means without causing the display means and the menu items to correspond one-to-one . A display device having a plurality of display means for displaying information, a main body, a lid portion movably attached to the main body, and a panel movably attached to the main body or the lid portion. , Provided as one of the plurality of display means, which is provided in the main body, is accommodated when the lid portion is moved to the first position, and is exposed when the lid portion is moved to the second position. One display means and a second display as another one of the plurality of display means provided on the lid portion and exposed on an upper surface when the lid portion moves to the second position And a third display means as another one of the plurality of display means provided on the panel and exposed on the upper surface when the panel moves to a predetermined position A computer that controls the computer A recording medium in which the ram is recorded,
A display control step for performing control to display one piece of information on each of the plurality of display means;
A detection step of detecting which of the information displayed on each of the plurality of display means is selected;
A lid state detecting step for detecting the state of the lid;
A panel state detecting step for detecting the state of the panel, and
In the display control step, each of a plurality of pieces of information associated with the selected information is displayed on each of the plurality of display units, and the plurality of pieces of information are displayed according to the state of the lid and the state of the panel. Directory for controlling on / off of display means, and further displaying the display means and the menu items in a one-to-one correspondence according to user selection when displaying a plurality of menu items on the plurality of display means A program for controlling switching between display and multi-screen display for displaying the menu items across the plurality of display means without causing the display means and the menu items to correspond one-to-one is recorded. recoding media.

Images