JPH05280953A - Method for recoginizing object - Google Patents

Method for recoginizing object

Info

Publication number
JPH05280953A
JPH05280953A JP4075055A JP7505592A JPH05280953A JP H05280953 A JPH05280953 A JP H05280953A JP 4075055 A JP4075055 A JP 4075055A JP 7505592 A JP7505592 A JP 7505592A JP H05280953 A JPH05280953 A JP H05280953A
Authority
JP
Japan
Prior art keywords
dimensional
neural network
input
model
input layer
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
JP4075055A
Other languages
Japanese (ja)
Inventor
Mitsuhiro Sakai
光宏 坂井
Atsushi Inagawa
淳 稲川
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
N T T DATA TSUSHIN KK
Original Assignee
N T T DATA TSUSHIN KK
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by N T T DATA TSUSHIN KK filed Critical N T T DATA TSUSHIN KK
Priority to JP4075055A priority Critical patent/JPH05280953A/en
Publication of JPH05280953A publication Critical patent/JPH05280953A/en
Pending legal-status Critical Current

Links

Landscapes

  • Length Measuring Devices By Optical Means (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

PURPOSE:To discriminate and recognize even an object having a complicated three-dimensional shape whose restraint conditions as to how it looks like are difficult to describe without being affected by difference in a way it looks due to rotation, movement, etc., which has been a problem in matching between two-dimensional patterns. CONSTITUTION:Each unit of an input layer of a back-propagation type neural network 5 is made to correspond with a set of three-dimensionally arranged points, a three-dimensional shape itself of an object 1 to be recognized is learned, and a three-dimensional model 2 created by using three-dimensional information on points on an object surface obtained by a three-dimensional measuring device is input to the network. Thus three-dimensional patterns are matched with each other.

Description

【発明の詳細な説明】Detailed Description of the Invention

【0001】[0001]

【産業上の利用分野】本発明は、バックプロパゲ−ショ
ン型のニュ−ラルネットワ−クを用いた物体認識方法に
関する。
BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an object recognition method using a back propagation type neural network.

【0002】[0002]

【従来の技術】従来、透明な物体や金属等の物体の3次
元形状を認識するため、超音波による計測を行っていた
が、超音波で得られる映像は解像度が低かったので、次
に音響映像再生法とニュ−ラルネットワ−クを組み合わ
せた方法により、物体を認識する方法が提案されている
(例えば、『日経コンピュ−タ』1991.2.25、pp.77〜94
参照)。この学習では、モデルを超音波映像として取り
込み、1つのニュ−ラルネットワ−クで成形させ、別の
ニュ−ラルネットワ−クで判別させることを繰り返し行
う。また、実際の処理では、学習させた2つのニュ−ラ
ルネットワ−クを使用して認識させている。さらに、ニ
ュ−ラルネットワ−クをパタ−ン認識に応用する研究も
進んでおり、1976年に発表されたニュ−ラルネットの学
習アルゴリズムであるバックプロパゲ−ションでは、入
力と望ましい出力を与えていくことにより、自動的にニ
ュ−ラルネットを学習させている。すなわち、入力デ−
タは入力層、中間層、出力層の階層構造を有する各ユニ
ットで順次、変換されて次のユニットに伝わり、出力層
から出力が得られる。その出力値と望ましい出力値とを
比べて、その差を減らすように結合の強さを変える。そ
の結合の強さの計算は、出力層のユニットから始めて中
間層のユニットに移るが、中間層ユニットではその前段
の結合の強さが決まらなければ計算できない。従って、
最後の入力層まで遡及して初めて計算が可能となる。こ
のように、学習は入力デ−タの処理とは逆方向、つまり
後ろ向きに進む。バックプロパゲ−ションによる学習
は、先ず学習用デ−タを入力し、その結果を出力する。
結果のエラ−を減らすように、後ろ向きに結合の強さを
変える。再び、学習用デ−タを入力して、その結果を出
力する。この動作を収束するまで繰り返すのである(例
えば、『日経エレクトロニクス』8−10,1987.no.42
7,pp.115〜126、“ニュ−ラルネットをパタ−ン認識,
信号処理,知識処理に使う”参照)。
2. Description of the Related Art Conventionally, ultrasonic waves are used for recognizing a three-dimensional shape of a transparent object or an object such as a metal. However, since an image obtained by ultrasonic waves has a low resolution, the sound is next detected. A method of recognizing an object by a method combining a video reproducing method and a neural network has been proposed (for example, "Nikkei Computer" 1991.2.25, pp.77-94).
reference). In this learning, a model is taken in as an ultrasonic image, shaped by one neural network, and discriminated by another neural network. In the actual processing, the learned two neural networks are used for recognition. Furthermore, research is being conducted to apply the neural network to pattern recognition. In backpropagation, which is a neural network learning algorithm announced in 1976, the input and the desired output are given. , Is automatically learning the neural net. That is, the input data
The data is sequentially converted by each unit having a hierarchical structure of an input layer, an intermediate layer, and an output layer and transmitted to the next unit, and an output is obtained from the output layer. The output value is compared with the desired output value and the strength of the coupling is changed so as to reduce the difference. The calculation of the bond strength starts from the unit of the output layer and moves to the unit of the middle layer, but the unit of the middle layer cannot be calculated unless the strength of the connection of the preceding stage is determined. Therefore,
The calculation can be done only by going back to the last input layer. In this way, learning proceeds in the opposite direction to the processing of input data, that is, backward. In learning by back propagation, learning data is first input and the result is output.
Varying the bond strength backwards to reduce the resulting error. The learning data is input again and the result is output. This operation is repeated until it converges (for example, "Nikkei Electronics" 8-10, 1987. no. 42).
7, pp.115-126, “Pattern recognition of neural nets,
Use for signal processing and knowledge processing ”).

【0003】[0003]

【発明が解決しようとする課題】一般に、ニュ−ラルネ
ットワ−クを使用した従来の物体認識方法においては、
入力のパタ−ンを2次元に限定していた。また、3次元
形状をニュ−ラルネットワ−クにより表現するために
は、3次元パタ−ン自体ではなく、その見え方に関する
拘束条件をニュ−ラルネットワ−クのユニット間の結合
で表わしていた。なお、関連技術の従来文献としては、
例えば、“『Parallel DistributedProcessing』
D.E.Rumelhart,J.Mc-Clelland"がある。とこ
ろで、上述した従来のバックプロパゲ−ション型ニュ−
ラルネットワ−クによる物体認識方法では、画像から得
られる2次元情報を入力として、物体を3次元的ではな
く、固定された一方向からの2次元パタ−ンとしてしか
学習していなかった。そのため、実画像中に含まれてい
る物体の回転、移動等による見え方の変化に対応できな
いという問題があった。この問題点を解決するために
は、物体を多方向から見た2次元パタ−ンが必要であ
り、かつ大規模なニュ−ラルネットワ−ク群を用意する
必要があった。また、上述した3次元形状の拘束条件を
ニュ−ラルネットワ−クにより表現では、ネットワ−ク
内のユニット間の接続を学習ではなく、作成者が自力で
決定する必要があり、その上に複雑な形状の物体に関し
ては、拘束条件が表現しきれなくなってしまうという問
題があった。本発明の目的は、これら従来の課題を解決
し、物体の見え方に煩わされることなく、複雑な形状の
物体の認識を行うことが可能な物体認識方法を提供する
ことにある。
Generally, in the conventional object recognition method using the neural network,
The input pattern was limited to two dimensions. Further, in order to express a three-dimensional shape by a neural network, the constraint condition regarding the appearance of the three-dimensional pattern is expressed by the coupling between the units of the neural network, not by the three-dimensional pattern itself. In addition, as a related art document,
For example, "" Parallel Distributed Processing "
D. E. Rumelhart, J.M. Mc-Clellland ". By the way, the above-mentioned conventional back-propagation type news
In the object recognition method using the Lar network, two-dimensional information obtained from an image is input, and the object is learned not as a three-dimensional object but as a two-dimensional pattern from a fixed one direction. Therefore, there is a problem that it is not possible to cope with a change in appearance due to rotation, movement, etc. of an object included in the actual image. In order to solve this problem, a two-dimensional pattern in which an object is viewed from multiple directions is required, and it is necessary to prepare a large-scale neural network group. Further, in expressing the constraint condition of the above-mentioned three-dimensional shape by the neural network, it is necessary for the creator to determine the connection between the units in the network by himself rather than learning, and in addition, it is complicated. There is a problem that the constraint condition cannot be expressed for the object having a shape. An object of the present invention is to solve these conventional problems and provide an object recognition method capable of recognizing an object having a complicated shape without being bothered by how the object looks.

【0004】[0004]

【課題を解決するための手段】上記目的を達成するた
め、本発明の物体認識方法は、バックプロパゲ−ション
型ニュ−ラルネットワ−クの入力層の各ユニットを、3
次元状に配置された点の集合と対応付けて、認識対象物
体の3次元形状そのものを学習させ、次に3次元計測装
置を用いて得られる物体表面の点の3次元情報を用い3
次元モデルを作成し、3次元モデルに対して大きさおよ
び方向を調整した後、3次元モデルとニュ−ラルネット
ワ−クの入力層との対応付けを行い、3次元モデルを入
力層に入力することにより、3次元パタ−ンどうしのマ
ッチングを行わせ、ニュ−ラルネットワ−クが先に学習
したパタ−ンのうちで、入力された物体に最も近いパタ
−ンを出力させることを特徴としている。
In order to achieve the above object, the object recognition method of the present invention uses three units in each unit of the input layer of a back-propagation type neural network.
The three-dimensional shape itself of the recognition target object is learned in association with a set of points arranged in a three-dimensional manner, and then the three-dimensional information of the points on the surface of the object obtained by using the three-dimensional measurement device is used.
After creating a three-dimensional model, adjusting the size and direction of the three-dimensional model, associating the three-dimensional model with the input layer of the neural network and inputting the three-dimensional model into the input layer. According to this, the three-dimensional patterns are matched with each other, and the pattern closest to the input object among the patterns previously learned by the neural network is output.

【0005】[0005]

【作用】本発明においては、バックプロパゲ−ション型
ニュ−ラルネットワ−クの入力層の各ユニットを3次元
状に配置された点の集合と対応づけて、認識対象物体の
3次元形状そのものを学習させ、3次元計測装置を用い
て得られる物体表面の点の3次元情報を用いて作成した
3次元モデルをネットワ−クへの入力とすることによ
り、3次元パタ−ンどうしのマッチングを行い、2次元
パタ−ンどうしのマッチングの際に問題となった回転や
移動等による見え方の違いに煩わされることなく、見え
方に関する拘束条件を記述するのが困難な複雑な形状の
物体に対しても、判別および認識を行えるようにしてい
る。このように、ニュ−ラルネットワ−クに3次元パタ
−ンを学習させて、3次元計測装置により得られた物体
の3次元モデルとマッチングさせることにより、物体の
見え方に煩わされることなく、複雑な形状の物体の認識
を行うことができる。
In the present invention, each unit of the input layer of the back propagation type neural network is associated with a set of three-dimensionally arranged points to learn the three-dimensional shape itself of the object to be recognized. By inputting the 3D model created using the 3D information of the points on the surface of the object obtained by the 3D measuring device to the network, the 3D patterns are matched with each other. Even for objects with complicated shapes where it is difficult to describe constraints on the appearance without being bothered by the difference in appearance due to rotation or movement, which is a problem when matching two-dimensional patterns. , Discrimination and recognition can be performed. As described above, the neural network is trained to learn the three-dimensional pattern and matched with the three-dimensional model of the object obtained by the three-dimensional measuring apparatus, so that the appearance of the object is not bothered and complicated. It is possible to recognize objects having various shapes.

【0006】[0006]

【実施例】以下、本発明の実施例を、図面により詳細に
説明する。図1は、本発明の一実施例を示す物体認識方
法の説明図である。図1において、1は認識させようと
する物体、2は物体1の3次元形状モデル、3は3次元
形状モデルを正規化したもの、4は正規化モデルとニュ
−ラルネットワ−クの入力層との対応付け、5はバック
プロパゲ−ション型ニュ−ラルネットワ−クである。な
お、ニュ−ラルネットワ−ク5の2枚の壁は左側が入力
層、右側が中間層、右側の集約端子は出力層である。先
ず、バックプロパゲ−ション型ニュ−ラルネットワ−ク
5に、判別または認識したい物体1の3次元形状を学習
させる。このとき、認識したい物体の3次元形状モデル
2は、n×n×n個の点からなる3次元座標系中の点の
集合として、後述のような表現で示される。そして、ニ
ュ−ラルネットワ−クの入力層のユニットの総数もn×
n×n個とし、この座標系の各点と1対1に対応させる
ことにより3次元形状を学習させる。
Embodiments of the present invention will now be described in detail with reference to the drawings. FIG. 1 is an explanatory diagram of an object recognition method according to an embodiment of the present invention. In FIG. 1, 1 is an object to be recognized, 2 is a three-dimensional shape model of the object 1, 3 is a normalized three-dimensional shape model, 4 is a normalized model and an input layer of a neural network. 5 is a back propagation type neural network. The two walls of the neural network 5 have an input layer on the left side, an intermediate layer on the right side, and an output layer on the aggregated terminal on the right side. First, the back-propagation type neural network 5 is made to learn the three-dimensional shape of the object 1 to be discriminated or recognized. At this time, the three-dimensional shape model 2 of the object to be recognized is represented by a later-described expression as a set of points in a three-dimensional coordinate system consisting of n × n × n points. The total number of units in the input layer of the neural network is also n ×
A three-dimensional shape is learned by making the number n × n and making one-to-one correspondence with each point of this coordinate system.

【0007】次に、入力として与えられた物体から、3
次元計測装置、例えば光切断法によるレ−ザレンジファ
インダを用いて物体表面の点の3次元座標を計測し、3
次元形状モデル2を作成する。この3次元形状モデル2
を正規化して、大きさおよび方向を調整し(正規化モデ
ル3)、次にニュ−ラルネットワ−ク5の入力層との対
応付けを行い(対応付け4)、そして、ニュ−ラルネッ
トワ−ク5の入力層に入力することにより、ニュ−ラル
ネットワ−ク5が先に学習したパタ−ンのうちで最も近
いものが出力される。図3は、本発明における物体認識
方法の動作フロ−チャ−トである。先ず、バックプロパ
ゲ−ション型ニュ−ラルネットワ−ク5に認識させたい
物体1の3次元形状を学習させる(ステップ101)。
次に、入力として与える物体1から物体表面の点の3次
元座標を計測し、3次元形状モデル2を作る(ステップ
102)。次に、この3次元形状モデル2を正規化し
て、大きさと方向を調整する(ステップ103)。次
に、正規化モデル3とニュ−ラルネットワ−ク5の入力
層との対応付けを行って、入力層の一辺に含まれるユニ
ットの長さと等しくなるように拡大または縮小し(ステ
ップ104)、次にニユ−ラルネットワ−ク5の入力層
に入力する(ステップ105)。その結果、ニュ−ラル
ネットワ−ク5は先に学習したパタ−ンのうちの最もそ
れに近いパタ−ンを出力する(ステップ106)。
Next, from the object given as input, 3
A three-dimensional measuring device, for example, a laser range finder by a light section method is used to measure the three-dimensional coordinates of a point on the surface of the object.
A dimensional shape model 2 is created. This 3D shape model 2
Is adjusted to adjust the size and direction (normalized model 3), and then the neural network 5 is associated with the input layer (association 4), and then the neural network 5 is associated. By inputting to the input layer of the above, the closest one of the patterns previously learned by the neural network 5 is output. FIG. 3 is an operation flowchart of the object recognition method according to the present invention. First, the back-propagation type neural network 5 is made to learn the three-dimensional shape of the object 1 to be recognized (step 101).
Next, the three-dimensional coordinates of points on the surface of the object are measured from the object 1 given as an input to create a three-dimensional shape model 2 (step 102). Next, the three-dimensional shape model 2 is normalized to adjust the size and direction (step 103). Next, the normalized model 3 and the input layer of the neural network 5 are associated with each other, and enlarged or reduced so that the length is equal to the length of a unit included in one side of the input layer (step 104). To the input layer of the neural network 5 (step 105). As a result, the neural network 5 outputs the pattern closest to the previously learned patterns (step 106).

【0008】図2は、本発明で用いられる3次元形状モ
デルとそのニュ−ラルネットワ−クへの入力方法を示す
図である。ここでは、説明を簡単にするために、円柱状
の物体のモデルを例として用いる。物体の3次元形状モ
デルは、物体の表面に位置する点の集合として表わされ
る。これをニュ−ラルネットワ−クに入力する場合に
は、点の集合のモ−メント軸を求め、その第1軸をX
軸、第2軸をY軸、第3軸をZ軸に合わせるようにモデ
ルを回転させて、向きに関する正規化を行う。次に、X
軸方向の最大値、最小値の差が、3次元状に配列された
入力層の一辺に含まれるユニットの長さと等しくなるよ
うに拡大縮小を行い、長さの正規化を行う。このような
準備処理を行った後に、対応付けられた正規化モデルは
バックプロパゲ−ション型ニュ−ラルネットワ−ク5の
入力層に入力される。
FIG. 2 is a diagram showing a three-dimensional shape model used in the present invention and a method of inputting the model into a neural network. Here, in order to simplify the description, a cylindrical object model is used as an example. A three-dimensional shape model of an object is represented as a set of points located on the surface of the object. When inputting this to the neural network, the moment axis of the set of points is obtained, and the first axis is set to X.
The model is rotated so that the axis and the second axis are aligned with the Y axis and the third axis is aligned with the Z axis, and the orientation is normalized. Then X
The scaling is performed so that the difference between the maximum value and the minimum value in the axial direction is equal to the length of the unit included in one side of the input layer arranged in a three-dimensional manner, and the length is normalized. After performing such a preparation process, the associated normalized model is input to the input layer of the back-propagation type neural network 5.

【0009】以上説明したように、本発明によれば、ニ
ュ−ラルネットワ−クに3次元パタ−ンを学習させた
後、3次元計測装置により得られた物体の3次元モデル
とマッチングさせるので、物体の見え方に煩わされず
に、複雑な形状の物体を認識することができるという効
果がある。
As described above, according to the present invention, the neural network is made to learn the three-dimensional pattern and then matched with the three-dimensional model of the object obtained by the three-dimensional measuring device. There is an effect that an object having a complicated shape can be recognized without being bothered by how the object looks.

【発明の効果】【The invention's effect】

【0010】[0010]

【図面の簡単な説明】[Brief description of drawings]

【図1】本発明の一実施例を示す体物認識方法の説明図
である。
FIG. 1 is an explanatory diagram of a body recognition method according to an embodiment of the present invention.

【図2】本発明で用いられる3次元形状モデルとそのニ
ュ−ラルネットワ−クへの入力方法の説明図である。
FIG. 2 is an explanatory diagram of a three-dimensional shape model used in the present invention and an input method for the neural network.

【図3】本発明の一実施例を示す物体認識方法の動作フ
ロ−チャ−トである。
FIG. 3 is an operation flowchart of an object recognition method showing an embodiment of the present invention.

【符号の説明】[Explanation of symbols]

1 物体 2 3次元形状モデル 3 正規化モデル 4 ニュ−ラルネットワ−クの入力層 5 ニュ−ラルネットワ−ク DESCRIPTION OF SYMBOLS 1 Object 2 3 dimensional shape model 3 Normalization model 4 Input layer of neural network 5 Universal network

Claims (1)

【特許請求の範囲】[Claims] 【請求項1】 バックプロパゲ−ション型ニュ−ラルネ
ットワ−クの入力層の各ユニットを、3次元状に配置さ
れた点の集合と対応付けて、認識対象物体の3次元形状
そのものを学習させ、次に3次元計測装置を用いて得ら
れる物体表面の点の3次元情報を用い3次元モデルを作
成し、該3次元モデルに対して大きさおよび方向を調整
した後、該3次元モデルと上記ニュ−ラルネットワ−ク
の入力層との対応付けを行い、該3次元モデルを該入力
層に入力することにより、3次元パタ−ンどうしのマッ
チングを行わせ、該ニュ−ラルネットワ−クが先に学習
したパタ−ンのうちで、入力された物体に最も近いパタ
−ンを出力させることを特徴とする物体認識方法。
1. The unit of the input layer of a back-propagation neural network is associated with a set of points arranged three-dimensionally to learn the three-dimensional shape itself of an object to be recognized, A three-dimensional model is created using the three-dimensional information of the points on the object surface obtained by using a three-dimensional measuring device, and the size and direction of the three-dimensional model are adjusted. -Associating the three-dimensional model with the input layer of the laural network and inputting the three-dimensional model into the input layer to perform matching between the three-dimensional patterns, and the neural network learns first. Among the above patterns, the object recognition method is characterized in that the pattern closest to the input object is output.
JP4075055A 1992-03-31 1992-03-31 Method for recoginizing object Pending JPH05280953A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP4075055A JPH05280953A (en) 1992-03-31 1992-03-31 Method for recoginizing object

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP4075055A JPH05280953A (en) 1992-03-31 1992-03-31 Method for recoginizing object

Publications (1)

Publication Number Publication Date
JPH05280953A true JPH05280953A (en) 1993-10-29

Family

ID=13565139

Family Applications (1)

Application Number Title Priority Date Filing Date
JP4075055A Pending JPH05280953A (en) 1992-03-31 1992-03-31 Method for recoginizing object

Country Status (1)

Country Link
JP (1) JPH05280953A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1603071A1 (en) * 2004-06-01 2005-12-07 Fuji Jukogyo Kabushiki Kaisha Three-dimensional object recognizing system
JP2010519509A (en) * 2007-02-19 2010-06-03 ネーデルランデ オルガニサチエ ヴォール トエゲパスト−ナツールウェテンスハペリエク オンデルゾエク ティーエヌオー Ultrasonic surface monitoring method
CN103791851A (en) * 2012-10-30 2014-05-14 财团法人工业技术研究院 Non-contact three-dimensional object measuring method and device
CN104596442A (en) * 2015-02-10 2015-05-06 新维畅想数字科技(北京)有限公司 Assisted three-dimensional scanning device and method

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1603071A1 (en) * 2004-06-01 2005-12-07 Fuji Jukogyo Kabushiki Kaisha Three-dimensional object recognizing system
US7545975B2 (en) 2004-06-01 2009-06-09 Fuji Jukogyo Kabushiki Kaisha Three-dimensional object recognizing system
JP2010519509A (en) * 2007-02-19 2010-06-03 ネーデルランデ オルガニサチエ ヴォール トエゲパスト−ナツールウェテンスハペリエク オンデルゾエク ティーエヌオー Ultrasonic surface monitoring method
US8583407B2 (en) 2007-02-19 2013-11-12 Nederlandse Organisatie Voor Toegepast-Natuurwetenschappelijk Onderzoek Tno Ultrasonic surface monitoring
CN103791851A (en) * 2012-10-30 2014-05-14 财团法人工业技术研究院 Non-contact three-dimensional object measuring method and device
CN104596442A (en) * 2015-02-10 2015-05-06 新维畅想数字科技(北京)有限公司 Assisted three-dimensional scanning device and method

Similar Documents

Publication Publication Date Title
CN110074813B (en) Ultrasonic image reconstruction method and system
US5245696A (en) Evolution and learning in neural networks: the number and distribution of learning trials affect the rate of evolution
WO2021049005A1 (en) Information processing device and electronic apparatus equipped with same
JP2004534287A (en) Self-indicating method and apparatus for creating invariant stimulus expressions during systematic changes in sensing state
CN109598279A (en) Based on the zero sample learning method for generating network from coding confrontation
TWI731542B (en) Classification model building apparatus and classification model building method thereof
CN114937066A (en) Point cloud registration system and method based on cross offset features and space consistency
CN114925748B (en) Model training and modal information prediction method, related device, equipment and medium
WO2024060395A1 (en) Deep learning-based high-precision point cloud completion method and apparatus
CN112507947A (en) Gesture recognition method, device, equipment and medium based on multi-mode fusion
CN117218498B (en) Multi-modal large language model training method and system based on multi-modal encoder
Han et al. Underwater acoustic target recognition method based on a joint neural network
JP7298825B2 (en) Learning support device, learning device, learning support method, and learning support program
KR102274581B1 (en) Method for generating personalized hrtf
Santos et al. Speaker-independent visual speech recognition with the Inception V3 model
JPH05280953A (en) Method for recoginizing object
CN112820320B (en) Cross-modal attention consistency network self-supervision learning method
CN115708120A (en) Face image processing method, device, equipment and storage medium
Falcon Perez Machine-learning-based estimation of room acoustic parameters
CN116561533A (en) Emotion evolution method and terminal for virtual avatar in educational element universe
US20220057857A1 (en) Computer-implemented simulation of a virtual hand
CN114782592A (en) Cartoon animation generation method, device and equipment based on image and storage medium
CN113869186A (en) Model training method and device, electronic equipment and computer readable storage medium
Zhang et al. Deep meta-relation network for visual few-shot learning
Hussain et al. Feature Specific Hybrid Framework on composition of Deep learning architecture for speech emotion recognition