JP6825357B2 - Marketing equipment - Google Patents
Marketing equipment Download PDFInfo
- Publication number
- JP6825357B2 JP6825357B2 JP2016251931A JP2016251931A JP6825357B2 JP 6825357 B2 JP6825357 B2 JP 6825357B2 JP 2016251931 A JP2016251931 A JP 2016251931A JP 2016251931 A JP2016251931 A JP 2016251931A JP 6825357 B2 JP6825357 B2 JP 6825357B2
- Authority
- JP
- Japan
- Prior art keywords
- unit
- subject
- voice
- emotion analysis
- facial expression
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000008451 emotion Effects 0.000 claims description 98
- 230000008921 facial expression Effects 0.000 claims description 56
- 238000011156 evaluation Methods 0.000 claims description 31
- 238000000034 method Methods 0.000 claims description 26
- 238000001514 detection method Methods 0.000 claims description 16
- 230000008569 process Effects 0.000 claims description 16
- 238000003384 imaging method Methods 0.000 claims 2
- 230000001815 facial effect Effects 0.000 description 18
- 230000006870 function Effects 0.000 description 14
- 230000008859 change Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 238000004590 computer program Methods 0.000 description 4
- 238000011160 research Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 2
- 238000010195 expression analysis Methods 0.000 description 2
- IJGRMHOSHXDMSA-UHFFFAOYSA-N Atomic nitrogen Chemical compound N#N IJGRMHOSHXDMSA-UHFFFAOYSA-N 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 235000013361 beverage Nutrition 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000003292 diminished effect Effects 0.000 description 1
- 229910001873 dinitrogen Inorganic materials 0.000 description 1
- 239000007789 gas Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 239000010813 municipal solid waste Substances 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Image Analysis (AREA)
Description
本発明は，試供品の評価を被験者から自動的に得るための技術に関する。 The present invention relates to a technique for automatically obtaining an evaluation of a free sample from a subject.
商品・サービスの販売販促活動として，商品・サービスを実際に使用した被験者から，商品・サービスに関する意見を集めるマーケティング調査を行うことがある。商品・サービスを実際に使用した被験者から集めた意見はマーケティング調査後に集計され，商品・サービスの販売戦略を立案する際に参考にされる。 As a sales promotion activity for products and services, marketing research may be conducted to collect opinions on products and services from subjects who actually used the products and services. Opinions collected from subjects who actually used the product / service are aggregated after the marketing research and used as a reference when formulating a sales strategy for the product / service.
マーケティング調査の一つの手法として，商品・サービスに係る質問項目に対する回答を被験者から集めるアンケート調査が以前より用いられている。アンケート調査は，マーケティング調査において有用な手法の一つであるが，アンケート調査では，選択式の質問項目に対する回答する時の意見しか被験者から得られないので，商品・サービスを実際に使用している時の意見を得たい時はヒアリング調査が行われる。 As one method of marketing research, a questionnaire survey that collects answers from subjects to question items related to products and services has been used for some time. Questionnaire surveys are one of the useful methods in marketing research, but in questionnaire surveys, only the opinions when answering multiple-choice question items can be obtained from the subjects, so the products and services are actually used. When you want to get the opinion of the time, a hearing survey is conducted.
ヒアリング調査は，商品・サービスを実際に使用している被験者にインタビューし，例えば，商品・サービスの評価などを被験者に直接聞く手法である。例えば，飲み物（飲料）に係るヒアリング調査を行う場合，ヒアリング調査の対象になる飲み物を被験者に渡し，飲み物を口にした直後における評価を被験者から得る。 The hearing survey is a method of interviewing a subject who is actually using the product / service and directly asking the subject about the evaluation of the product / service, for example. For example, when conducting a hearing survey on drinks (beverages), the drink to be the subject of the hearing survey is given to the subject, and the subject is evaluated immediately after the drink is eaten.
商品に係るヒアリング調査を行う場合，ヒアリング調査を行う側の人が，マーケティングの対象になる商品である試供品を被験者に手渡しで提供していたが，特許文献１で開示されているワインディスペンサーのように，試供品（特許文献１ではワイン）を被験者に提供する装置も開発されているので，試供品の評価を被験者から自動的に得られることが望まれている。試供品の評価を被験者から自動的に得ることができれば，試供品の評価を被験者から自動的に得るための機能を，試供品を被験者に提供する装置（例えば，ワインディスペンサー）に組み込むことができる。 When conducting a hearing survey on a product, the person conducting the hearing survey handed a free sample, which is a product to be marketed, to the subject, but the wine dispenser disclosed in Patent Document 1 As described above, since a device for providing a free sample (wine in Patent Document 1) to a subject has also been developed, it is desired that the evaluation of the free sample can be automatically obtained from the subject. If the evaluation of the free sample can be automatically obtained from the subject, the function for automatically obtaining the evaluation of the free sample from the subject can be incorporated into a device (for example, a wine dispenser) that provides the free sample to the subject. ..
マーケティング調査に係る発明として，例えば，特許文献１では， 質問項目の内容の深さに応じて階層構造化した複数の質問項目データを記憶し，インタビューの残り時間が少ない場合に，浅い階層の質問項目を優先して推薦し，インタビューの残り時間が多い場合に，深い階層の質問項目を優先して推薦する装置が開示されているが，ヒアリング調査に係る従来の発明は，マーケティング調査にて被験者に問う質問項目に注目した発明で，試供品に係る評価を被験者から自動的に得るための発明ではない。 As an invention related to a marketing search, for example, in Patent Document 1, a plurality of question item data structured hierarchically according to the depth of the content of the question item is stored, and when the remaining time of the interview is short, a shallow hierarchy question is asked. A device that prioritizes and recommends items and preferentially recommends deep-level question items when the remaining time of the interview is long is disclosed, but the conventional invention related to the hearing survey is a subject in the marketing survey. This invention focuses on the question items to be asked, and is not an invention for automatically obtaining an evaluation of a sample from a subject.
また，アンケート調査やヒアリング調査は，試供品を試した後に，ある一定の期間が経過した後に調査されることもあることから，試供品を試した時の印象などが薄れてしまって正確な調査にならないことも考えられる。更に，これらの調査は集計の作業などが必要となることから，マーケティングデータ化するまでに時間がかかってしまっている。 In addition, questionnaire surveys and hearing surveys may be conducted after a certain period of time has passed after trying the free sample, so the impression when trying the free sample is diminished and an accurate survey is conducted. It is possible that it will not be. Furthermore, since these surveys require aggregation work, it takes time to convert them into marketing data.
このことからして，早期に試供品の評価を被験者から得ることができれば，マーケティング活動のＰＤＣＡサイクルを手早く回していくことができるため，本発明は，試供品を試したその時の被験者の状態を捉え，試供品の評価を自動的に被験者から取得できる装置を提供することを課題とする。 From this, if the evaluation of the free sample can be obtained from the subject at an early stage, the PDCA cycle of the marketing activity can be quickly started. Therefore, the present invention describes the state of the subject at the time when the free sample is tried. The challenge is to provide a device that can capture and automatically obtain the evaluation of free samples from the subjects.
上述した課題を解決する本発明は，カメラを用いて被験者の映像を撮影する映像撮影部と，マイクを用いて被験者の音声を受信する音声受信部と，前記映像撮影部が撮影した映像に含まれる顔画像を時系列で表情解析し，被験者の表情に対応する感情カテゴリの経時変化を示す表情感情解析データを生成する表情感情解析部と，前記音声受信部が受信した音声を時系列で音声認識し，被験者が発した単語に対応する感情カテゴリの経時変化を示す音声感情解析データを生成する音声感情解析部と，前記映像撮影部が撮影した映像から被験者の顔画像を検出する処理を行い，顔画像を検出してから顔画像が検出できなくなるまでの顔検出区間で前記映像撮影部が撮影した映像を前記表情感情解析部に引き渡し，前記顔検出区間における前記表情感情解析データを試供品の評価情報の一つとして取得する処理と，前記顔検出区間に前記音声受信部が受信した音声を前記音声感情解析部に引き渡し，前記顔検出区間における前記音声感情解析データを試供品の評価情報の一つとして取得する処理を実行する制御部とを備えたことを特徴とするマーケティング装置である。
また，本発明では，試供品を被験者に提供する試供品提供部を前記マーケティング装置に備えさせ，前記制御部が，前記映像撮影部が撮影した被験者の試供品に対する評価情報に，前記試供品提供部が試供品を被験者に提供した顔検出区間内の時間を含ませると，試供品を提供した前後における被験者の感情の変化を分析できるようになる。
更に，本発明では， 顔画像に基づく人物属性を推定する人物属性推定部を前記マーケティング装置に備えさせ，前記制御部が，前記映像撮影部が撮影した映像から検出した顔画像に基づく人物属性を人物属性推定部に推定させ，前記映像撮影部が撮影した被験者の試供品に対する評価情報に，前記人物属性推定部が推定した人物属性を含ませると，試供品を提供した被験者の人物属性に基づいて被験者の感情の変化を分析できるようになる。
The present invention for solving the above-mentioned problems is included in a video capturing unit that shoots a subject's image using a camera, an audio receiving unit that receives a subject's voice using a microphone, and an image captured by the video capturing unit. A facial expression emotion analysis unit that analyzes facial expressions in time series and generates facial expression emotion analysis data indicating changes in the emotion category corresponding to the subject's facial expression, and a voice received by the voice receiving unit in time series. A voice emotion analysis unit that recognizes and generates voice emotion analysis data indicating the time course of the emotion category corresponding to the word uttered by the subject, and a process of detecting the facial image of the subject from the video taken by the video shooting unit are performed. , The image taken by the video capturing unit in the face detection section from the detection of the face image to the time when the face image cannot be detected is delivered to the facial expression emotion analysis unit, and the facial expression emotion analysis data in the face detection section is sampled. The process of acquiring as one of the evaluation information of the above, the voice received by the voice receiving unit in the facial expression detection section is passed to the voice emotion analysis unit, and the voice emotion analysis data in the face detection section is used as the evaluation information of the sample. It is a marketing device characterized by having a control unit that executes acquisition processing as one of the above.
Further, in the present invention, the marketing device is provided with a free sample providing unit that provides a free sample to the subject, and the control unit provides the free sample to the evaluation information of the subject's sample taken by the video capturing unit. If the time within the face detection section in which the free sample is provided to the subject is included, the change in the subject's emotions before and after the free sample is provided can be analyzed.
Further, in the present invention, the marketing device is provided with a person attribute estimation unit that estimates the person attribute based on the face image, and the control unit obtains the person attribute based on the face image detected from the image captured by the video capturing unit. When the person attribute estimation unit is made to estimate and the evaluation information of the subject's sample taken by the videographing unit includes the person attribute estimated by the person attribute estimation unit, it is based on the person attribute of the subject who provided the sample. It will be possible to analyze changes in the subject's emotions.
上述した本発明によれば，試供品を試したその時の被験者の状態を捉え，試供品を提供した被験者から試供品の評価を自動的に取得する装置を提供できる。 According to the present invention described above, it is possible to provide a device that captures the state of a subject at the time of trying a free sample and automatically obtains an evaluation of the free sample from the subject who provided the free sample.
ここから，本発明の好適な実施形態を記載する。なお，以下の記載は本発明の技術的範囲を束縛するものでなく，理解を助けるために記述するものである。 From here, preferred embodiments of the present invention will be described. The following description does not constrain the technical scope of the present invention, but is described to aid understanding.
図１は，本実施形態に係るマーケティング装置１を説明する図である。図１で図示したマーケティング装置１は，試供品（図１では，ワイン）の評価情報を被験者２から自動的に得る機能と，試供品を被験者２に提供する機能（図１では，ワインディスペンサー）を備えているが，マーケティング装置１が備える機能は，試供品の評価情報を被験者２から自動的に得る機能のみであってもよい。 FIG. 1 is a diagram illustrating a marketing device 1 according to the present embodiment. The marketing device 1 illustrated in FIG. 1 has a function of automatically obtaining evaluation information of a free sample (wine in FIG. 1) from the subject 2 and a function of providing a free sample to the subject 2 (wine dispenser in FIG. 1). However, the function provided in the marketing device 1 may be only the function of automatically obtaining the evaluation information of the free sample from the subject 2.
図１において，本実施形態に係るマーケティング装置１とネットワーク３を介して接続しているサーバ４は，被験者２から自動的に得た試供品の評価情報の送信先となる装置である。試供品の提供者は，ネットワーク３を介してマーケティング装置１から送信された試供品の評価情報を集計し，集計結果を参考にして試供品の販売戦略を立案する。 In FIG. 1, the server 4 connected to the marketing device 1 according to the present embodiment via the network 3 is a device to which the evaluation information of the free sample automatically obtained from the subject 2 is transmitted. The free sample provider aggregates the evaluation information of the free sample transmitted from the marketing device 1 via the network 3, and formulates a sales strategy of the free sample with reference to the aggregated result.
図２は，本実施形態に係るマーケティング装置１のブロック図である。本実施形態に係るマーケティング装置１は，被験者２から得た試供品の評価情報などの記憶に用いるメモリ１７，無線または有線によりネットワーク３に接続するネットワークインターフェース１８を備え，試供品の評価情報を被験者２から自動的に得る機能として，カメラ１２ａを用いて被験者２の映像を撮影する映像撮影部１２と，マイク１４ａを用いて被験者２の音声を受信する音声受信部１４と，映像撮影部１２が撮影した映像に含まれる顔画像を時系列で表情解析し，被験者２の表情に対応する感情カテゴリの経時変化を示す表情感情解析データを生成する表情感情解析部１１と，音声受信部１４が受信した音声を時系列で音声認識し，被験者２が発した単語に対応する感情カテゴリの経時変化を示す音声感情解析データを生成する音声感情解析部１３と，顔画像に基づく人物属性を推定する人物属性推定部１５と，映像撮影部１２が撮影した映像から被験者２の試供品に対する評価情報を取得する処理を実行する制御部１０を備える。また，本実施形態に係るマーケティング装置１は，試供品を被験者２に提供する試供品提供部１６（本実施形態では，ワインディスペンサー）を備える。 FIG. 2 is a block diagram of the marketing device 1 according to the present embodiment. The marketing device 1 according to the present embodiment includes a memory 17 used for storing evaluation information of the sample obtained from the subject 2, a network interface 18 connected to the network 3 wirelessly or by wire, and uses the evaluation information of the sample as the subject. As functions automatically obtained from 2, the video shooting unit 12 that shoots the image of the subject 2 using the camera 12a, the audio receiving unit 14 that receives the sound of the subject 2 using the microphone 14a, and the video shooting unit 12 The facial expression emotion analysis unit 11 and the voice receiving unit 14 receive the facial expression analysis data included in the captured video by facial expression analysis in chronological order and generate facial expression emotion analysis data indicating the time course of the facial expression category corresponding to the facial expression of the subject 2. A voice emotion analysis unit 13 that recognizes the spoken voice in chronological order and generates voice emotion analysis data indicating changes in the emotion category corresponding to the word uttered by the subject 2, and a person who estimates the person's attributes based on the facial image. It includes an attribute estimation unit 15 and a control unit 10 that executes a process of acquiring evaluation information for the sample of the subject 2 from the video captured by the video capturing unit 12. Further, the marketing device 1 according to the present embodiment includes a free sample providing unit 16 (in this embodiment, a wine dispenser) that provides a free sample to the subject 2.
マーケティング装置１のメモリ１７としてはＲＡＭまたはＮＶＭ（Non-volatile memory）を利用できる。マーケティング装置１のネットワークインターフェース１８としては，無線によりネットワークに接続するＷｉｆｉ（登録商標），または，有線によりネットワークに接続するＥｔｈｅｒｎｅｔ（登録商標）を利用できる。マーケティング装置１の映像撮影部１２は，ＣＣＤやＣＭＯＳを撮像素子とするカメラ１２ａと，カメラ１２ａが出力したアナログ形式の映像をデジタル形式の映像に変換する回路基板などにより構成される。マーケティング装置１の音声受信部１４は，音声を集音するマイク１４ａと，マイク１４ａが出力したアナログ形式の音声をデジタル形式の音声に変換する回路基板などにより構成される。 RAM or NVM (Non-volatile memory) can be used as the memory 17 of the marketing device 1. As the network interface 18 of the marketing device 1, Wifi (registered trademark) that connects to the network wirelessly or Ethernet (registered trademark) that connects to the network by wire can be used. The image capturing unit 12 of the marketing device 1 is composed of a camera 12a having a CCD or CMOS as an image sensor, a circuit board that converts an analog format image output by the camera 12a into a digital format image, and the like. The voice receiving unit 14 of the marketing device 1 includes a microphone 14a that collects voice, a circuit board that converts analog-format voice output by the microphone 14a into digital-format voice, and the like.
マーケティング装置１の試供品提供部１６は，試供品を被験者２に提供する機構等により実現される。本実施形態の試供品提供部１６は，試供品であるワインを被験者２に提供できるように，被験者２に提供する数本のワインボトル１６ａ，ワインボトル１６ａ内に配置されたパイプと接続しているワインの注ぎ口１６ｂ，注ぎ口１６ｂからワインを出すときに被験者２が操作するスイッチ１６ｃを備え，スイッチ１６ｃが押されると，所定量のガス（例えば，窒素ガス）をワインボトル１６ａ内に充填し，所定量のワインが注ぎ口から出る構造になっている。また，図１で図示した試供品提供部１６は，ポイントカードやスマートフォンと近距離無線通信するリーダライタ１６ｄ，ワインを説明する映像などを流すディスプレイ１６ｅ，ワインの試飲に用いる紙コップ１６ｆ，および，使用済の紙コップ１６ｆを入れるゴミ箱１６ｇなどを備える。なお，ワインを説明する映像などを流すディスプレイ１６ｅをタッチパネルにすれば，タッチパネルを利用したアンケート調査を行うこともできる。 The free sample providing unit 16 of the marketing device 1 is realized by a mechanism or the like that provides a free sample to the subject 2. The sample providing unit 16 of the present embodiment is connected to several wine bottles 16a provided to the subject 2 and pipes arranged in the wine bottles 16a so that the sample wine can be provided to the subject 2. The wine bottle 16a is provided with a switch 16c operated by the subject 2 when the wine is discharged from the spout 16b and the spout 16b, and when the switch 16c is pressed, a predetermined amount of gas (for example, nitrogen gas) is filled in the wine bottle 16a. However, the structure is such that a predetermined amount of wine comes out of the spout. Further, the free sample providing unit 16 illustrated in FIG. 1 includes a reader / writer 16d for short-range wireless communication with a point card or a smartphone, a display 16e for displaying an image explaining wine, a paper cup 16f used for wine tasting, and a paper cup 16f. It is equipped with a trash can 16g or the like for storing a used paper cup 16f. If the display 16e, which displays a video explaining wine, is used as a touch panel, a questionnaire survey using the touch panel can be performed.
マーケティング装置１が備える表情感情解析部１１は，映像撮影部１２が撮影した映像に含まれる顔画像を時系列で表情解析し，被験者２の表情に対応する感情カテゴリの経時変化を示す表情感情解析データを生成する機能で，表情感情解析部１１としてマーケティング装置１のプロセッサを機能させるためのコンピュータプログラム，または，表情感情解析部１１として機能する専用のＩＣチップにより実現できる。 The facial expression emotion analysis unit 11 included in the marketing device 1 analyzes facial expressions included in the video captured by the video capturing unit 12 in chronological order, and expresses facial expression emotion analysis showing changes in the emotion category corresponding to the facial expression of the subject 2. It is a function to generate data, and can be realized by a computer program for operating the processor of the marketing device 1 as the facial expression emotion analysis unit 11 or a dedicated IC chip functioning as the facial expression emotion analysis unit 11.
図３は，本実施形態に係るマーケティング装置１の表情感情解析部１１の動作を説明する図である。本実施形態に係るマーケティング装置１の表情感情解析部１１は，ＦＡＣＳ（Facial Action Coding System）に係る技術を利用して，映像撮影部１２が撮影した被験者２の顔画像に対応する感情カテゴリを特定する。ＦＡＣＳ（Facial Action Coding System）を利用して感情カテゴリを特定する際，まず，マーケティング装置１の表情感情解析部１１は，映像のフレームに含まれる顔画像を検出する処理を行う（Ｓ１）。マーケティング装置１の表情感情解析部１１は，映像のフレームから顔画像を検出すると，顔画像内におけるフェイシャルポイントを特定する（Ｓ２）。フェイシャルポイントは，顔の表情を規定する顔のポイントで，ＦＡＣＳでは約５００個のフェイシャルポイントを特定する。次に，マーケティング装置１の表情感情解析部１１は，フェイシャルポイントを特定した顔画像を時系列（例えば，映像のフレーム毎）で追跡し（Ｓ３），各々のフェイシャルポイントの変化から表情の動きを特定する（Ｓ４）。マーケティング装置１が備える表情感情解析部１１は，表情の動きの組合せにより定義した複数の感情カテゴリ（驚き，恐怖，嫌悪，怒り，幸福，悲しみ）を記憶し，フェイシャルポイントを用いて特定した表情の動きにより定義されている感情カテゴリを特定し（Ｓ５），図３の手順は終了する。なお，マーケティング装置１の表情感情解析部１１は，図３のＳ２において，顔画像内におけるフェイシャルポイントを特定した後，図３のＳ３−Ｓ５を時系列で実行することで，被験者２の表情に対応する感情カテゴリの経時変化を示す表情感情解析データを生成する。例えば，表情感情解析データは，被験者２の表情が発声した時間と，被験者２の表情に対応する感情カテゴリを記述したＣＳＶ形式のテキストデータである。 FIG. 3 is a diagram illustrating the operation of the facial expression emotion analysis unit 11 of the marketing device 1 according to the present embodiment. The facial expression emotion analysis unit 11 of the marketing device 1 according to the present embodiment identifies the emotion category corresponding to the facial image of the subject 2 taken by the video photographing unit 12 by using the technique related to FACS (Facial Action Coding System). To do. When specifying an emotion category using FACS (Facial Action Coding System), first, the facial expression emotion analysis unit 11 of the marketing device 1 performs a process of detecting a facial image included in a video frame (S1). When the facial expression emotion analysis unit 11 of the marketing device 1 detects the facial image from the frame of the image, it identifies the facial point in the facial image (S2). Facial points are facial points that define facial expressions, and FACS specifies about 500 facial points. Next, the facial expression emotion analysis unit 11 of the marketing device 1 tracks the facial image in which the facial point is specified in a time series (for example, for each frame of the image) (S3), and changes the facial expression from the change of each facial point. Identify (S4). The facial expression emotion analysis unit 11 included in the marketing device 1 memorizes a plurality of emotion categories (surprise, fear, disgust, anger, happiness, sadness) defined by a combination of facial expression movements, and uses facial points to identify facial expressions. The emotion category defined by the movement is specified (S5), and the procedure of FIG. 3 ends. The facial expression emotion analysis unit 11 of the marketing device 1 identifies the facial points in the facial image in S2 of FIG. 3, and then executes S3-S5 of FIG. 3 in chronological order to obtain the facial expression of the subject 2. Generate facial expression emotion analysis data showing changes over time in the corresponding emotion category. For example, the facial expression emotion analysis data is CSV format text data that describes the time when the facial expression of the subject 2 is uttered and the emotion category corresponding to the facial expression of the subject 2.
マーケティング装置１が備える音声感情解析部１３は，音声受信部１４が受信した音声を時系列で音声認識し，被験者２が発した単語に対応する感情カテゴリの経時変化を示す音声感情解析データを生成する機能で，音声感情解析部１３としてマーケティング装置１のプロセッサを機能させるコンピュータプログラム，または，音声感情解析部１３として機能する専用のＩＣチップにより実現することができる。 The voice emotion analysis unit 13 included in the marketing device 1 recognizes the voice received by the voice reception unit 14 in chronological order, and generates voice emotion analysis data indicating the time course of the emotion category corresponding to the word uttered by the subject 2. This function can be realized by a computer program that functions the processor of the marketing device 1 as the voice emotion analysis unit 13 or a dedicated IC chip that functions as the voice emotion analysis unit 13.
図４は，本実施形態に係るマーケティング装置１の音声感情解析部１３の動作を説明する図である。本実施形態に係るマーケティング装置１の音声感情解析部１３は，音声受信部１４が出力した音声から感情カテゴリを特定する際，音声受信部１４が出力した音声に対応する単語を音声認識する（Ｓ１０）。マーケティング装置１の音声感情解析部１３は，感情カテゴリに対応する単語を感情カテゴリ毎に記憶し，音声認識して得られた単語に対応する感情カテゴリを特定し（Ｓ１１），図４の手順は終了する。なお，マーケティング装置１の音声受信部１４は，図４のＳ１０−Ｓ１１を時系列で実行することで，被験者２が発話した単語に対応する感情カテゴリの経時変化を示す音声感情解析データを生成する。例えば，音声感情解析データは，被験者２が発話した時間と，被験者２が発話した単語に対応する感情カテゴリを記述したＣＳＶ形式のテキストデータである。 FIG. 4 is a diagram illustrating the operation of the voice emotion analysis unit 13 of the marketing device 1 according to the present embodiment. When the voice emotion analysis unit 13 of the marketing device 1 according to the present embodiment specifies an emotion category from the voice output by the voice receiving unit 14, the voice emotion analysis unit 13 recognizes a word corresponding to the voice output by the voice receiving unit 14 (S10). ). The voice emotion analysis unit 13 of the marketing device 1 stores the words corresponding to the emotion categories for each emotion category, identifies the emotion category corresponding to the words obtained by voice recognition (S11), and the procedure of FIG. 4 is finish. The voice receiving unit 14 of the marketing device 1 executes S10-S11 of FIG. 4 in chronological order to generate voice emotion analysis data indicating the time course of the emotion category corresponding to the word spoken by the subject 2. .. For example, the voice emotion analysis data is CSV format text data that describes the time when the subject 2 utters and the emotion category corresponding to the word spoken by the subject 2.
マーケティング装置１が備える人物属性推定部１５は，映像撮影部１２が出力した映像に含まれる顔画像を解析して被験者２の人物属性を特定する機能で，マーケティング装置１が備える人物属性推定部１５としてプロセッサを機能させるコンピュータプログラムにより実現することができる。なお，顔画像から人物属性（性別・年齢）を推定する手法としては，人物属性毎に顔の特徴点を記憶し，顔画像から抽出した特徴点に対応する人物属性を推定する手法が広く用いられている。 The person attribute estimation unit 15 included in the marketing device 1 is a function of analyzing the face image included in the video output by the video shooting unit 12 to identify the person attribute of the subject 2, and the person attribute estimation unit 15 included in the marketing device 1. It can be realized by a computer program that makes the processor function as. As a method for estimating the person attributes (gender / age) from the face image, a method of memorizing the feature points of the face for each person attribute and estimating the person attributes corresponding to the feature points extracted from the face image is widely used. Has been done.
本実施形態に係るマーケティング装置１の制御部１０は，マーケティング装置１に実装されたプロセッサを動作させるコンピュータプログラムにより実現される機能で，試供品を提供する被験者２から自動的に試供品の評価情報を取得する機能に係る処理を実行する。 The control unit 10 of the marketing device 1 according to the present embodiment is a function realized by a computer program that operates a processor mounted on the marketing device 1, and evaluation information of the free sample is automatically obtained from the subject 2 who provides the free sample. Executes the process related to the function to acquire.
図５は，本実施形態に係るマーケティング装置１の制御部１０の動作を説明する図である。本実施形態に係るマーケティング装置１の電源がＯＮになると，マーケティング装置１の制御部１０等が起動し，マーケティング装置１の制御部１０は，映像撮影部１２と音声受信部１４を少なくとも作動させた後，映像撮影部１２が出力した映像から顔画像を検出する処理を実行する（Ｓ２０）。マーケティング装置１の映像撮影部１２は，所定のフレームレート（３０ｆｐｓや６０ｆｐｓ）で映像を出力し，マーケティング装置１の制御部１０は，映像撮影部１２が出力した映像のフレームから顔画像を検出する。 FIG. 5 is a diagram illustrating the operation of the control unit 10 of the marketing device 1 according to the present embodiment. When the power of the marketing device 1 according to the present embodiment is turned on, the control unit 10 and the like of the marketing device 1 are activated, and the control unit 10 of the marketing device 1 at least operates the video shooting unit 12 and the audio receiving unit 14. After that, a process of detecting a face image from the image output by the image capturing unit 12 is executed (S20). The video shooting unit 12 of the marketing device 1 outputs a video at a predetermined frame rate (30 fps or 60 fps), and the control unit 10 of the marketing device 1 detects a face image from the frame of the video output by the video shooting unit 12. ..
映像撮影部１２が出力した映像のフレームから顔画像が検出できると，マーケティング装置１の制御部１０は，映像撮影部１２が出力した映像をメモリ１７に録画する処理と，音声受信部１４が出力した音声をメモリ１７に録音する処理を開始する（Ｓ２１）。 When the face image can be detected from the frame of the video output by the video shooting unit 12, the control unit 10 of the marketing device 1 records the video output by the video shooting unit 12 in the memory 17, and the audio receiving unit 14 outputs it. The process of recording the recorded voice in the memory 17 is started (S21).
そして，映像の録画および音声データの録音をしている間，試供品を提供する動作を行ったことを示す試供品提供信号が試供品提供部１６から入力されると，マーケティング装置１の制御部１０は，この時の時間（録画を開始してからの時間，または，日時）を試供品提供時間としてメモリ１７に記憶し（Ｓ２２），映像の録画および音声データの録音を続行する。 Then, when the free sample providing signal indicating that the operation of providing the free sample is performed is input from the free sample providing unit 16 while recording the video and the audio data, the control unit of the marketing device 1 10 stores the time at this time (time from the start of recording or the date and time) in the memory 17 as the free sample provision time (S22), and continues recording the video and audio data.
映像撮影部１２が出力した映像から顔画像が検出できなくなると，マーケティング装置１の制御部１０は，映像の録画および音声の録音を終了する（Ｓ２３）。 When the face image cannot be detected from the video output by the video shooting unit 12, the control unit 10 of the marketing device 1 ends the video recording and the audio recording (S23).
映像の録画および音声データの録音を終了すると，マーケティング装置１の制御部１０は，まず，メモリ１７に録画した映像に含まれる顔画像をマーケティング装置１の人物属性推定部１５に引き渡し，メモリ１７に録画した映像に含まれる顔画像に対応する人物属性を推定する人物属性推定処理（Ｓ２４）を実行する。 When the video recording and the audio data recording are completed, the control unit 10 of the marketing device 1 first delivers the face image included in the video recorded in the memory 17 to the person attribute estimation unit 15 of the marketing device 1 and transfers it to the memory 17. The person attribute estimation process (S24) for estimating the person attribute corresponding to the face image included in the recorded video is executed.
次に，マーケティング装置１の制御部１０は，顔画像を検出してから顔画像が検出できなくなるまでの区間である顔検出区間に映像撮影部１２が撮影した映像を表情感情解析部１１に引き渡し，顔検出区間における被験者２の表情に対応する感情カテゴリの経時変化を示す表情感情解析データを表情感情解析部１１から取得する表情感情解析処理（Ｓ２５）を実行する。 Next, the control unit 10 of the marketing device 1 delivers the image captured by the image capturing unit 12 to the facial expression emotion analysis unit 11 in the face detection section, which is the section from the detection of the face image to the time when the face image cannot be detected. , The facial expression emotion analysis process (S25) for acquiring the facial expression emotion analysis data indicating the time course of the emotion category corresponding to the facial expression of the subject 2 in the facial expression detection section from the facial expression emotion analysis unit 11 is executed.
次に，マーケティング装置１の制御部１０は，顔検出区間に音声受信部１４が撮影した音声を音声感情解析部１３に引き渡し，顔検出区間における被験者２の音声に対応する感情カテゴリの経時変化を示す音声感情解析データを音声感情解析部１３から取得する音声感情解析処理（Ｓ２６）を実行する。 Next, the control unit 10 of the marketing device 1 delivers the voice captured by the voice receiving unit 14 to the voice emotion analysis unit 13 in the face detection section, and changes the emotion category corresponding to the voice of the subject 2 in the face detection section with time. The voice emotion analysis process (S26) for acquiring the indicated voice emotion analysis data from the voice emotion analysis unit 13 is executed.
次に，マーケティング装置１の制御部１０は，Ｓ２２でメモリ１７に記憶した試供品提供時間，人物属性推定処理（Ｓ２４）で得られた人物属性，表情感情解析処理（Ｓ２５）で得られた表情感情解析データおよび音声感情解析処理（Ｓ２６）で得られた音声感情解析データを試供品の評価情報としてサーバへ送信して（Ｓ２７），この手順は終了する。 Next, the control unit 10 of the marketing device 1 has the sample provision time stored in the memory 17 in S22, the person attribute obtained in the person attribute estimation process (S24), and the facial expression obtained in the facial expression emotion analysis process (S25). The emotion analysis data and the voice emotion analysis data obtained in the voice emotion analysis process (S26) are transmitted to the server as evaluation information of the sample (S27), and this procedure ends.
被験者２の表情に対応する感情カテゴリの経時変化を示す表情感情解析データを試供品の評価情報に含ませることで，被験者２が行動で示した感情の変化を時系列で分析でき，被験者２の音声に対応する感情カテゴリの経時変化を示す音声感情解析データを試供品の評価情報に含ませることで，被験者２の発話で示した感情の変化を時系列で分析できる。また，表情感情解析データと音声感情解析データの２つを試供品の評価情報に含ませることで，表情に基づく感情カテゴリが音声に基づく感情カテゴリと一致しているか否かなど，音声に基づく感情カテゴリを利用して表情に基づく感情カテゴリを分析できる。更に，試供品の評価情報に試供品提供時間を含ませることで，試供品を提供する前の被験者２の感情と，試供品を提供した後の被験者２の感情の変化を分析できる。更に，被験者２の人物属性を試供品の評価情報に含ませることで，人物属性に基づいて被験者２の感情を分析できる。 By including the facial expression emotion analysis data showing the temporal change of the emotion category corresponding to the facial expression of the subject 2 in the evaluation information of the sample, the change of the emotion shown by the subject 2 in the behavior can be analyzed in chronological order, and the subject 2 can analyze the emotion change. By including the voice emotion analysis data showing the change over time of the emotion category corresponding to the voice in the evaluation information of the sample, the change in emotion shown in the speech of the subject 2 can be analyzed in chronological order. In addition, by including the facial expression emotion analysis data and the voice emotion analysis data in the evaluation information of the sample, the emotion based on the voice, such as whether the emotion category based on the facial expression matches the emotion category based on the voice, etc. You can use categories to analyze emotion categories based on facial expressions. Further, by including the free sample provision time in the evaluation information of the free sample, it is possible to analyze the change in the emotion of the subject 2 before the free sample is provided and the emotion of the subject 2 after the free sample is provided. Further, by including the personal attribute of the subject 2 in the evaluation information of the free sample, the emotion of the subject 2 can be analyzed based on the personal attribute.
１ マーケティング装置
１０ 制御部
１１ 表情感情解析部
１２ 映像撮影部
１２ａ カメラ
１３ 音声感情解析部
１４ 音声受信部
１４ａ マイク
１５ 人物属性推定部
１６ 試供品提供部
２ 被験者
1 Marketing device 10 Control unit 11 Facial expression emotion analysis unit 12 Video shooting unit 12a Camera 13 Voice emotion analysis unit
14 Voice receiver 14a Microphone 15 Human attribute estimation unit 16 Free sample provider 2 Subject
Claims (3)
マイクを用いて被験者の音声を受信する音声受信部と，
前記映像撮影部が撮影した映像に含まれる顔画像を時系列で表情解析し，被験者の表情に対応する感情カテゴリの経時変化を示す表情感情解析データを生成する表情感情解析部と，
前記音声受信部が受信した音声を時系列で音声認識し，被験者が発した単語に対応する感情カテゴリの経時変化を示す音声感情解析データを生成する音声感情解析部と，
前記映像撮影部が撮影した映像から被験者の顔画像を検出する処理を行い，顔画像を検出してから顔画像が検出できなくなるまでの顔検出区間で前記映像撮影部が撮影した映像を前記表情感情解析部に引き渡し，前記顔検出区間における前記表情感情解析データを試供品の評価情報の一つとして取得する処理と，前記顔検出区間に前記音声受信部が受信した音声を前記音声感情解析部に引き渡し，前記顔検出区間における前記音声感情解析データを試供品の評価情報の一つとして取得する処理を実行する制御部と，
を備えたことを特徴とするマーケティング装置。 A video shooting unit that shoots the subject's video using a camera,
A voice receiver that receives the subject's voice using a microphone,
A facial expression emotion analysis unit that analyzes facial expressions included in the images taken by the video imaging unit in chronological order and generates facial expression emotion analysis data showing changes in emotion categories corresponding to the subject's facial expressions over time.
A voice emotion analysis unit that recognizes the voice received by the voice reception unit in chronological order and generates voice emotion analysis data indicating changes in the emotion category corresponding to the words uttered by the subject over time.
The process of detecting the face image of the subject from the image captured by the image capturing unit is performed, and the image captured by the image capturing unit is expressed as the facial expression in the face detection section from the detection of the face image to the time when the face image cannot be detected. The process of handing over to the emotion analysis unit and acquiring the facial expression emotion analysis data in the face detection section as one of the evaluation information of the sample, and the voice received by the voice receiving unit in the face detection section are obtained by the voice emotion analysis unit. A control unit that executes a process of acquiring the voice emotion analysis data in the face detection section as one of the evaluation information of the sample.
A marketing device characterized by being equipped with.
A person attribute estimation unit that estimates a person attribute based on a face image is provided, and the control unit causes the person attribute estimation unit to estimate a person attribute based on a face image detected from a video captured by the video shooting unit, and the video shooting The marketing apparatus according to claim 1 or 2, wherein the evaluation information of the sample of the subject photographed by the department includes the person attribute estimated by the person attribute estimation unit.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2016251931A JP6825357B2 (en) | 2016-12-26 | 2016-12-26 | Marketing equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2016251931A JP6825357B2 (en) | 2016-12-26 | 2016-12-26 | Marketing equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| JP2018106419A JP2018106419A (en) | 2018-07-05 |
| JP6825357B2 true JP6825357B2 (en) | 2021-02-03 |
Family
ID=62784594
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| JP2016251931A Active JP6825357B2 (en) | 2016-12-26 | 2016-12-26 | Marketing equipment |
Country Status (1)
| Country | Link |
|---|---|
| JP (1) | JP6825357B2 (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110766502B (en) * | 2018-07-27 | 2024-06-18 | 北京京东尚科信息技术有限公司 | Commodity evaluation method and system |
| CN110399837B (en) * | 2019-07-25 | 2024-01-05 | 深圳智慧林网络科技有限公司 | User emotion recognition method, device and computer readable storage medium |
| JP7283336B2 (en) * | 2019-09-30 | 2023-05-30 | 富士通株式会社 | IMPRESSION ESTIMATION METHOD, IMPRESSION ESTIMATION PROGRAM AND IMPRESSION ESTIMATION DEVICE |
| CN113823323B (en) * | 2021-09-30 | 2023-12-12 | 深圳万兴软件有限公司 | Audio processing method and device based on convolutional neural network and related equipment |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050289582A1 (en) * | 2004-06-24 | 2005-12-29 | Hitachi, Ltd. | System and method for capturing and using biometrics to review a product, service, creative work or thing |
| JP2013171494A (en) * | 2012-02-22 | 2013-09-02 | Sony Corp | Image processing device, method, and program |
| JP2015018301A (en) * | 2013-07-09 | 2015-01-29 | 株式会社リコー | Sales support terminal, program, and information processing method |
| JP6467965B2 (en) * | 2015-02-13 | 2019-02-13 | オムロン株式会社 | Emotion estimation device and emotion estimation method |
-
2016
- 2016-12-26 JP JP2016251931A patent/JP6825357B2/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| JP2018106419A (en) | 2018-07-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6825357B2 (en) | Marketing equipment | |
| US20040001616A1 (en) | Measurement of content ratings through vision and speech recognition | |
| WO2018079106A1 (en) | Emotion estimation device, emotion estimation method, storage medium, and emotion count system | |
| CN109460752B (en) | Emotion analysis method and device, electronic equipment and storage medium | |
| US11341148B2 (en) | Information processing system and information processing method to specify persons with good affinity toward each other | |
| US20170364854A1 (en) | Information processing device, conduct evaluation method, and program storage medium | |
| US20110282662A1 (en) | Customer Service Data Recording Device, Customer Service Data Recording Method, and Recording Medium | |
| US20080243494A1 (en) | Dialog detecting apparatus, dialog detecting method, and computer program product | |
| US20180047030A1 (en) | Customer service device, customer service method, and customer service system | |
| CN101378455A (en) | Apparatus including function to specify image region of main subject from obtained image, and method to specify image region of main subject from obtained image | |
| US8319818B2 (en) | Information processing device, information processing method and storage medium storing computer program | |
| CN111353804A (en) | Service evaluation method, device, terminal equipment and medium | |
| JP6405826B2 (en) | Information processing apparatus, information processing method, and program | |
| CN107705180A (en) | Shopping cart, shopping cart based reminding method and device | |
| CN107224290A (en) | The recording medium of action resolver, action analytic method and embodied on computer readable | |
| CN113764099A (en) | Psychological state analysis method, device, equipment and medium based on artificial intelligence | |
| JP7146373B2 (en) | Customer service system | |
| CN103594086B (en) | Speech processing system, device and method | |
| CN110677448A (en) | Associated information pushing method, device and system | |
| JP6855737B2 (en) | Information processing equipment, evaluation systems and programs | |
| JP7206741B2 (en) | HEALTH CONDITION DETERMINATION SYSTEM, HEALTH CONDITION DETERMINATION DEVICE, SERVER, HEALTH CONDITION DETERMINATION METHOD, AND PROGRAM | |
| CN113409822B (en) | Object state determining method and device, storage medium and electronic device | |
| JP7021488B2 (en) | Information processing equipment and programs | |
| WO2004104986A1 (en) | Voice output device and voice output method | |
| JP2020067562A (en) | Device, program and method for determining action taking timing based on video of user's face |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| A621 | Written request for application examination |
Free format text: JAPANESE INTERMEDIATE CODE: A621 Effective date: 20191030 |
|
| TRDD | Decision of grant or rejection written | ||
| A01 | Written decision to grant a patent or to grant a registration (utility model) |
Free format text: JAPANESE INTERMEDIATE CODE: A01 Effective date: 20201215 |
|
| A61 | First payment of annual fees (during grant procedure) |
Free format text: JAPANESE INTERMEDIATE CODE: A61 Effective date: 20201228 |
|
| R150 | Certificate of patent or registration of utility model |
Ref document number: 6825357 Country of ref document: JP Free format text: JAPANESE INTERMEDIATE CODE: R150 |