JP6206913B2 - Laughter promotion program and laughter promotion device - Google Patents

Description

  The present invention relates to a laughter promotion program and a laughter promotion device.

  There is a research report that cognitive functions are less likely to deteriorate in elderly people who have a lot of communication with people than in elderly people who have fewer people. Supporting communication between people can be expected to extend the healthy life expectancy. Therefore, efforts to support communication between elderly people are being carried out in various places.

  On the other hand, laughter and smiles are very important to facilitate communication between people. By the way, although a person's smile is a facial expression, an attempt has been made to evaluate the degree of smile of a person, for example, as in the technique described in Patent Document 1 below.

Japanese Patent No. 5022940

  By the way, the above-mentioned patent document relates to a laughter measurement method, and there is still room for examination regarding this application.

  For example, in the communication between people, if there is a person who laughs at an interesting place in a timely manner, the place becomes exciting. However, so-called “laughing Ueto” people are not always there, and even if they talk about the same thing, there are cases where laughter does not cease and sometimes there is no laughter at all.

  In other words, if a device such as a robot can be assigned to the function of so-called “laughing Ueto” that makes people laugh and cheer well, it will be possible to laugh even if there is no laughing Uedo person, and more communication between people It is expected to be smooth.

  In view of the above problems, an object of the present invention is to provide a laughter promotion program and a laughter promotion device that promote laughter by providing a trigger for laughter.

  As described in the above-mentioned problem, the present inventors have intensively studied the application of the laughter measurement method, and it is possible to smoothly perform this communication by providing an opportunity to laugh in communication between people. It was thought that it was possible to complete the present invention.

That is, when the laughter promotion program according to one aspect of the present invention calculates smile level data based on the face image data, determines whether or not the laughter data can be reproduced based on the smile level data, and reproduces the laughter data. If it is judged, the program is a laughter promotion program that reproduces the laughter data, and after the laughter data is reproduced based on the smile degree data, the first standby time elapses without reproducing the laughter data. Then, smile level data is calculated again based on the face image data, and a second waiting time is determined based on the smile level data.

A laughter promoting device according to another aspect of the present invention is a laughter promoting device including an imaging device, a speaker, and an information processing device that controls the imaging device and the speaker. If it is determined that the smile level data is calculated based on the face image data, the smile level determination unit that determines whether or not the laughter data is reproduced based on the smile level data, and the laughter data is determined to be reproduced, A laughter promoting device having a laughing voice reproducing unit for reproducing data, wherein the laughter determination unit determines a first waiting time after the laughing voice reproducing unit reproduces the laughter data based on the smile degree data; The laughter data is not reproduced, and the smile level data calculation unit calculates smile level data again based on the face image data after the first waiting time, and then the laughter determination unit Based on the face of data, and defining a second standby time.

  As described above, it is possible to provide a laughter promotion program and a laughter promotion device that promote laughter by providing a laughing opportunity.

It is a figure which shows the functional block of the laughter promotion apparatus which concerns on embodiment. It is a figure which shows the image of face image data. It is a figure which shows the flow of the laughter promotion method which concerns on embodiment. It is a figure which shows the flow of the laughter promotion method which concerns on embodiment. It is a figure which shows an example of the external appearance of the laughter promotion apparatus which concerns on embodiment. It is a figure which shows the result of use of the laughter promotion apparatus which concerns on an Example. It is a figure which shows the result of use of the laughter promotion apparatus which concerns on an Example.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. However, the present invention can be implemented in many different forms, and is not limited to the following embodiments and examples.

  FIG. 1 is a diagram showing functional blocks of a laughter promoting device (hereinafter referred to as “this device”) 1 according to the present embodiment. As shown in the figure, the present apparatus 1 is a laughter promoting device including an imaging device 2, a speaker 3, and an information processing device 4 that controls the imaging device 2 and the speaker 3. Includes a smile degree data calculation unit that calculates smile degree data based on the face image data acquired by the imaging apparatus 2, a laughter / non-permission determination unit that determines whether or not to reproduce the laughter data based on the smile degree data, and laughter data And a laughter playback unit that plays back the laughter data.

  The imaging device 2 in the device 1 can acquire information around the device 1 as an image. More specifically, it captures the face (expression) of a person around the device 1, Can be acquired and output to the outside of the imaging apparatus 2. An example of the imaging device 2 is not limited as long as it has the above function, but a so-called digital camera including an imaging element such as a CCD or a CMOS can be exemplified. A commercially available digital camera is sufficient.

  The device 1 includes a speaker 3 and can emit laughter data stored in advance in the information processing device 4 as sound. This speaker is not particularly limited, and a generally commercially available product including a coil, a magnet, a diaphragm and the like can be sufficiently used.

  In the present embodiment, the information processing device 4 is connected to the imaging device 2 and the speaker 3, and can control these and perform various processes.

  The information processing apparatus 4 in the apparatus 1 is not limited as long as information processing and control of each apparatus can be performed, and a plurality of integrated circuits are connected to electrodes and electrodes so that predetermined processing and control can be performed. It may be fixedly arranged on a printed circuit board on which a circuit is formed and electrically connected thereto, or a so-called computer may be used. In the former case, there is an advantage that the processing time can be shortened and the space can be saved because it is specialized for a specific process. In the latter case, the program can be easily rewritten and improved easily. There is an advantage such as. In the case of a computer, a commercially available one can be used. A computer generally includes a CPU (Central Processing Unit), a memory, a recording medium such as a hard disk, and a board including a printed circuit board on which electrodes and circuits for connecting these are formed. is there. In the case of using a computer, a predetermined function can be realized by storing a program in a recording medium such as a hard disk, and reading and executing the program in a memory.

  The information processing apparatus 4 can realize the functions shown in this figure by using the above configuration. Specifically, the information processing device 4 determines whether or not the laughter data can be reproduced based on the smile level data calculating unit 41 that calculates smile level data based on the face image data acquired by the imaging device 2. When it is determined that the laughter determination unit 42 to perform and the laughter data are to be reproduced, the laughter unit 42 functions as a laughter player 43 that reproduces the laughter data.

  The smile level data calculation unit 31 of the information processing device 4 calculates smile level data based on the face image data. Here, “face image data” refers to data including image information relating to a human face. The face image data may be the image information data acquired by the imaging device 2 as it is as the face image data, or may be image data created by extracting a human face area from the image information data. When image information data acquired by the imaging device 2 is used as face image data as it is, data corresponding to a human face area (face area data) is extracted from the face image data, and a smile is generated based on the face area data. It is preferable to calculate the degree data. FIG. 2 shows an image when face information data is extracted using image information data as face image data. When a plurality of human faces are included in the image information data, it is preferable to extract a plurality of face area data from the face image data and calculate smile degree data for each face area data. In addition, in the case where the face of a plurality of people is included in the image information data, if there is a possibility that a person who does not participate in communication may accidentally appear in the face of the face image data, Of these, it is also preferable to perform a process of narrowing down an area from which face area data is extracted.

  As is clear from the above description, the number of face image data for calculating smile degree data may be for one person, but is preferably for a plurality of persons. There are generally a plurality of persons who are the targets of communication promotion (hereinafter referred to as “subjects”), and it is possible to grasp the overall atmosphere by including a plurality, preferably all of the target persons, in the judgment targets. Can improve the accuracy of laughter.

  The smile level data calculation unit 41 of the information processing device 4 calculates smile level data from the face image data as described above. “Smile level data” is data that expresses a person's smile numerically, and it is possible to determine how much a person's smile is based on this numerical value. For example, when the numerical value is large, the smile is larger, and when the numerical value is low, it means that the person is not smiling or not smiling.

  As described above, the method of calculating the smile degree data is not limited as long as a smile of a person can be expressed numerically and used as an objective determination criterion. For example, “Shihong Lao and Masato Kawade. Vision-based face understanding technologies and therology applications. 3rd. , Koichi Kinoshita, Hiroki Tsuji, Masato Kawade, Real-time smile degree estimation, Interaction, No. 2008, pp. 47-48, 2008. ”.

  Further, when there are a plurality of target persons and there are a plurality of face image data, the calculation of the smile degree data is not limited, and it is more preferable to obtain and average each of the plurality of face image data. preferable. By taking the average, the processing becomes simple and evaluation as a whole becomes possible.

  Further, the laughter / non-permission determining unit 42 of the information processing apparatus determines whether or not the laughter voice data can be reproduced based on the smile degree data. Specifically, it is preferable to compare the numerical value of the smile level data obtained by the smile level data calculation unit with the threshold value data. More specifically, it is preferable to determine that the laughter data is reproduced if the smile degree data is larger than the threshold data, and to determine that the laughter data is not reproduced if the smile degree data is smaller than the threshold data.

  Here, as is clear from the above description, the “threshold data” is predetermined smile degree data, and is data necessary to determine whether or not to reproduce the laughter voice data. The threshold data is not limited as long as it can be used as this standard, but it may be set in advance as a predetermined value. However, after the operation of the apparatus is started and data is collected for a predetermined period. You may make it fix in.

  Note that when threshold data is determined after collecting data for a predetermined period after the start of the operation of the present apparatus, it is preferable to perform the determination based on ambient sound data acquired in the surroundings. For example, while acquiring face image data for a certain period and calculating smile level data based on the face image data, while acquiring ambient sound data, data relating to the correlation between the ambient sound data and this smile level data By acquiring the threshold value and determining threshold data that is considered appropriate, it becomes possible to promote laughter more suited to the actual situation. More specifically, it is considered that the ambient sound includes a so-called “laughing voice”, and if the laughing voice is included, the ambient sound may increase. This is useful when there are few ambient sounds other than human voice in a room such as a conference room. Therefore, if this ambient sound becomes louder, it is assumed that the subject is laughing, and the loudness of this ambient sound is compared with the smile level.For example, the smile level is slightly lower than the smile level when the ambient sound is louder. If data is played back, laughter can be promoted efficiently. Moreover, you may perform the process which determines whether it is a laughing voice by analyzing the volume of the surrounding sound, the periodicity of a volume, etc.

  The acquisition of the ambient sound data is not limited as long as the ambient sound can be output as data to the information processing apparatus, but a sound collector including a so-called microphone is preferable.

  In the present embodiment, the laughter playback unit 43 plays back laughter data. Specifically, when the laughter determination unit 32 determines to reproduce the laughter, the laughter data is output to the speaker and the sound is reproduced from the speaker, and when it is determined not to reproduce the laughter, the laughter data is not reproduced. .

  In the present embodiment, “laughing voice data” is voice data relating to a person's laughing voice, which is output to a speaker and is transmitted from the speaker to the subject as a person's laughing voice. The laughter data is preferably stored in advance in a computer. The laughing voice that is the basis of the laughing voice data may be a laughing voice of an actual person or a synthesized voice created by a computer program or the like. The laughter data is preferably a plurality of types of laughter. For example, it may be a laughing voice of a large number of people or a laughing voice that makes one person laugh. In this case, it is also preferable to select the type of laughter data to be reproduced based on the calculated value of smile level data. It should be noted that the length of the laughing voice is sufficient as long as it can promote laughing, for example, it may be one second or longer.

  In this apparatus, it is convenient and easy to reproduce the laughter data by outputting it to the speaker. However, a magnetic recording medium such as a tape is separately provided in advance and the laughter data is recorded on the information processing apparatus. On the basis of the instruction, reading and reproducing operations of the magnetic recording medium such as a tape may be performed.

  In addition, the laughter determination unit 42 of the information processing apparatus, although not limited thereto, forcibly reproduces the laughter data for the first waiting time after determining the reproduction of the laughter data based on the smile data. It is preferable not to have it. This is because “laughter” lasts for a certain period of time, and the degree of smile is maintained to some extent while laughing. If this waiting time is not provided, laughter is always played back in one laughter operation. This is because it is necessary to avoid that the laughing voice stops as a result. This period is not particularly limited, and can be determined by acquiring data for a predetermined period after the start of the operation of the apparatus. However, a certain period may be determined in advance. When the predetermined period is set in advance, it is not limited, but for example, it is preferable that the period is 1 second or more and 20 seconds or less, preferably 10 seconds or less after the laughing voice is reproduced. Here, “to not reproduce laughter data” means to determine whether to acquire face image data, not to calculate smile level data, and to reproduce laughter data. The process of not performing is also included.

  In addition, the laughter / prohibition determination unit 42 of the information processing apparatus calculates the smile degree data again based on the face image data after the first standby time has elapsed, and based on the smile degree data, It is preferable to determine a second waiting time. Specifically, this is processing for determining whether or not laughter continues after the first waiting time has elapsed. When the smile level data is calculated again based on the face image data after the first waiting time has elapsed, and the smile level data is higher than the threshold value data, it can be determined that the previous laughter continues, Provide a second waiting time. This prevents the laughter from stopping. On the other hand, if the smile level data is lower than the threshold data, it can be determined that laughter has once settled, so a second waiting time of 0 seconds or a relatively short time is provided to prepare for the next laughter playback. That is, the second waiting time is set to an appropriate time based on smile level data and threshold data. When it is determined that laughter continues, the second waiting time may be determined by acquiring data for a predetermined period after the start of the operation of the apparatus. The period is preferably 20 seconds or longer. On the other hand, when it can be determined that laughter has stopped after the elapse of the first waiting time, the time is preferably shorter than the above time, and more specifically, a period of 0 second to 10 seconds is preferable. Of course, this determination can also include ambient sound data as described above. In addition, the flow of a series of processes of this apparatus including this process is shown in FIGS.

  Moreover, in this embodiment, it is also preferable that the present apparatus 1 is provided with a frame body for adjusting the appearance so as to smooth the surroundings and smoothly measure communication. For example, by making the appearance a robot like a doll, it becomes possible to soften people and facilitate communication. An example of the appearance in this case is shown in FIG.

  As described above, according to the present embodiment, it is possible to provide a laughter promotion program and a laughter promotion device that promote laughter by providing an opportunity to laugh.

  Here, the laughter promoting device described in the above embodiment was actually created and its effect was confirmed. This will be specifically described below.

  First, a computer connected with a camera, a microphone, and a speaker was connected to the frame body shown in FIG. In addition, the program and laughter data according to the above embodiment were recorded in this computer, and the apparatus was completed. For the calculation of smile degree data, OKA Vision (OMRON) was used.

  And the said apparatus was made to participate in the free conversation of 9 minutes by three participants. In this free conversation, the length of the laughter playback time of this device is 1.5 seconds, the threshold data θ is 60, the first standby time is 2 seconds, and the second standby time is when laughter continues. It was set to 10 seconds when it was higher than the threshold data, and 0 seconds when laughter was not continuing (when it was below the threshold data).

  The result is shown in FIG. This figure is categorized according to the origin of laughter that occurs during conversation. In the same scene, the rate of laughing between the talker and the device was 50%, the rate of laughing only with the device was 35%, and the rate of laughing only with the talker was 15%. It was confirmed that the rate of laughing by both the device and the talker was as high as 50%.

  FIG. 7 is a diagram showing which person laughed first when both the apparatus shown in FIG. 6 and the conversation person laughed. As a result, it was confirmed that the number of cases where the device laughed first was 54%, and the case where the device laughed later was 46%. In other words, it was confirmed that the device was able to contribute to many laughters in the form of invitation laughter and dragged laughter, but the timing was appropriate.

  As mentioned above, the usefulness of this invention has been confirmed by the present Example.

The present invention has industrial applicability as a laughter promotion program and a laughter promotion device.

Claims (4)

On the computer,
Calculate smile level data based on face image data,
Judgment of whether or not to reproduce the laughter data based on the smile degree data,
When it is determined that the laughter data is to be reproduced, the laughter promotion program for reproducing the laughter data,
After reproducing the laughter data based on the smile degree data, the first standby time without reproducing the laughter data,
A laughter promotion program that calculates smile level data again based on face image data after the first standby time elapses and determines a second standby time based on the smile level data.   The laughter promotion program according to claim 1, wherein whether or not the laughter voice data can be reproduced is determined by comparison with threshold data.   The laughter promotion program according to claim 1, wherein whether or not the laughter voice data can be reproduced is determined based on the smile level data based on the acquired ambient sound data. An imaging device;
Speakers,
A laughter promoting device comprising: an information processing device that controls the imaging device and the speaker;
The information processing apparatus includes a smile degree data calculation unit that calculates smile degree data based on face image data, a laughter / non-permission determination unit that determines whether to reproduce laughter data based on the smile degree data, and the laughter data. A laughter promoting device having a laughter playback unit that plays back the laughter data when it is determined to be played back ,
The laughter determination unit does not reproduce the laughter data by setting a first waiting time after the laughter reproduction unit reproduces the laughter data based on the smile degree data,
The smile level data calculation unit calculates smile level data again based on the face image data after the first waiting time has elapsed, and then
The laughter propriety determining unit is a laughter promoting device that determines a second waiting time based on the smile degree data.