TWI787205B - Expression recording system, stroller, and expression recording program - Google Patents

Abstract

The expression recording system (1) of the present invention has a camera (3) installed in a stroller (2), and a terminal device (4) held by a user of the stroller (2). The camera (3) can take pictures of the occupant of the stroller (2), and the terminal device (4) can communicate with the camera (3). If the terminal device (4) is input with the camera image captured by the camera (3), it will detect the predetermined expression of the occupant of the stroller (2) from the camera image, and when the predetermined expression is detected, the camera will (3) Send a request to capture the camera image around the stroller (2). Thereby, it is possible to record the predetermined expression of the baby sitting in the baby carriage (2) when going out of the baby carriage (2).

Description

Expression recording system, stroller, and expression recording program

The present invention relates to an expression recording system for recording the predetermined expressions of the occupant of the stroller.

Traditionally, strollers have been used when traveling with infants and toddlers. Known baby carriages have done various ingenuities in order to make the infant sitting on the seat part as comfortable as possible. For example, a seat portion on which an infant sits is formed of a member having excellent cushioning properties, thereby improving ride comfort (refer to Patent Document 1).

Also, in the past, in order to make the baby carriage a means of transportation that infants want to ride actively, it has also been proposed that there is a baby carriage that can enjoy content such as video and music while riding in the baby carriage (see Patent Document 2).

In the above-mentioned conventional stroller, since a comfortable ride can be obtained, or content such as video or music can be enjoyed, the infant riding in the stroller can enjoy going out of the stroller, thereby increasing the chance of showing a smiling face. However, no system has been proposed so far that records the smiles of infants and toddlers riding in the stroller while they are out in the stroller, and there is still room for development.

[Prior Art Literature] [Patent Document]

Patent Document 1: Japanese Patent Laid-Open No. 2004-216998

Patent Document 2: Japanese Patent Laid-Open No. 2008-308053

The present invention has been accomplished under the background described above. The object of the present invention is to provide an expression recording system capable of recording predetermined expressions of infants riding in the stroller when going out of the stroller.

One aspect of the present invention is an expression recording system. The expression recording system is equipped with: a camera installed on a baby carriage, which can take camera images of the occupants of the baby carriage; and a terminal device, which is controlled by the baby carriage Possessed by the user, it can communicate with the camera; and the terminal device has: a data input unit, which inputs the camera image captured by the camera; an expression detection unit, which determines the occupant of the stroller from the camera image detection of expressions; and a surrounding shooting request unit, which sends a surrounding shooting request to the camera to capture images of the camera around the stroller when a predetermined expression is detected.

Another aspect of the present invention is a stroller. The stroller is equipped with a camera capable of capturing images of the occupant. The camera can communicate with a terminal device held by the user of the stroller. The terminal device is installed on the stroller. When the occupant's predetermined expression is detected from the camera image captured by the camera, a surrounding shooting request is sent to the camera, and the camera takes a camera image around the stroller according to the surrounding shooting request.

Another aspect of the present invention is a program, which is executed by a terminal device held by the user of the stroller. The terminal device can communicate with a camera installed on the stroller, and the camera can take pictures of the occupant of the stroller. The program enables the terminal device to perform the following processing: if the camera image captured by the camera is input, the predetermined expression of the occupant of the stroller is captured from the camera image; and when the predetermined expression is detected , to send to the camera a surrounding shooting request for capturing camera images around the stroller.

Another aspect of the present invention is an expression recording system. The expression recording system is equipped with: a camera installed on a mobile body, which can take a camera image of the occupant of the mobile body; and a terminal device, which is controlled by the user. Possessed and can communicate with the camera; and the terminal device is equipped with: a data input unit, which inputs the camera image captured by the camera; an expression detection unit, which detects the predetermined expression of the occupant of the mobile object from the camera image ; and a surrounding shooting request unit, which sends a surrounding shooting request to the camera to capture a camera image around a moving object when a predetermined expression is detected.

As explained below, the present invention has other aspects. Therefore, the disclosure of the present invention provides an aspect of a part of the present invention, and is not intended to limit the scope of the present invention described and applied for here.

1‧‧‧Expression recording system (smile recording system)

2‧‧‧Prams

3‧‧‧Camera

4‧‧‧Terminal device

5‧‧‧arm

6‧‧‧Seating Department

10‧‧‧Touch panel

11‧‧‧Speaker

12‧‧‧Storage Department

13‧‧‧Department of Communications

14‧‧‧The first control department

140‧‧‧Data input department

141‧‧‧Expression detection department

142‧‧‧Surrounding shooting requirements department

143‧‧‧Display processing unit

144‧‧‧Continue to shoot the request department

145‧‧‧Image selection department

15‧‧‧The second control department

150‧‧‧Sentiment Analysis Department

151‧‧‧Play processing unit

152‧‧‧Position Obtaining Department

153‧‧‧Records Processing Department

154‧‧‧Notification Processing Department

155‧‧‧Direction information acquisition department

156‧‧‧Looking direction detection processing unit

Fig. 1 is an explanatory diagram of an expression recording system (smile recording system) according to an embodiment of the present invention.

Fig. 2 is a block diagram of a terminal device according to an embodiment of the present invention.

Fig. 3 is an explanatory view of the same screen display in the embodiment of the present invention.

Fig. 4 is an explanatory diagram of displaying the detection position of a smile according to the embodiment of the present invention.

Fig. 5 is a flow chart of the same screen display process according to the embodiment of the present invention.

Fig. 6 is a flow chart of smile image selection processing according to the embodiment of the present invention.

Fig. 7 is a flow chart of video and music playback processing in the embodiment of the present invention.

Fig. 8 is a flow chart of smile detection position recording processing and proximity notification processing according to the embodiment of the present invention.

Fig. 9 is a flow chart of gaze direction detection processing according to the embodiment of the present invention.

The detailed description of the present invention is described below. However, the following detailed description and accompanying drawings are not intended to limit the invention.

The facial expression recording system of the present invention is provided with: a camera, which is installed on a baby carriage, and can take a camera image of the occupant of the baby carriage; communication; and the terminal device is equipped with: a data input unit, which inputs the camera image captured by the camera; an expression detection unit, which detects the predetermined expression of the occupant of the stroller from the camera image; and a surrounding shooting request unit, When a predetermined expression is detected, it sends a request to the camera to capture a camera image around the stroller.

With this configuration, it is possible to take a camera image of the occupant (for example, an infant) while going out in the stroller. Moreover, for example, if the occupant of the stroller shows a predetermined expression when passing a certain location, the camera image around the stroller at the location will be captured. Thereby, the camera image of the predetermined expression of the occupant of the baby carriage, and the camera image of the surroundings of the baby carriage including factors causing the occupant to express the predetermined expression (objects the occupant likes, etc.) can be obtained.

In addition, in the expression recording system of the present invention, the terminal device may also be equipped with a display processing unit that compares the camera image of the occupant when a predetermined expression is detected with the surrounding area when the predetermined expression is detected. The camera images are displayed on the same screen.

With this configuration, it is possible to easily grasp the correspondence between the predetermined expression of the occupant of the stroller (such as an infant, etc.) and the cause of the occupant's expression of the predetermined expression (objects the occupant likes, etc.) on the same screen .

In addition, in the facial expression recording system of the present invention, the terminal device may also be equipped with: a continuation shooting request unit, which sends a continuation shooting request to the camera to continue taking pictures of the occupant of the stroller while a predetermined expression is detected. Requirements; and an image selection unit, which selects camera images with a predetermined expression level above a predetermined value from among the camera images of the occupants that continue to be photographed.

With this configuration, while the occupant of the stroller (for example, infants, etc.) shows a predetermined expression, the camera image (camera image of the predetermined expression) is continuously captured on the occupant, and the predetermined expression is automatically selected from it Higher level camera images. Thereby, camera images with characteristic expressions can be obtained.

Also, in the facial expression recording system of the present invention, the terminal device may also be equipped with: an emotion analysis unit, which analyzes the emotion of the occupant of the stroller from the camera image; Play video or music based on the emotion of the occupant of the stroller.

With this configuration, the emotion of the occupant is analyzed from the camera image of the occupant of the stroller (for example, an infant, etc.), and the image or music corresponding to the emotion is automatically played. Thereby, it is possible to play video or music that matches the emotion of the occupant while the stroller is out and about.

In addition, in the facial expression recording system of the present invention, the terminal device may also be equipped with: a position information acquisition unit, which obtains the position information of the baby carriage from the camera; a recording processing unit, which detects the position of the baby carriage when a predetermined expression is detected. It is recorded as the expression detection position; and the notification processing unit notifies the user of the baby stroller when the baby carriage approaches the expression detection position.

With this structure, the position (expression detection position) where the occupant of the stroller (such as infants, etc.) shows a predetermined expression is recorded, and the user will be notified when he approaches the position next time. Thereby, the user of the stroller can grasp the location where the occupant of the stroller shows a predetermined expression (the location that the occupant likes), and can know that the location is close to the location while the stroller is out.

Also, in the facial expression recording system of the present invention, the terminal device may also be equipped with: a direction information acquisition unit, which obtains direction information of the stroller from the camera; and a line-of-sight detection processing unit, which detects a predetermined The direction of the stroller when the expression is on is detected as the line of sight of the occupant.

With this configuration, when the occupant of the stroller (such as an infant) expresses a predetermined expression, the direction of sight (direction of observation) of the occupant can be known. Thereby, the favorite object of the passenger can be specified.

The stroller of the present invention is equipped with a camera capable of taking camera images of the occupant. The camera can communicate with a terminal device held by the user of the stroller. When a predetermined expression is detected, a surrounding shooting request is sent to the camera, and the camera captures a camera image around the stroller according to the surrounding shooting request.

With this stroller, similarly to the above-mentioned system, the camera image of the predetermined expression of the occupant of the stroller and the factors (such as the favorite object of the occupant) including the reason for the occupant's expression of the predetermined expression can be obtained. Camera image of the surroundings of the stroller.

The program of the present invention is executed by the terminal device held by the user of the stroller. The terminal device can communicate with the camera installed in the stroller, and the camera can take the camera image of the occupant of the stroller. And the program enables the terminal device to perform the following processing: if the camera image captured by the camera is input, then the predetermined expression of the occupant of the stroller is detected from the camera image; and when the predetermined expression is detected, the camera sends Requirements for surrounding shooting of camera images around the stroller.

This program can also obtain the camera image of the predetermined expression of the occupant of the stroller and the baby including the cause of the occupant's expression of the predetermined expression (the favorite object of the occupant, etc.) similarly to the above-mentioned system. Camera images around the car.

According to the present invention, it is possible to record the predetermined expression of an infant sitting in the stroller while going out of the stroller.

(implementation form)

An expression recording system according to an embodiment of the present invention will be described below using drawings. In this embodiment, a case where it is used in an expression recording system such as a stroller for infants and young children is exemplified. The facial expression recording system has the function of recording the predetermined facial expressions of infants and toddlers riding in the stroller when going out with the stroller. Hereinafter, although the example of the predetermined facial expression is described with the example of "smiling face", other facial expressions such as "sad face", "angry face" or "funny face" can be implemented in the same way.

The configuration of an expression recording system (smile recording system) according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is an explanatory diagram showing a schematic configuration of an expression recording system according to this embodiment. As shown in FIG. 1 , the expression recording system 1 includes a camera 3 mounted on a stroller 2 and a terminal device 4 held by a user of the stroller 2 . Furthermore, the occupant of the stroller 2 is, for example, an infant, and the user of the stroller 2 is, for example, a guardian of the infant.

First, the configuration of the camera 3 will be described with reference to FIG. 1 . The camera 3 is attached to the stroller 2 so as to capture a camera image of the occupant of the stroller 2 . Camera images can be either still images or moving images. For example, the camera 3 is mounted on the arm 5 of the baby carriage 2 in a state where the shooting direction is directed toward the occupant of the baby carriage 2 (that is, in a state in which the arm 5 is facing). The arm 5 is formed, for example, in an arc shape or an arc shape, and is arranged so as to straddle the front of the seat portion 6 of the stroller 2 (in front of the occupant sitting on the seat portion 6). Furthermore, the camera 3 may be built in the arm 5 or may be detachably attached to the arm 5 .

In addition, the camera 3 is configured to take pictures of the surroundings of the stroller 2 . For example, the camera 3 is configured to use a wide-angle lens to capture images of the surroundings of the stroller 2 at a wide angle. Alternatively, the camera 3 is configured to capture images of the entire periphery of the stroller 2 using a 360-degree lens. In addition, the camera 3 may also have a left-right rotation function or a vertical rotation function. In this case, the camera 3 is configured to be able to photograph the surroundings (or the entire surroundings) of the stroller 2 by rotating the camera lens in the left-right direction or in the up-down direction.

In addition, the camera 3 is equipped with a GPS that communicates with GPS (Global Positioning System; Global Position System) satellites to obtain position information (such as longitude and latitude information) indicating the current position of the camera 3 (position of the stroller 2) Function. In addition, the camera 3 has a gyroscope function for acquiring direction information (for example, orientation information) indicating the direction the camera 3 is currently facing (the direction in which the stroller 2 is facing). In addition, the camera 3 has a function of communicating with the terminal device 4 wirelessly or by wire. Therefore, the camera 3 can send the location information or direction information to the terminal device 4 in addition to the data of the captured camera image. In addition, the camera 3 can receive a request signal such as a surrounding shooting request or a continuation shooting request from the terminal device 4 , which will be described later. Furthermore, the battery of the camera 3 can be installed in the camera 3 itself or in the stroller 2 .

Next, the configuration of the terminal device 4 will be described with reference to FIG. 2 . FIG. 2 is a block diagram illustrating the configuration of the terminal device 4. As shown in FIG. The terminal device 4 is, for example, a portable terminal device 4 such as a smart phone. As shown in FIG. 2 , the terminal device 4 includes a touch panel 10 , a speaker 11 , a storage unit 12 , a communication unit 13 , a first control unit 14 , and a second control unit 15 . Furthermore, the first control unit 14 and the second control unit 15 may also be constituted by one control unit.

The touch panel 10 also has the functions of an input unit and a display unit. Therefore, the user of the terminal device 4 (the user of the stroller 2 ) can input various information from the touch panel 10 . Moreover, the touch panel 10 displays various information so that the user of the terminal device 4 (the user of the stroller 2 ) can confirm. The speaker 11 has a function of outputting audio to the user of the terminal device 4 (the user of the stroller 2 ).

The storage unit 12 is composed of a memory and the like, and can store various data. For example, data of camera images captured by the camera 3 are stored in the storage unit 12 . Data such as images or music may also be stored in the storage unit 12 . Also, programs for realizing various functions of the terminal device 4 (including expression recording function) are stored in the storage unit 12 . Various functions of the terminal device 4 can be realized by executing the program.

The communication unit 13 has a function of communicating with external devices wirelessly or by wire. The external device also includes the aforementioned camera 3 . Therefore, the communication unit 13 has the function of communicating with the camera 3 wirelessly or wiredly, and the terminal device 4 can receive the aforementioned location information or direction information from the camera 3 in addition to the data of the captured camera image. In addition, the terminal device 4 may transmit a request signal such as a surrounding shooting request or a continuation shooting request to the camera 3, which will be described later. Furthermore, the communication method can utilize known methods for it.

The first control unit 14 is a control unit for carrying out main control about the expression recording function, and is provided with a data input unit 140, an expression detection unit 141, a surrounding shooting request unit 142, a display processing unit 143, a continuation shooting request unit 144, and Image selection unit 145 .

The data input unit 140 has the function of an input interface for inputting various data in order to realize the expression recording function. For example, a camera image captured by the camera 3 (for example, a camera image of an occupant of the stroller 2 ) is input to the data input unit 140 . The expression detection unit 141 has a function of detecting a predetermined expression of the occupant of the stroller 2 from the camera image input to the data input unit 140 . For example, the expression detecting unit 141 can detect the smiling face of the occupant of the stroller 2 by performing image processing for detecting a smiling face on a camera image. Furthermore, the method for detecting smiley faces can be implemented using known methods. Similarly, known methods can also be used to detect other predetermined expressions (eg, crying face detection, angry face detection, funny face detection, etc.).

The surrounding shooting request unit 142 is a function of sending a surrounding shooting request to the camera 3 to capture a camera image around the baby carriage 2 when the facial expression detection unit 141 detects the smiling face of the occupant of the baby carriage 2 from the camera image. The surrounding shooting request is sent from the terminal device 4 to the camera 3 via the communication unit 13 . When the camera 3 receives the surrounding shooting request, it takes a camera image around the stroller 2 and sends the camera image (the camera image around the stroller 2 ) back to the terminal device 4 .

The display processing unit 143 has a function of displaying the camera image (the camera image around the stroller 2 ) and the camera image of the occupant when the smile is detected on the same screen. Fig. 3 is an explanatory diagram showing an example of the same screen display. As shown in FIG. 3 , the display processing unit 143 aligns and displays the camera image of the occupant when the smile is detected and the camera image around the stroller 2 at that time on the same screen. Furthermore, in the example of FIG. 3, although the camera image of the occupant when the smiling face was detected and the camera image around the stroller 2 at that time are displayed vertically, as long as they are on the same screen, they can also be displayed by Other arrangement methods such as left-right arrangement. In addition, the display processing unit 143 may also have a function of connecting continuous camera images and playing (displaying) them as a moving image. In this case, the animation generated from the camera image of the occupant when the smiling face is detected and the animation generated from the camera image around the stroller 2 at that time can be displayed side by side on the same screen.

The continuation shooting request unit 144 is a function of sending a continuation shooting request to the camera 3 to continue taking camera images of the occupant of the baby carriage 2 while the expression detection unit 141 detects the smiling face of the occupant of the baby carriage 2 from the camera image. . The continuation shooting request is sent from the terminal device 4 to the camera 3 via the communication unit 13 . If the camera 3 receives the request to continue shooting, it will continue to take camera images of the occupant of the stroller 2 . The plurality of camera images captured in this way (camera images of the smiling faces of the occupants that continue to be captured) are sent back from the terminal device 4 in the direction that the camera 3 is facing.

The image selection unit 145 has a function of selecting a camera image having a degree of smile equal to or greater than a predetermined value from among the plurality of camera images (camera images of smiling faces of the occupants that are continuously photographed). For example, the image selection unit 145 detects smiling faces from a plurality of camera images, and calculates a smiling face degree for each detected smiling face. Then, the image selection unit 145 selects one camera image with the largest smile degree from the camera images whose smile degree is equal to or greater than a predetermined value. Furthermore, multiple camera images may be selected as long as the smile degree is higher than a predetermined value. Also, for the calculation of the degree of smile, a known method can be used. Similarly, the calculation of the degree of other predetermined expressions (such as the degree of crying face, degree of angry face, degree of funny face, etc.) can also be done by using known methods.

The 2nd control part 15 is the control part in order to carry out the secondary control about expression record function, possesses emotion analysis part 150, broadcast processing part 151, position acquisition part 152, record processing part 153, notification processing part 154, the direction toward An information acquisition unit 155 and a line-of-sight direction detection processing unit 156 .

The emotion analysis unit 150 has a function of analyzing the emotion of the occupant of the baby carriage 2 from the camera image captured by the camera 3 (camera image of the occupant of the baby carriage 2 ). For example, the emotion analysis unit 150 can analyze the emotions of the occupant of the baby carriage 2 as "happy", "anger", "sorrow", and "joy" from the camera image (the camera image of the occupant of the baby carriage 2). Which of the. Furthermore, the method of sentiment analysis can be done using known methods.

The playback processing unit 151 has a function of playing video or music according to the analysis result of the emotion analysis unit 150 (the emotion of the occupant of the stroller 2 obtained from the analysis result). For example, if the emotion of the occupant of the baby carriage 2 obtained from the analysis result is "happy" or "happy", the playback processing unit 151 plays an image or music with a cheerful atmosphere; If the mood is "anger" or "sorrow", then play heavy images or music.

The location obtaining unit 152 has a function of obtaining location information of the stroller 2 . For example, the location obtaining unit 152 has a GPS function for obtaining current location information (for example, longitude and latitude information) of the terminal device 4 . The position obtaining unit 152 obtains the position information of the terminal device 4 as the position information of the stroller 2 . Also, as mentioned above, since the camera 3 has a GPS function for obtaining position information (such as longitude and latitude information) showing the current position of the camera 3, the position obtaining unit 152 can also obtain the position information of the camera 3 from the camera 3 as The location information of the baby carriage 2 (the location information of the baby carriage 2 where the camera 3 is installed).

The recording processing unit 153 has a function of recording the position (for example, latitude and longitude) of the baby carriage 2 when the expression detection unit 141 detects the smile of the occupant of the baby carriage 2 from the camera image as the smile detection position. The information on the detection position of the smile is recorded in the storage unit 12 . Furthermore, the storage unit 12 may also store the smile detection positions of other users (the positions where the smiles of the occupants of the baby carriage 2 are detected).

The notification processing unit 154 has a function of notifying the user of the baby carriage 2 when the baby carriage 2 approaches the smile detection position. The notification processing unit 154 notifies the user of the stroller 2 when the current position of the stroller 2 (obtained from the camera 3 by the position acquisition unit 152 ) is close to the smile detection position (stored in the storage unit 12 ). For example, the user of the stroller 2 is notified when entering within a circular area with a predetermined radius centered on the smile detection position. Notification to the user of the stroller 2 can be performed by known methods such as sound, light emission, or vibration. In addition, the notification processing unit 154 may have a function of notifying the user of the stroller 2 corresponding to the analysis result of the emotion analysis unit 150 (the emotion of the occupant of the stroller 2 obtained as the analysis result). For example, the notification processing unit 154 may notify the nearest smile detection position when the emotion of the occupant of the stroller 2 obtained from the analysis result is "anger" or "sorrow". Notification to the user of the stroller 2 can be performed using, for example, the touch panel 10 (notification by screen display) or the speaker 11 (notification by audio).

The facing direction information acquisition unit 155 has a function of detecting the facing direction of the stroller 2 . As mentioned above, the camera 3 has a gyroscope function to acquire information (such as orientation information) indicating the direction in which the camera 3 is currently facing, and the direction information obtaining unit 155 obtains the direction information of the camera 3 from the camera 3 as The direction of the stroller 2 (the direction of the stroller 2 where the camera 3 is installed).

The line-of-sight direction detection processing unit 156 has the direction of the baby carriage 2 when the smiling face of the occupant of the baby carriage 2 is detected from the camera image by the expression detection unit 141 as the direction of sight of the occupant of the baby carriage 2 ( For example, the function of detecting directions such as east, west, north, south, etc.). Furthermore, the method for detecting the direction of sight can be done using known methods. For example, it is also possible to focus on the eyes of the occupant of the stroller 2 in the camera image, perform image analysis, etc., to calculate the line of sight of the occupant, and calculate the direction of the stroller 2 at that point in time (traveling) Direction) is taken into consideration, and the line of sight direction (for example, directions such as east, west, north, south, etc.) of the stroller 2 is detected. The information of the detected line-of-sight direction is stored in the storage unit 12.

Furthermore, the display processing unit 143 may also display the smile detection location on a map. FIG. 4 is an explanatory diagram showing an example of smile detection position display. In the example of FIG. 4, the smile detection position is displayed on the map as a "smile mark". Also, in FIG. 4 , the current position of the stroller 2 is shown by a circular mark, and the direction (traveling direction) of the baby carriage 2 is shown by a triangular mark. In the example shown in FIG. 4 , the direction in which the stroller 2 is facing is right on the map.

The operation of the facial expression recording system 1 constructed as above will be described below with reference to the flow charts in FIGS. 5 to 9 .

Fig. 5 is a flow chart of the same screen display process in the facial expression recording system 1 of the present embodiment. On the terminal device 4, the camera image (moving image) of the occupant of the stroller 2 captured by the camera 3 is always displayed in a real-time viewing manner. When using the terminal device 4 to perform the same screen display process, as shown in FIG. 5, first, the camera image is input from the camera 3 to the terminal device 4 (S10), and the baby carriage is executed from the input camera image. 2. The process of detecting the smiling face of the passenger (S11).

Then, when the smiling face of the occupant of the stroller 2 is detected from the camera image (S12), a surrounding shooting request is sent from the terminal device 4 to the camera 3 (S13), and the surrounding shooting request is sent to the camera 3 (S13). The camera images around the stroller 2 are captured by the camera 3 . When the camera image of the surroundings of the stroller 2 captured in this way is input to the terminal device 4 (S14), the camera image of the occupant of the stroller 2 (camera image of a smile) when a smiling face is detected is compared with the smiling face. The camera image around the stroller 2 at the time of detection is displayed on the same screen (S15).

FIG. 6 is a flow chart of the smile image selection process of the facial expression recording system 1 of the present embodiment. As shown in FIG. 6, in the case of using the terminal device 4 to perform smile image selection processing, first, the camera image is input from the camera 3 to the terminal device 4 (S20), and the baby is selected from the input camera image. The process of detecting the smiling faces of the occupants of the car 2 (S21).

Then, when the smiling face of the occupant of the stroller 2 is detected from the camera image (S22), a request to continue shooting is sent from the terminal device 4 to the camera 3 (S23), and the procedure for the occupant of the stroller 2 is repeated. Shooting of camera images (camera images of smiling faces). And if the smiling face of the occupant of the stroller 2 is not detected from the camera image (S22), then continue to select a smiling face whose degree of smile is more than a predetermined value from the captured camera images (camera images of smiling faces). The maximum camera image (S24). Furthermore, multiple camera images may be selected as long as the smile degree is higher than a predetermined value of the camera images.

Fig. 7 is a flow chart of video and music playback processing of the facial expression recording system 1 of the present embodiment. As shown in FIG. 7 , when using the terminal device 4 to perform video and music playback processing, if the camera image is input from the camera 3 to the terminal device 4 (S30), the emotion of the occupant of the stroller 2 (such as "happy" ", "anger", "sorrow", "happy") from the camera image is analyzed (S31). Then, images or music corresponding to the emotion of the occupant of the stroller 2 will be played (S32). For example, when it is analyzed that the emotion of the occupant of the stroller 2 is "happy" or "joyful", cheerful images or music are played. On the other hand, when it is analyzed that the emotion of the occupant of the stroller 2 is "anger" or "sorrow", heavy images or music are played.

FIG. 8 is a flow chart of smile detection position recording processing and proximity notification processing in the facial expression recording system 1 of the present embodiment. As shown in FIG. 8, when using the terminal device 4 to perform smile detection position recording processing and proximity notification processing, first, when the camera image is input from the camera 3, the position information of the camera 3 (baby) is obtained from the camera 3. position information of car 2) (S40). Then, if the smile of the occupant of the stroller 2 is detected from the input camera image (S41), the position of the stroller 2 at that time is recorded in the storage unit 12 as the smile detection position (S42).

Thereafter, (for example, when the baby carriage 2 goes out next time, etc.), when the camera image is input from the camera 3, the position information of the camera 3 (position information of the baby carriage 2) is obtained (S43), and the baby carriage 2 is compared with the recorded smile detection position, and if it is determined that the stroller 2 is close to the smile detection position (for example, within a predetermined radius of the smile detection position) (S44), the user of the stroller 2 is notified of the content (S45 ).

FIG. 9 is a flow chart of gaze direction detection processing of the facial expression recording system 1 of the present embodiment. As shown in FIG. 9 , when using the terminal device 4 to perform gaze direction detection processing, first, when a camera image is input from the camera 3 , the direction information of the stroller 2 is obtained from the camera 3 ( S50 ). Then, if the smiling face of the occupant of the baby carriage 2 is detected from the input camera image (S51), the direction in which the baby carriage 2 is facing at that time is regarded as the line of sight of the occupant of the baby carriage 2 (S52). It is detected, and the detected line-of-sight direction is recorded in the storage unit 12 (S53).

According to the facial expression recording system 1 of the present embodiment as described above, it is possible to record the smiling faces of infants riding in the stroller 2 while the stroller 2 is out and about.

That is, in this embodiment, the camera image can be taken of the occupant (for example, infant etc.) while the stroller 2 is going out. And, for example, if the occupant of the stroller 2 shows a smiling face while passing a certain spot, a camera image will be captured around the stroller 2 at that spot. Thereby, the camera image of the smiling face of the occupant of the baby carriage 2 and the camera image of the surroundings of the baby carriage 2 including the cause of the occupant's smiling face (objects the occupant likes, etc.) can be obtained.

Also, in this embodiment, it is possible to easily grasp the relationship between the smiling face of the occupant of the stroller 2 (such as an infant) and the cause of the occupant's smiling face (objects the occupant likes, etc.) on the same screen. Correspondence.

Also, in this embodiment, while the occupant of the stroller 2 (for example, an infant) shows a smiling face, the camera image (camera image of the smiling face) is continuously and continuously taken on the occupant, and the Choose a camera image with a higher degree of smile (of a better smile). In this way, better camera images of smiling faces can be obtained.

Also, in this embodiment, the emotion of the occupant of the stroller 2 (such as infants, etc.) can be analyzed from the camera image of the occupant, so that the video or music corresponding to the emotion is automatically played. Thereby, it is possible to play video or music matching the mood of the occupant while the stroller 2 is out and about.

Also, in this embodiment, the position (smile detection position) where the occupant of the stroller 2 (for example, an infant) shows a smiling face is recorded, and the user will be notified when approaching the position next time. Thereby, the user of the baby carriage 2 can grasp the place where the occupant of the baby carriage 2 smiles (the place that the occupant likes), and can know the content when the baby carriage 2 is out and about when approaching the place.

Also, in this embodiment, when the occupant of the stroller 2 (for example, an infant) shows a smiling face, it is possible to know the line of sight direction (observation direction) of the occupant. Thereby, the favorite object of the passenger can be specified.

As mentioned above, although the embodiment of this invention was described by way of example, the scope of this invention is not limited to these, It can change and deform|transform according to the objective within the range described in a claim.

For example, in the aforementioned embodiments, an example in which the camera 3 is attached to the stroller 2 has been described, but the scope of the present invention is not limited thereto. For example, the camera 3 may also be installed in a child car safety seat provided on a seat of a mobile body such as a car. Furthermore, when the child car safety seat does not have a component equivalent to the arm 5 of the baby carriage 2, the camera 3 can be arranged at an appropriate position outside the child car safety seat (such as the inner roof wall, the seat back of the front seat, etc.) , the inner surface of the pillar (pillar), the upper surface of the central storage box panel (console panel), etc.).

Also, the number of cameras 3 is not limited to one. In particular, when the camera 3 is installed in a car, in addition to a camera (in-vehicle camera) that captures a camera image of the expression (smile) of the occupant, a camera that captures a camera image around the vehicle ( external camera). Also, a plurality of cameras 3 may be used for each of the in-vehicle camera or the out-vehicle camera.

Thereby, the camera image can be taken of the occupant (such as an infant, etc.) sitting in the child car safety seat while going out in the car. And, for example, if the occupant shows a smiling face when passing a certain spot, the camera images around the vehicle at that spot are all captured. Thereby, the camera image of the smiling face of the occupant and the camera image of the surroundings of the vehicle including the cause of the occupant's smiling face (objects the occupant likes, etc.) can be obtained.

In the above, although the preferred embodiment of the present invention that can be considered at present has been described, it should be understood that various modifications can be made to the embodiment, and the present invention is intended to be included in the scope of the appended patent application. All variations within true spirit and scope.

(industrial availability)

As mentioned above, the facial expression recording system of the present invention has the effect of recording the smiling faces of infants sitting in the stroller while going out of the stroller, and can be applied to strollers for infants and the like, and has industrial potential. availability.

1‧‧‧Expression recording system (smile recording system)

2‧‧‧Prams

3‧‧‧Camera

4‧‧‧Terminal device

5‧‧‧arm

6‧‧‧Seating Department

10‧‧‧Touch panel

Claims (10)

An expression recording system, characterized in that it is equipped with: a camera installed in a stroller, capable of capturing camera images of the occupant of the stroller; and a terminal device held by the user of the stroller , can communicate with the above-mentioned camera; and the above-mentioned terminal device has: a data input unit, which inputs the above-mentioned camera image captured by the above-mentioned camera; The predetermined expression of the above-mentioned stroller is detected; the surrounding shooting request unit, when the above-mentioned predetermined expression is detected by the above-mentioned expression detection unit from the camera image of the occupant of the above-mentioned stroller captured by the above-mentioned camera, sends a picture to the above-mentioned camera Requirements for photographing the surroundings of the camera image around the stroller; a location information acquisition unit, which acquires the position information of the stroller; a record processing unit, which detects the position of the stroller when the predetermined expression is detected as an expression detection and the notification processing unit notifies the user of the baby carriage if the baby carriage approaches the expression detection position. The expression recording system according to claim 1, wherein the terminal device is equipped with a display processing unit, and the display processing unit compares the camera image of the occupant when the predetermined expression is detected with the image of the occupant when the predetermined expression is detected. The above surrounding camera images are displayed on the same screen. The facial expression recording system according to claim 1, wherein the above-mentioned terminal device is provided with: a continuation shooting request unit which, during the period when the predetermined facial expression is detected, responds to the above-mentioned The machine sends a request for continuing to capture the camera image of the occupant of the above-mentioned baby carriage; and the image selection unit selects the level of the predetermined expression from the camera image of the above-mentioned occupant that continues to be captured. camera image. The expression recording system according to claim 2, wherein the terminal device is provided with: a continuation shooting request unit, which sends a camera image of the occupant of the baby carriage to the above camera during the period when the predetermined expression is detected. A request to continue shooting; and an image selection unit, which selects a camera image with a predetermined expression level above a predetermined value from among the camera images of the above-mentioned occupant to continue shooting. The facial expression recording system according to any one of Claims 1 to 4, wherein the above-mentioned terminal device is equipped with: an emotion analysis part, which analyzes the emotion of the occupant of the above-mentioned baby carriage from the above-mentioned camera image; and a playback processing part , which plays video or music according to the emotion of the occupant of the baby stroller obtained as a result of the analysis. The facial expression recording system according to any one of claims 1 to 4, wherein the above-mentioned terminal device is equipped with: a direction information acquisition part, which obtains the direction information of the baby carriage from the above-mentioned camera; and a line-of-sight detection processing part wherein the direction in which the baby carriage is facing when the predetermined expression is detected is detected as the direction of sight of the occupant. The facial expression recording system according to claim 5, wherein the above-mentioned terminal device is equipped with: a direction information acquisition unit, which obtains the direction of the baby carriage from the above-mentioned camera information; and a line-of-sight detection processing unit that detects the direction in which the stroller is facing when the predetermined expression is detected as the line-of-sight direction of the occupant. A baby stroller, which is equipped with a camera capable of photographing a camera image of an occupant, wherein the camera can communicate with a terminal device held by the user of the baby stroller, and the terminal device is installed on the baby stroller When the predetermined expression of the person is detected from the camera image of the occupant of the baby stroller captured by the camera by the expression detection unit, a surrounding imaging request is sent to the camera, and the location information of the baby stroller is acquired. The camera captures camera images around the baby carriage according to the surrounding shooting request, and the terminal device records the position of the baby carriage when the predetermined expression is detected as the expression detection position, and if the baby carriage approaches The above-mentioned facial expression detection position is then notified to the user of the above-mentioned stroller. A facial expression recording program executed by a terminal device held by a user of a stroller, characterized in that the terminal device can communicate with a camera installed on the stroller, and the camera can take pictures of the stroller The camera image of the occupant of the stroller, the above-mentioned program causes the above-mentioned terminal device to perform the following processing: if the above-mentioned camera image captured by the above-mentioned camera is input, the predetermined expression of the occupant of the above-mentioned baby carriage is calculated from the above-mentioned camera image detection; and when the above-mentioned predetermined expression is detected by the expression detection part from the camera image of the occupant of the above-mentioned stroller captured by the above-mentioned camera, sending the above-mentioned The surrounding shooting request of the camera image around the stroller, and obtaining the position information of the stroller, the terminal device records the position of the stroller when the predetermined expression is detected as the expression detection position, and if the above When the baby carriage approaches the expression detection position, the user of the baby carriage is notified. A facial expression recording system is characterized in that it is equipped with: a camera installed on a moving body capable of capturing camera images of occupants of the moving body; and a terminal device held by the user and capable of communicating with the above-mentioned The camera communicates; and the above-mentioned terminal device is provided with: a data input part, which inputs the above-mentioned camera image captured by the above-mentioned camera; an expression detection part, which detects the predetermined expression of the occupant of the above-mentioned mobile body from the above-mentioned camera image a surrounding imaging request unit, which transmits to the camera to capture the surrounding area of the mobile body when the predetermined expression is detected by the expression detection unit from the camera image of the occupant of the mobile body captured by the camera The request for capturing the surroundings of the camera image; the position information acquisition unit, which acquires the position information of the above-mentioned moving body; the record processing unit, which records the position of the above-mentioned mobile body when the above-mentioned predetermined expression is detected as the expression detection position; and a notification processing unit that notifies a user of the mobile body when the mobile body approaches the expression detection position.

Images