JP5687889B2 - Vehicle interior monitoring device - Google Patents

Description

  The present invention relates to a vehicle interior monitoring apparatus that images and displays a vehicle interior of a vehicle.

  2. Description of the Related Art Conventionally, a vehicle interior monitoring device that captures an image of the interior of a vehicle interior with a camera disposed in the vehicle interior of a vehicle and displays the image on a vehicle-mounted monitor provided in a car navigation system or the like is known. Such a vehicle interior monitoring device plays a role of ensuring the safety of the passenger by displaying the state of the passenger at a position that is difficult for the driver who is driving to view.

  And in order to further ensure the safety of passengers, in recent years, when detecting abnormalities in the rear seat using various sensors, etc., or detecting that the child is crying, the state of such detection direction is shown. A vehicle interior monitoring device for displaying is also proposed (see, for example, Patent Document 1 or Patent Document 2).

JP 2006-347232 A JP 2008-221989

  However, when the conventional technology is used, there is a problem that the driver cannot immediately confirm the state of the passenger to be monitored. Specifically, in the prior art, since the state of the detection direction is displayed after detecting an abnormality such as a child crying, the driver who is a supervisor can only confirm the state passively. It was.

  For this reason, for example, when the child in the rear seat does not cry but is in a serious state, the rescue measures may be delayed. In addition, even if the state of the child's crying is displayed, if the child moves while crying, the child's state may not be displayed reliably.

  For these reasons, it has become a major issue how to realize a vehicle interior monitoring device that allows the driver to confirm the state of the passenger to be monitored actively and accurately. Such a problem is also a problem that occurs when the driver is on the side to be monitored.

  The present invention has been made in order to solve the above-described problems caused by the prior art, and is a vehicle interior in which the supervisor can confirm the state of the passenger to be monitored actively and accurately. An object is to provide a monitoring device.

To solve the above problems and achieve the object, the present invention provides a vehicle interior monitoring device for displaying images the vehicle interior, the personal information including the designation and characteristics for each occupant to the cabin Personal information registration means for registering, voice recognition means for recognizing the voice of the passenger, and a call to the specific passenger including the designation extracted from the voice recognized by the voice recognition means Based on the personal information, the feature acquisition means for acquiring the feature of the occupant corresponding to the designation, and the position of the occupant corresponding to the feature acquired by the feature acquisition means are searched for in the vehicle interior. a position search unit, and a display image generating means for generating a display image corresponding to searched the position by the position search unit, the personal information registration means, before The designation included in the voice after having converted to text, and registers in association with the feature of the passenger corresponding to the text and the designation.

According to the present invention, the registration of personal information including names and characteristics for each passenger in the passenger compartment , recognizing the voice of the passenger, and the specific boarding including the name extracted from the recognized voice Based on the call to the passenger, the characteristics of the passenger corresponding to the name are acquired from the personal information, the position of the passenger in the passenger compartment corresponding to the acquired characteristic is searched, and the display image corresponding to the searched position is displayed. For the registration and registration of the personal information, the name contained in the voice is converted into text, and the text is associated with the characteristics of the passenger corresponding to the name. There is an effect that the supervisor can confirm the state of the passenger to be monitored objectively and accurately.

FIG. 1 is a diagram showing an outline of a vehicle interior monitoring method according to the present invention. FIG. 2 is a block diagram illustrating a configuration of the vehicle interior monitoring apparatus. FIG. 3 is a diagram illustrating an arrangement example of a camera and a microphone. FIG. 4 is a diagram for explaining the operation up to registration of personal information. FIG. 5 is a diagram illustrating an example of setting trigger information. FIG. 6 is a diagram for explaining speech recognition processing in the speech recognition unit and call word identification processing in the call word identification unit. FIG. 7 is a diagram for explaining an example of a call and operations of a feature acquisition unit, a position search unit, and a display video generation unit associated therewith. FIG. 8 is a diagram for explaining the target search process in the position search unit and the display video generation process in the display video generation unit. FIG. 9 is a flowchart showing a processing procedure executed by the vehicle interior monitoring apparatus. FIG. 10 is a diagram showing a modification of registration of personal information. FIG. 11 is a diagram for explaining a calling example and a modified example of operations of the feature acquisition unit, the position search unit, and the display video generation unit associated therewith.

  Exemplary embodiments of a vehicle interior monitoring method according to the present invention will be described below in detail with reference to the accompanying drawings. In the following, the outline of the vehicle interior monitoring method according to the present invention will be described with reference to FIG. 1, and then an embodiment of the vehicle interior monitoring device to which the vehicle interior monitoring method according to the present invention is applied will be described with reference to FIGS. This will be described using.

  In the following, a case where the supervisor is mainly a driver will be described as an example. In the following, a case where the vehicle interior monitoring apparatus has an in-vehicle camera as an imaging unit and an in-vehicle monitor as a display unit will be described. Displaying in the form of “monitoring” shall be described.

  First, the outline | summary of the vehicle interior monitoring method which concerns on this invention is demonstrated using FIG. FIG. 1 is a diagram showing an outline of a vehicle interior monitoring method according to the present invention. 1A shows an arrangement example of seating positions in the passenger compartment 51 when the vehicle 50 is viewed from above, and FIG. 1B shows an overview of the vehicle interior monitoring method according to the present invention. For each.

  As shown in FIG. 1A, the vehicle 50 is provided with a passenger compartment 51. The passenger compartment 51 is provided with seating positions 51RR and 51RL in the rear part thereof and seating positions 51FR and 51FL in the front part thereof. It shall be.

  Further, it is assumed that the passenger a is seated at the seating position 51RR, the passenger b is seated at the seating position 51RL, the passenger c is seated at the seating position 51FR, and the passenger d is seated at the seating position 51FL. . The driver and the supervisor are passengers c. In addition, the "x" mark in a figure represents that the passenger's facial expression cannot be visually recognized from the passenger c.

  Here, according to the conventional technique, even if the passenger c wants to check the state of another passenger, the passenger c cannot immediately check the state of the passenger. Specifically, according to the conventional method, the corresponding seating position is not monitored unless an abnormality is detected at the seating position 51RR or 51RL. That is, the passenger c can confirm the state of the passenger a or b only passively.

  In addition, the state of the passenger a or b can be confirmed by the passenger c actively looking back and gazing at the rearview mirror. However, looking back on the passenger c, who is also the driver, and gazing at the room mirror may not be appropriate as a driving operation.

  In particular, it is extremely difficult to immediately confirm the state of the passenger a who sits in the blind spot of the passenger c, whether it is passive or active.

  Therefore, as shown in FIG. 1B, in the vehicle interior monitoring method according to the present invention, the position of the monitoring subject in the captured image is searched based on the active “calling” of the monitoring subject to the monitoring subject. And, it was decided to monitor the situation of the subject to be monitored immediately.

  First, in the vehicle interior monitoring method according to the present invention, personal information including “name” and “feature” is stored in the personal information 15a that is a database for all passengers a to d when getting into the vehicle 50 or the like. Register (see (B-1) in FIG. 1).

  Subsequently, for example, while the vehicle 50 is traveling, the passenger c makes a “call” to the passenger a with “Okay? Mr. a” (see (B-2) in FIG. 1). . Here, in the vehicle interior monitoring method according to the present invention, the “feature” of “Mr. a” is acquired from the personal information 15a based on the “call” “Okay?

  1B-3 shows a case where the “feature” of “Mr. a” is a face image of “Mr. a” (see the portion surrounded by the closed curve 1 in the figure). The “feature” is not limited to the face image. This point will be described later with reference to FIG.

  In the vehicle interior monitoring method according to the present invention, “Mr. a” that matches the “feature” is searched from the captured image 11a of the in-vehicle camera based on the acquired “feature” (see (B-4) in FIG. 1). ), The position of “Mr. a” in the captured image 11a is specified (see (B-5) in FIG. 1).

  In the vehicle interior monitoring method according to the present invention, the occupant c does not interfere with the driving operation based on the position of the identified “Mr. a” (see the portion surrounded by the closed curve 2 in FIG. 1 (B-5)). Is displayed on the display unit 13 (corresponding to the vehicle-mounted monitor) so that it can be surely seen without causing any trouble (see (B-6) in FIG. 1).

  In addition, although (B-6) of FIG. 1 has shown the example which expanded and displayed the position of "Mr. a", it does not need to be restricted to this enlarged display. This will be described later with reference to FIG. Details of a series of operations triggered by the above “calling” will be described later with reference to FIGS.

  As described above, in the vehicle interior monitoring method according to the present invention, based on the active “call” by the supervisor to the monitoring target person, the position of the monitoring target person in the captured image is searched and specified, and the monitoring target person is immediately detected. It was decided to monitor the situation.

  Therefore, according to the vehicle interior monitoring method according to the present invention, it is possible to cause the monitor to confirm the state of the passenger who is the monitoring target actively and accurately.

  In the description with reference to FIG. 1, the case where the monitoring person's position is searched and specified after the monitoring person's “calling” is used as a trigger to obtain the characteristics of the monitoring person has been described. In order to increase the position, each passenger's position may be determined in advance without using “calling” as a trigger. This point will be described later with reference to FIG.

  Below, the Example about the vehicle interior monitoring apparatus to which the vehicle interior monitoring method demonstrated using FIG. 1 is applied is described in detail. In the following embodiment, as in the case described with reference to FIG. 1, the case where the driver and the monitor are the passengers c and the monitoring target is the passengers a will be mainly described. In the following, the above “call” is referred to as “call word”.

  FIG. 2 is a block diagram illustrating a configuration of the vehicle interior monitoring apparatus 10 according to the present embodiment. In FIG. 2, only the components necessary for explaining the characteristics of the vehicle interior monitoring apparatus 10 are shown, and descriptions of general components are omitted.

  As shown in FIG. 2, the vehicle interior monitoring apparatus 10 includes a camera 11, a microphone 12, a display unit 13, a control unit 14, and a storage unit 15.

  The control unit 14 also includes a call word input unit 14a, a personal video input unit 14b, a registration information generation unit 14c, a personal information registration unit 14d, a voice recognition unit 14e, a call word identification unit 14f, and feature acquisition. 14 g, a position search unit 14 h, and a display video generation unit 14 i. The storage unit 15 stores personal information 15a and trigger information 15b.

  The camera 11 is a camera device that captures images of the passenger and the passenger compartment 51 (see FIG. 1), and corresponds to an in-vehicle camera. Note that the camera device that images each occupant when the personal information 15a is registered in advance (see FIG. 1) is not limited to the in-vehicle camera. In addition, an infrared camera may be used in preparation for the vehicle 50 traveling at night.

  The microphone 12 is an input device that inputs various sounds in the passenger compartment 51 including the “name” of the passenger and the “call word” uttered by the supervisor as acoustic signals.

When the personal information 15a is registered in advance, the microphone 12 calls an acoustic signal to the word input unit 14a, and the camera 11 outputs an imaging signal to the personal video input unit 14b. Further, when monitoring the person to be monitored, the microphone 12 outputs an acoustic signal to the voice recognition unit 14e, and the camera 11 outputs a captured video signal to the position search unit 14h.

  Here, the example of arrangement | positioning of the camera 11 and the microphone 12 in the vehicle interior 51 is demonstrated using FIG. FIG. 3 is a diagram illustrating an arrangement example of the camera 11 and the microphone 12. 3A shows an arrangement example of the camera 11, and FIG. 3B shows an arrangement example of the microphone 12.

  As shown in (A-a) of FIG. 3, the camera 11 can be disposed at the center of the front part of the passenger compartment 51 such as the center of the dashboard or the vicinity of the room mirror. The advantage of this arrangement is that the entire vehicle compartment 51 can be seen. The position is not limited to the above position as long as the entire vehicle compartment 51 can be seen, and can be installed at other positions.

  In addition, as shown in FIG. 3 (Ab), the camera 11 may be disposed in the center of the ceiling portion of the passenger compartment 51, for example, in the vicinity of the room lamp, in order to see the entire passenger compartment 51. . In such a case, it is more preferable to use a fish-eye lens or the like as the lens to be used so that it is easy to confirm the appearance including the expression of each passenger.

  Further, as shown in (Ac) of FIG. 3, a plurality of cameras 11 may be arranged. In addition, (Ac) of FIG. 3 shows an example in which the camera 11 is arranged in front of each passenger.

  In such a case, the camera 11 for the passenger sitting in the front part of the passenger compartment 51 is disposed, for example, in the vicinity of a dashboard or a front window. Moreover, the camera 11 for the passenger seated in the rear part of the passenger compartment 51 can be disposed on the back surface of the seating position 51FR or the seating position 51FL, for example, on the back surface of the headrests of the driver seat and the passenger seat.

  As shown in FIG. 3 (A-c), when a plurality of cameras 11 are arranged, when monitoring the state of the person to be monitored, the camera 11 that most easily confirms the state of the person to be monitored. There is an advantage that it is only necessary to select the captured image.

  This point will be described later with reference to FIG. In the following, it is assumed that the camera 11 is arranged in the arrangement example shown in FIG.

  Further, as shown in FIG. 3B-a, the microphone 12 can be disposed in front of the seating position 51FR, that is, in the vicinity of the dashboard or the front window in front of the driver's seat. In such a case, there is an advantage that it is possible to reliably collect “calling words” uttered by a driver who often becomes a supervisor.

  Further, as shown in (B-b) of FIG. 3, the microphone 12 may be disposed in the center of the ceiling portion of the passenger compartment 51, for example, in the vicinity of the room lamp. In such a case, the use of an omnidirectional microphone or the like has an advantage that the “calling word” can be reliably collected regardless of which passenger is the supervisor.

  In the following, it is assumed that the microphone 12 is arranged in the arrangement example shown in FIG.

  Returning to the description of FIG. 2, the display unit 13 will be described. The display unit 13 is a display device such as an in-vehicle monitor that displays the monitoring video of the monitoring subject generated and output by the display video generation unit 14i.

  The control unit 14 registers the personal information of each passenger, obtains the “feature” of the “calling” passenger from the personal information, and determines the position of the passenger corresponding to the “feature” from the camera 11. This is a processing unit that performs a process of searching for and specifying the captured video, generating a display video based on the specified position, and causing the display unit 13 to display the display video.

  In registering the personal information 15a, the call word input unit 14a extracts the “name” of each passenger based on the acoustic signal input from the microphone 12, and then uses the extracted “name” voice signal as registration information. It is a processing part which performs the process output to the production | generation part 14c.

  When registering the personal information 15a, the personal video input unit 14b generates image data representing “features” of each passenger based on the imaging signal input from the camera 11, and then generates the generated image data as registration information. It is a process part which performs the process output to the part 14c.

  The registration information generation unit 14c converts the text signal of the “name” input from the calling word input unit 14a into text, and then converts the textized “name” and image data input from the personal video input unit 14b. Is a processing unit that performs a process of generating registration information that is information in which The registration information generation unit 14c also performs a process of outputting the generated registration information to the personal information registration unit 14d.

  The personal information registration unit 14d is a processing unit that performs processing for storing the registration information input from the registration information generation unit 14c in the personal information 15a.

  Here, the operation up to the registration of the personal information 15a will be described with reference to FIG. FIG. 4 is a diagram for explaining the operation up to the registration of the personal information 15a. In the figure, the portion surrounded by the closed curve 14a is the operation in the calling word input unit 14a, the portion surrounded by the closed curve 14b is the operation in the personal video input unit 14b, and the portion surrounded by the closed curves 14c and 14d is. This corresponds to the operations in the registration information generation unit 14c and the personal information registration unit 14d, respectively. In addition, the portion surrounded by the closed curves 14c and 14d includes an example of registration of the personal information 15a.

  First, as shown in the portion surrounded by the closed curve 14 a, the calling word input unit 14 a is a “name” (name) of a registration target person uttered by a registrant (here, passenger c) input from the microphone 12. Here, the voice signal “Mr. a” is output to the registration information generation unit 14c.

  On the other hand, as shown in the portion surrounded by the closed curve 14b, the personal video input unit 14b receives the image data (here, the face image) of the person to be registered (here, the passenger a) input from the camera 11. After the generation, the generated image data is output to the registration information generation unit 14c.

  Then, as shown in the portions surrounded by the closed curves 14c and 14d, the registration information generation unit 14c and the personal information registration unit 14d convert the voice signal of “Mr. a” input from the call word input unit 14a into text. Then, it is associated with “the face image of the passenger a” input from the personal video input unit 14b and registered in the personal information 15a.

  In addition, there is an advantage that by making “name” such as “Mr. a” into text, any of the passengers can become a supervisor who utters “call word”.

  Here, a registration example of the personal information 15a shown in the portion surrounded by the closed curves 14c and 14d will be described. The personal information 15a includes a “No.” item, a “name” item, and a “feature” item.

  The “No.” item is an item in which a record number uniquely indicating each record registered for each “name” is stored. The “name” item is an item in which the “name” of each passenger converted into text as described above is stored. The “feature” item is an item in which data (in this case, a face image) indicating the characteristics of the passenger corresponding to each “name” is stored.

  As shown in the figure, the name stored in the “name” item may be of any type such as “name”, “nickname”, or “honor”. For example, FIG. 4 shows an example in which “name” is registered in various ways such as “Mr. a”, “Mr. b”, “c”, “Mr. d”, and “d Senior”. Further, the same person may be registered with different “names” such as “Mr. d” or “d senior”.

  Returning to the description of FIG. 2, the voice recognition unit 14e will be described. The voice recognition unit 14e extracts the voice uttered by the “passenger” based on the acoustic signal input from the microphone 12 and monitors the voice signal of the extracted voice when performing monitoring of the monitoring subject. It is a processing part which performs the process output to the part 14f.

  The calling word identifying unit 14f is a processing unit that performs processing for converting the “calling word” uttered by the monitor toward the monitoring target person based on the voice signal input from the voice recognition unit 14e.

  Further, the call word identification unit 14f disassembles the text “call word” into words other than “name” and “name”, and then the words other than the “name” become “triggers” related to monitoring of the monitoring subject. The process of determining whether or not the word is “ At the time of such determination, the trigger information 15b in the storage unit 15 is referred to.

  In addition, the call word identification unit 14f also performs a process of outputting a “name” to the feature acquisition unit 14g when the monitoring start of the monitoring subject is determined by the determination.

  Further, the call word identification unit 14f also performs a process of terminating the monitoring when the monitoring target person's “end” of the monitoring is determined by the determination.

  Here, the trigger information 15b of the storage unit 15 referred to by the calling word identification unit 14f will be described with reference to FIG. FIG. 5 is a diagram illustrating a setting example of the trigger information 15b.

  As illustrated in FIG. 5, the trigger information 15b is information in which a word to be a “trigger” is set in advance for monitoring of the monitoring target person. The trigger information 15b includes a “category” item and a “word” item.

  The “category” item is an item in which a category value that roughly classifies operations related to monitoring by the monitoring target person is stored. For example, as shown in FIG. 5, by storing the category value “start”, it is possible to set a word (hereinafter referred to as “start word”) for starting monitoring of the monitoring subject.

  Further, by storing the category value “end”, it is possible to set a word (hereinafter referred to as “end word”) for ending the monitoring of the monitoring target person.

  The “word” item is an item in which a specific example of a word to be “trigger” is stored. For example, FIG. 5 shows an example in which “video display”, “video start”, “OK?”, “Wake up?”, Etc. are set as “start words”.

  In addition, an example is shown in which “display end”, “video stop”, “more good”, and the like are set as the “end word”. In the following description, it is assumed that the trigger information 15b is set in this setting example, and “OK” is used as the “start word”, and “OK” is used as the “end word”. .

  Next, the speech recognition process in the speech recognition unit 14e and the call word identification process in the call word identification unit 14f will be described with reference to FIG. FIG. 6 is a diagram for explaining speech recognition processing in the speech recognition unit 14e and calling word identification processing in the calling word identification unit 14f.

  In the figure, the portion surrounded by the closed curve 14e corresponds to the operation in the speech recognition unit 14e, and the portion surrounded by the closed curve 14f corresponds to the operation in the calling word identification unit 14f.

  First, as shown in (1) of FIG. 6, the voice recognition unit 14 e outputs a voice signal of voice uttered by the passenger c or the like input from the microphone 12 to the calling word identification unit 14 f. Note that FIG. 6 (1) shows an example in which a voice of “calling word” “Okay? Mr. a” is input from the microphone 12, and in the following, the description will be made on the assumption of the “calling word”. Proceed.

  Then, as shown in (2) of FIG. 6, the calling word identification unit 14f converts the “calling word” “Okay? Mr. a” into text based on the voice signal input from the voice recognition unit 14e.

  Then, as shown in (3) of FIG. 6, the calling word identifying unit 14 f is a word other than “named” and “named” as “named okay? Decompose into "Is it all right?"

  Here, the calling word identifying unit 14f determines whether or not “OK”, which is a word other than “name”, matches a word “trigger” related to monitoring of the monitoring target set in advance in the trigger information 15b. Is determined.

  As described above, the word “OK” is assumed to be set as the “start word” in the trigger information 15b, and therefore the calling word identification unit 14f is the word “name”. “Mr. a” is output to the feature acquisition unit 14g (see “to feature acquisition unit” shown in FIG. 6).

  Also, if the word “OK” is set as the “end word”, or if the word “OK” is not set, the vehicle interior monitoring apparatus 10 determines that the voice recognition unit 14e Transfer control to action.

  Returning to the description of FIG. 2, the feature acquisition unit 14g will be described. The feature acquisition unit 14g is a processing unit that performs processing for acquiring, from the personal information 15a of the storage unit 15, the “feature” of the passenger that matches the “name” input from the calling word identification unit 14f. The feature acquisition unit 14g also performs a process of outputting the acquired “feature” to the position search unit 14h.

  The position search unit 14h is a processing unit that performs a process of searching and specifying the position of the passenger corresponding to the “feature” input from the feature acquisition unit 14g in the captured image of the camera 11. Such search and identification are performed by comparing the captured image of the camera 11 with image data representing “features”. The position search unit 14h also performs a process of outputting the specified position to the display video generation unit 14i.

  The display video generation unit 14i is a processing unit that performs a process of generating a display video in which the monitor can surely see the state of the monitoring target person based on the position of the monitoring target person input from the position search unit 14h.

  The display video generation unit 14i also performs a process of outputting the generated display video to the display unit 13 corresponding to the in-vehicle monitor. Details of the display video generation processing in the display video generation unit 14i will be described later with reference to FIG.

  Here, an example of a call and the operations of the feature acquisition unit 14g, the position search unit 14h, and the display video generation unit 14i associated therewith will be described with reference to FIG. FIG. 7 is a diagram for explaining an example of a call and operations of the feature acquisition unit 14g, the position search unit 14h, and the display video generation unit 14i associated therewith. The horizontal axis t shown in FIG. 7 represents the passage of time.

  As shown in (1) or (2) of FIG. 7, when a voice signal such as “I am hungry. Mr. a” or “I feel sick? If the word “I feel hungry” or “I feel sick?” Is not set in the trigger information 15b, the identification unit 14f does not output “Mr. a” corresponding to “name” to the feature acquisition unit 14g.

  On the other hand, as shown in (3) of FIG. 7, when a voice signal of “Okay? Mr. a” is input from the voice recognition unit 14e, the calling word identification unit 14f displays the word “OK”? If it matches the “start word” in the trigger information 15b, “san” corresponding to “name” is output to the feature acquisition unit 14g.

  Therefore, assuming the setting example of FIG. 5 described above, in the example shown in FIG. 7, until the time t3, the non-operating section of the feature acquisition unit 14g, the position search unit 14h, and the display video generation unit 14i Become. That is, the monitoring target person is not monitored until time t3.

  Further, as shown in FIG. 7, after the time t3, it becomes an operation section of the feature acquisition unit 14g, the position search unit 14h, and the display video generation unit 14i. That is, monitoring with the passenger a as a monitoring subject is performed.

  During such an operation period, for example, a voice signal (refer to (4) in FIG. 7) that does not include “name” such as “I feel sick?” Or a voice signal (“Mr. a” who has already been input) ( Even if (2) and (5) in FIG. 7 are input, the monitoring of the passenger a is continued.

  Then, as shown in (6) of FIG. 7, if a voice signal “I'm fine” that matches the “end word” of the trigger information 15 b is input, the feature acquisition unit 14 g and the position search unit again. 14h and the non-operation period of the display video generation unit 14i. Therefore, the monitoring of the passenger a ends at time t6.

  Next, the target search process in the position search unit 14h and the display video generation process in the display video generation unit 14i will be described with reference to FIG. FIG. 8 is a diagram for explaining target search processing in the position search unit 14h and display video generation processing in the display video generation unit 14i.

  8A shows a case where the monitoring subject is enlarged and FIG. 8B shows a case where cameras # 1 to # 4 which are a plurality of cameras 11 are arranged. ing.

  As illustrated in FIG. 8A-1, the position search unit 14h is, for example, a monitoring target in the captured video 11a based on a face image representing the “feature” of the monitoring target input from the feature acquisition unit 14g. The person's position (refer to the part surrounded by the closed curve 3 in the figure) is searched and specified.

  And as shown to (A-2) of FIG. 8, the display image | video production | generation part 14i produces | generates the display image | video which expands and displays the part enclosed by this closed curve 3, for example, displays on a vehicle-mounted monitor etc. It can be displayed on the unit 13.

  Therefore, in the captured image 11a, even when the monitoring subject is so small that the state cannot be well confirmed, it is possible to reliably confirm the state of the monitoring subject.

  Further, as illustrated in FIG. 8B-1, when a plurality of cameras # 1 to # 4 are disposed in accordance with the above-described arrangement example of FIG. May specify, for example, the camera 11 that is capturing the video that is most easily visible to the person being monitored. 8B-1 shows an example in which the position search unit 14h specifies the camera # 1.

  In such a case, as shown in (B-2) of FIG. 8, the display video generation unit 14 i may select the captured video of the camera # 1 and display it on the display unit 13. In this case, the display video generation unit 14i only needs to switch the cameras # 1 to # 4 without newly generating a display video, so that the immediacy of monitoring of the monitoring subject can be improved.

  Although not shown in FIG. 8, the display video generation unit 14 i may change the display video generated according to the vehicle speed of the vehicle 50. For example, when the vehicle speed is 0, only a face image may be generated as a display video as shown in FIG. When the vehicle speed is higher than 0, a display image combining the navigation screen and the face image may be generated.

  Thereby, even if the supervisor is the driver, the state of the monitoring subject can be confirmed while displaying the navigation information for assisting the driver. That is, it is possible to ensure the safety of the person being monitored while improving the convenience of driving.

  Returning to the description of FIG. 2, the storage unit 15 will be described. The storage unit 15 is a storage unit configured by a storage device such as a hard disk drive, a nonvolatile memory, or a register, and stores personal information 15a and trigger information 15b.

  Since the personal information 15a has already been described with reference to FIG. 4 and the trigger information 15b has been described with reference to FIG. 5, description thereof is omitted here.

  Next, a processing procedure executed by the vehicle interior monitoring apparatus 10 will be described with reference to FIG. FIG. 9 is a flowchart showing a processing procedure executed by the vehicle interior monitoring apparatus 10. In this flowchart, registration of personal information 15a (corresponding to step S101) and monitoring of a monitoring target person (corresponding to steps S102 to S109) are shown as a series of operations.

  Further, in steps S102 to S109, a processing procedure from “start” to “end” for one monitoring is shown.

  As shown in FIG. 9, first, registration information including “name” and “feature” generated by the registration information generation unit 14c based on the outputs of the calling word input unit 14a and the personal video input unit 14b is registered as personal information. The unit 14d registers in the personal information 15a that is a database (step S101).

  Subsequently, during traveling of the vehicle 50, the voice recognition unit 14e recognizes the voice of the passenger from the sound in the passenger compartment 51 collected by the microphone 12 (step S102). Then, the calling word identifying unit 14f identifies the “calling word” of the supervisor from the voice (step S103).

  Then, the calling word identification unit 14f analyzes the “calling word” and determines whether or not the “end word” is included (step S104).

  If it is determined that the “end word” is not included (step S104, Yes), the calling word identification unit 14f determines whether the “start word” is included (step S105). On the other hand, when the determination condition of step S104 is not satisfied (step S104, No), the vehicle interior monitoring apparatus 10 ends the process for one monitoring.

  Subsequently, when it is determined that the “start word” is included (step S105, Yes), the feature acquisition unit 14g displays the “feature” of the passenger that matches the “name” in the “call word”. Obtained from the personal information 15a (step S106).

  When the determination condition in step S105 is not satisfied (step S105, No), the vehicle interior monitoring apparatus 10 repeats the processing from step S102.

  Then, the position search unit 14h searches and specifies the position of the passenger corresponding to the “feature” acquired by the feature acquisition unit 14g based on the captured image of the camera 11 (step S107). Then, the display video generation unit 14i generates a display video of the passenger who is the monitoring subject based on the searched and specified position (step S108), and then outputs the display video to the display unit 13 (step S108). S109).

  Then, the vehicle interior monitoring apparatus 10 repeats the processing from step S102 until the “call word” includes the “end word”.

  As described above, in this embodiment, the vehicle interior monitoring device is configured such that the personal information registration unit registers in advance personal information related to each passenger. In addition, the voice recognition unit recognizes the passenger's voice from the sound in the passenger compartment, the call word identification unit identifies and analyzes the supervisor's “call word” from the passenger's voice, and the feature acquisition unit, The “feature” of the passenger that matches the “name” of the analyzed monitoring subject is acquired, and the position search unit searches and identifies the position of the passenger corresponding to the “feature”, and the display video generation unit The vehicle interior monitoring device is configured to generate and display a display image of the person to be monitored based on the specified position.

  Therefore, it is possible to cause the supervisor to confirm the state of the passenger who is the subject of monitoring actively and accurately.

  By the way, in the above-described embodiment, the case where the “feature” included in the personal information 15a is a face image of each passenger has been described. However, there may be cases where the faces of the passengers are similar, such as when a family boardes the vehicle 50 together.

  In such a case, the above-described “feature” is not limited to the face image, and other information is used as the “feature”, so that the position and the position of the passenger who is the monitoring target can be accurately searched. . Therefore, such a modification will be described below with reference to FIG.

  FIG. 10 is a diagram showing a modification of registration of the personal information 15a. 10A shows a modification based on a registered captured image, and FIG. 10B shows a modification that does not use a captured image.

  As illustrated in FIG. 10A, the personal information 15aa according to the modification may include an item for storing data other than the above-described face image in the “feature” item. For example, FIG. 10A shows an example in which the “feature” item further includes a “face” item, a “color scheme” item, and a “clothing design” item.

  Here, the “face” item, the “color scheme” item, and the “clothing design” item are items based on the captured image for registration. The “face” item is an item for storing a passenger's face image, as in the above-described embodiment.

  The “color scheme” item is an item for storing a color scheme distribution calculated based on the captured image. In FIG. 10A, an example in which a rough color distribution is described by color name is shown for easy understanding, but RGB values or the like can be used for actual stored data. .

  The “clothing design” item is an item for storing a design pattern of clothing worn by each passenger analyzed based on the captured image. For example, the example shown in FIG. 10A shows an example in which a solid stripe design pattern is stored only in the “clothing design” item of the passenger c whose “name” is “c”. ing.

  Therefore, in this example, it is possible to search and specify the position of the passenger c using only the storage data of the “clothing design” item without using the storage data of the “face” item and the “color scheme” item. Become. Of course, the position of the passenger may be searched and specified by combining the stored data of each item.

  Moreover, it is good also as identifying each passenger by using an IC tag, for example, without using the picked-up image for registration like the personal information 15ab which concerns on the modification shown to (B) of FIG. .

  Specifically, an IC tag is distributed to each passenger when boarding the vehicle 50, and each “name” is associated with each “IC tag ID” when registering the personal information 15ab. You can do that. In such a case, the processing load on the processing of the image data, which tends to increase the amount of data, can be reduced, so that it is possible to improve the immediacy of monitoring of the monitoring subject.

  Further, in the above-described embodiment, during the series of operations triggered by the supervisor's “calling word”, the characteristics of the passenger who is the monitoring target are acquired, and the position is searched and specified based on the acquisition of such characteristics. Explained the case of performing. However, the temporary position of each passenger may be estimated in advance without using the “calling word” as a trigger.

  Therefore, in the following, such a modification will be described with reference to FIG. FIG. 11 is a diagram for explaining a modification example of the operation of the calling example and the accompanying feature acquisition unit 14g, position search unit 14h, and display video generation unit 14i.

  In FIG. 11, the operation interval of the feature acquisition unit 14 g and the position search unit 14 h is indicated as “operation interval A”, and the operation interval of the display video generation unit 14 i is indicated as “operation interval B”. In the description using FIG. 11, portions different from those in FIG. 7 will be mainly described, and description of overlapping portions will be omitted or only a brief description will be given.

  As shown in FIG. 11, in the vehicle interior monitoring apparatus according to the modification, the feature acquisition unit 14g, the position search unit 14h, and the like can be operated without using the “calling word” as a trigger.

  Specifically, as shown in (0) of FIG. 11, the feature acquisition unit 14g and the position search unit 14h are operated in the background after registration of personal information at the time of “ride” (see t0 in the figure). (Refer to “Operation section A” in FIG. 11).

  Based on the “feature” of each passenger registered in the personal information 15a, the feature acquisition unit 14g and the position search unit 14h in the background operation tentatively determine the position of each passenger, for example, when the vehicle 50 starts to travel. Estimated as position.

  Although not shown in the drawings, when an event occurs such as when the passenger changes his / her seat, the position of each passenger is estimated as a provisional position again.

  Then, as shown in (3) of FIG. 11, when the calling word identifying unit 14f identifies the “calling word” indicating the start of monitoring such as “Okay? Mr. a”, the display image generating unit 14i The display video generation process is started based on the temporary position nearest to the passenger corresponding to the monitoring subject.

  Then, as shown in (6) of FIG. 11, the display video generation unit 14i displays the display video until the call word identification unit 14f identifies the “call word” indicating the end of the monitoring such as “It is OK”. The generation process is continued (see “operation section B” in FIG. 11).

  As described above, since the feature acquisition unit 14g, the position search unit 14h, and the like are operated in the background, the processing load for searching and specifying the position of the monitoring target person can be reduced. It becomes possible to improve the nature.

  In the example shown in FIG. 11, the “operation section A” is from the time of “boarding” to the time of identification of “more good” indicating the end of monitoring, but after t6 in the figure. It may be until “get off” or “engine stop”.

  In the above-described embodiments and modifications, the case where the position of the person to be monitored is specified based on “name” has been described, but the present invention is not limited to such a case. For example, although not shown in the figure, based on the directivity of the voice in the passenger compartment, it is analyzed whether the passenger is a “calling word” for the seated passenger and monitoring the passenger as a monitoring subject. Also good.

  As described above, the vehicle interior monitoring apparatus according to the present invention is useful when it is desired to make the supervisor confirm the state of the passenger to be monitored actively and accurately, and in particular, the blind spot for the driver. It is suitable for application to vehicle interior monitoring devices such as passenger cars for many people.

DESCRIPTION OF SYMBOLS 10 Car interior monitoring apparatus 11 Camera 11a Image picked-up image 12 Microphone 13 Display part 14 Control part 14a Calling word input part 14b Personal image input part 14c Registration information generation part 14d Personal information registration part 14e Voice recognition part 14f Calling word identification part 14g Feature acquisition Unit 14h position search unit 14i display image generation unit 15 storage unit 15a personal information 15aa, 15ab personal information 15b trigger information 50 vehicle 51 vehicle compartment

Claims (5)

A vehicle interior monitoring device that images and displays a vehicle interior,
Personal information registration means for registering personal information including names and features for each passenger in the passenger compartment ;
Voice recognition means for recognizing the voice of the passenger;
Feature acquisition for acquiring the characteristics of the occupant corresponding to the designation from the personal information based on a call to the specific occupant including the designation extracted from the voice recognized by the voice recognition means Means,
Position search means for searching a position of the passenger corresponding to the characteristic acquired by the characteristic acquisition means in the vehicle interior;
Display video generation means for generating a display video corresponding to the position searched by the position search means ,
The personal information registration means includes:
The vehicle interior monitoring device, wherein the name contained in the voice is converted into text, and the text and the feature of the passenger corresponding to the name are registered in association with each other . Further comprising word identifying means for extracting the call from the voice;
The word identification means is
The extracted call is converted into text, the text is separated into the name and a word other than the name, and when the word indicates the start of display, the characteristics of the passenger corresponding to the name the vehicle interior monitoring device according to claim 1, characterized in that obtained for the characteristic acquisition means. The position search means includes
If the personal information has already been registered by the personal information registration means, the position for each passenger is searched based on the characteristics included in the personal information even before the call is extracted. vehicle interior monitoring device according to claim 1 or 2, characterized in that. The display video generation means includes
By enlarging the captured image in the retrieved the position by the position search unit, the vehicle interior monitoring device according to claim 1, 2 or 3, characterized in that to generate the display image. The display video generation means includes
By selecting the captured image in has been the position searched by the position search unit, the vehicle interior monitoring device according to claim 1, 2 or 3, characterized in that to generate the display image.

Images