KR102534451B1 - Rear-seat conversation method linked with camera - Google Patents

Abstract

The present invention relates to a back seat conversation method in which a camera is interlocked.
The present invention includes the steps of activating a rear seat conversation mode in which a control unit mounted in a vehicle and supporting rear seat conversation supports communication between a driver of a vehicle and a first occupant in a rear seat of the vehicle using a microphone and a speaker installed in the vehicle; The step of specifying the first occupant among the at least one occupant in the rear seat by the control unit, the step of photographing the first occupant by a camera that is interlocked with the control unit and photographs the rear seat, and is provided in a driver's seat in the vehicle, and displaying, by an image display device linked to the control unit, an image of the first occupant.

Description

Rear-seat conversation method linked with camera}

The present invention relates to a camera-linked rear-seat conversation method, and more particularly, to a rear-seat conversation method in which a camera provided inside a vehicle is interlocked when communication is desired between a driver and a passenger inside a vehicle.

With the development of communication technology and voice processing technology, many improvements have also been made in hands-free systems for providing free calls to drivers or passengers in vehicles regardless of vehicle operation.

Recently, research is being conducted on a system capable of making a call between a driver and a passenger for communication between a driver and a passenger in a vehicle. In some vehicles, a system that provides a call between a driver and a passenger in a vehicle is being released.

In such a conventional rear-seat conversation mode system, when a driver selects a seat to make a call in conducting a call between a passenger and a driver, a conversation mode with a passenger seated in the corresponding seat is possible.

However, with such a conventional system, there is a problem in that it is difficult to perform a conversation mode with a desired passenger in a rear seat conversation mode unless the driver recognizes the seat of the passenger who wants to talk in advance.

Korean Patent Application 10-2007-0031230

The problem to be solved by the present invention is to provide a camera-linked rear-seat conversation method.

More specifically, the problem to be solved by the present invention is to provide a rear seat conversation method in which a camera provided inside a vehicle is interlocked when communication is desired between a driver and a passenger inside a vehicle, and its purpose is.

In the camera-linked rear seat conversation method according to the first embodiment of the present invention, a control unit mounted in a vehicle and supporting rear seat conversation uses a microphone and a speaker installed in the vehicle to communicate with a driver and a first occupant in the back seat of the vehicle. Activating a rear seat conversation mode supporting communication, specifying the first occupant among at least one occupant in the rear seat by the control unit, using a camera interlocked with the control unit to photograph the rear seat of the first occupant and displaying an image of the first occupant by an image display device provided in a driver's seat in the vehicle and interlocked with the control unit.

In the activating step, it is determined that the conversation between the driver and the at least one passenger detected by the microphone is being attempted, or the at least one passenger's mouth is captured by the camera. When it is determined that the rear seat conversation is being attempted through analysis, the rear seat conversation mode may be activated when a driver's seat conversation switch provided around the driver's seat is turned on.

When there are a plurality of microphones installed in the vehicle, in the step of activating, the conversation between the driver and at least one passenger, which is input from the plurality of microphones, is analyzed to analyze the position of the ignition in the vehicle to attempt the conversation in the rear seat. It can be judged whether or not

In the activating step, when a conversation between the driver and the at least one passenger is detected by the microphone, the camera photographs the at least one passenger, analyzes the mouth shape of each of the at least one passenger, and then the conversation in the rear seat. It can be judged whether it is an attempt for .

In the step of specifying the first occupant, the first occupant is identified by analyzing the mouth shape of each of the at least one occupant photographed by the camera, or when there are a plurality of microphones installed in the vehicle, the input of the plurality of microphones is performed. If a conversation switch is provided for specifying the first passenger by analyzing an ignition position from a conversation between the driver and at least one passenger or activating the rear seat conversation mode for each rear seat in which the passenger is seated, it is provided for each rear seat. The first occupant may be identified by analyzing a position of a turned-on dialog switch among a plurality of turned-on dialog switches.

The step of photographing the first occupant is the step of the camera primarily photographing the surrounding area of the first occupant including the first occupant, and the image analysis of the surrounding area by the controller and the specified second occupant. The method may include photographing the face of the first passenger secondarily by the camera.

A camera-linked rear-seat conversation method according to a second embodiment of the present invention includes the steps of turning on a first rear-seat conversation switch provided around the rear seat of a first occupant in the vehicle for a rear-seat conversation mode, the vehicle photographing the first occupant located around the first rear-seat conversation switch by a camera provided therein, displaying an image captured by the camera on an image display device installed in a driver's seat in the vehicle, and activating the rear seat conversation mode between the driver and the first passenger in the vehicle through a driver's seat microphone and a driver's seat speaker provided in the vehicle, and a first rear seat microphone and a first rear seat speaker provided around the rear seat of the first passenger; includes

The step of photographing the first occupant is a step in which the camera primarily photographs the surrounding area of the first rear seat conversation switch including the first occupant, and a determination unit receiving an image of the surrounding area from the camera Analyzing the image of the surrounding area, analyzing the movement of at least one occupant located in the surrounding area of the first rear seat conversation switch, and specifying the first occupant among the at least one occupant; It may include photographing the face of the first occupant.

In the step of specifying the first occupant, when there is only one occupant located in the area around the first rear-seat conversation switch, the determination unit identifies the one occupant as the first occupant, and the area around the first rear-seat conversation switch. When there are a plurality of occupants positioned on the occupant, the determination unit may identify the first occupant from among the plurality of occupants by analyzing motions or mouth shapes of each of the plurality of occupants.

The method for specifying the first occupant among the plurality of occupants includes the determination unit extracting a contour line for at least one of postures and mouth shapes of each of the plurality of occupants from an image of an area around the first rear-seat conversation switch; , It is possible to determine whether or not the first occupant is the first occupant by analyzing the movement of the contour line of each of the plurality of occupants.

Prior to turning on the first rear seat conversation switch, turning on a driver's seat conversation switch provided around the driver's seat in the vehicle for the rear seat conversation mode, and the driver's voice input through the driver's seat microphone in the vehicle The step of outputting through all of the plurality of rear seat speakers provided on both sides may be further included.

In the photographing step, if the camera captures a second occupant riding in the rear seat next to the first occupant, not the first occupant, the camera recognizes the driver's gesture and then the camera captures the first occupant again. can be filmed

In the step of activating the rear seat conversation mode, rear seat speakers other than the first rear seat speaker among the plurality of rear seat speakers may be turned off.

Between the displaying and activating the rear seat conversation mode, the first passenger's voice input through the first rear seat microphone is output through the driver's seat speaker and the interior of the vehicle for the rear seat conversation mode. The method may further include turning on a driver's seat conversation switch provided around the driver's seat.

In the step of activating the rear seat conversation mode, rear seat speakers other than the first rear seat speaker among a plurality of rear seat speakers provided on both sides of the vehicle may maintain a turned off state.

In the camera-linked rear seat conversation method according to an example of the present invention, when a first passenger who wants to talk to a driver turns on a first rear seat conversation switch, a camera provided in a vehicle photographs the first passenger and displays the image on the driver's seat image display device. In one state, a conversation mode in the rear seat may be performed between the driver and the first passenger.

1 is a diagram for explaining a first embodiment of a rear seat conversation system according to the present invention.
FIG. 2 is a diagram for explaining an example of a rear seat conversation method performed by the rear seat conversation system according to the first embodiment of the present invention shown in FIG. 1 .
FIG. 3 is an example for describing an example of a method of photographing a first occupant in FIG. 2 in more detail.
4 is a diagram for explaining a second embodiment of a rear seat conversation system according to the present invention.
FIG. 5 is a diagram for explaining an example of a rear seat conversation method performed by the rear seat conversation system according to the second embodiment of the present invention shown in FIG. 4 .
FIG. 6 is a diagram for explaining an example of a method of photographing a first occupant in FIG. 5 .
FIG. 7 is a diagram for explaining an example of a method of specifying a first occupant in FIG. 6 .
8 is a diagram for explaining a first modified example of a rear seat conversation method performed by the rear seat conversation system according to the second embodiment of the present invention.
9 is a diagram for explaining a second modified example of a rear seat conversation method performed by the rear seat conversation system according to the second embodiment of the present invention.

Hereinafter, the embodiments disclosed in this specification will be described in detail with reference to the accompanying drawings, but the same or similar elements are given the same reference numerals regardless of reference numerals, and redundant description thereof will be omitted. In addition, in describing the embodiments disclosed in this specification, if it is determined that a detailed description of a related known technology may obscure the gist of the embodiment disclosed in this specification, the detailed description thereof will be omitted.

Terms including ordinal numbers, such as first and second, may be used to describe various components, but the components are not limited by the terms. Terms are only used to distinguish one component from another.

Singular expressions include plural expressions unless the context clearly dictates otherwise.

In this application, each step described can be performed regardless of the listed order, except for the case where it must be performed in the listed order due to a special causal relationship.

In this application, terms such as "comprise" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded.

Hereinafter, the present invention will be described with reference to the accompanying drawings.

1 is a diagram for explaining a first embodiment of a rear seat conversation system according to the present invention.

The rear seat conversation method according to the present invention may be performed by interlocking a camera and may be performed by a rear seat conversation system.

The rear seat conversation system according to an example of the present invention may perform a rear seat conversation mode enabling communication between the driver D and the plurality of passengers C1 to Cn within the vehicle 10 .

The rear seat conversation mode refers to at least one occupant (e.g., Cn-1) may mean a state in which communication can be performed.

The rear seat conversation system is provided in the vehicle 10, and as shown in FIG. can include

As shown in FIG. 1 (a), the driver terminal 200 is provided around the driver's seat DS, and as shown in FIG. A driver's seat conversation switch 240 may be included.

The image display device 210 may include a display device provided in a driver's seat in a vehicle, interlocked with the control unit, displaying an image or video, and may include an input means such as a touch screen.

The driver's seat microphone 220 can receive the voice of the driver D in the rear seat conversation mode, and the driver's seat speaker 230 can output the voices of the passengers C1 to Cn in the rear seat conversation mode.

The driver's seat conversation switch 240 can connect the driver terminal 200 to the passenger terminals 400k1 to 400kn, and can be turned on by the driver D when the driver D wants to activate the rear seat conversation mode. there is.

When the driver's seat conversation switch 240 is turned on, the driver's seat microphone 220, the driver's seat speaker 230, and the rear seat speaker 410 and rear seat microphone 420 provided in the plurality of passenger terminals 400k1 to 400kn may be turned on. there is.

However, the present invention is not necessarily limited thereto, and when the rear seat conversation mode is automatically activated by detecting a conversation between the driver and the first passenger, the driver's seat microphone 220 and the rear seat microphone ( 420) is maintained in a pre-turned-on state to detect a conversation, and when the rear-seat conversation mode is activated, the driver's seat speaker 230 and the rear-seat speaker 410 may be subsequently turned on.

The driver's seat conversation switch 240 may be provided separately from the image display device 210 as shown in (a) of FIG.

In FIG. 1, a case where the driver's seat conversation switch 240 is provided is shown as an example, but in the present invention, the driver's seat conversation switch 240 is not essential and may be omitted in some cases.

The plurality of occupant terminals 400k1 to 400kn may be provided around the rear seats of the occupants C1 to Cn, as shown in (a) of FIG. 1 . For example, it may be provided in front of each rear seat BS of the occupants C1 to Cn.

In FIG. 1 , a case in which there are a plurality of passenger terminals 400k1 to 400kn is shown as an example, but in some cases, only one passenger terminal located in a rear seat may be provided.

In (a) of FIG. 1, a case in which one of the plurality of passenger terminals 400k1 to 400kn is provided for each of the two rear seats is shown as an example, but the present invention is not limited thereto, and the plurality of passenger terminals 400k1 to 400kn ) is also possible to be provided one by one for each rear seat.

Each of the plurality of occupant terminals 400k1 to 400kn may include a rear seat microphone 420 and a rear seat speaker 410 .

The rear seat microphone 420 may input the voices of the occupants C1 to Cn in the rear seat conversation mode, and the rear seat speaker 410 may output the driver D's voice in the rear seat conversation mode.

As shown in (a) of FIG. 1 , the camera 300 may be installed on the ceiling at the rear side of the driver's seat DS in the vehicle 10 .

As shown in (b) of FIG. 1 , the camera 300 can rotate about a horizontal axis, rotate about a vertical axis, and zoom-in or zoom-out. A pan-tilt-zoom camera (PTZ camera) may be used.

In addition, although FIG. 1 illustrates a case in which one camera 300 is used as an example, the present invention is not necessarily limited thereto, and the number of cameras 300 may be plural.

When the rear seat conversation mode is activated by the driver terminal 200 and the passenger terminals 400k1 to 400kn, the camera 300 may photograph the passenger (eg, Cn) performing the rear seat conversation mode, and the camera 300 The image of the occupant (eg, Cn) photographed by the driver may be output through the image display device 210 provided in the driver's seat DS during the period in which the conversation mode for the rear seat is activated.

The control unit 100 is mounted in a vehicle to support rear seat conversation, performs a function of controlling a rear seat conversation mode between the driver D and passengers C1 to Cn, and includes the driving unit 110, the determination unit 120 and A rear seat conversation activation unit 130 may be included.

The rear seat conversation activation unit 130 may activate a rear seat conversation mode between the driver D and the passengers C1 to Cn.

When the rear seat conversation mode is activated by the rear seat conversation activation unit 130, the driver's seat microphone 220 and the driver's seat speaker 230 are turned on, and the rear seat microphone of the corresponding passenger terminal (eg, 400kn) for which the rear seat conversation mode is activated ( 420) and the rear seat speaker 410 may be turned on.

In addition, when the driver's seat microphone 220 and the rear seat microphone 420 are turned on before the rear seat conversation mode is activated, the driver's seat speaker 230 and the rear seat speaker 410 may be additionally turned on as the rear seat conversation mode is activated. there is.

In addition, when the rear seat conversation mode is activated between the driver terminal 200 and a specific passenger terminal (eg, 400kn) by the rear seat conversation activation unit 130, the remaining passenger terminals (400k1 to 400kn) excluding the specific passenger terminal (400kn) -1) The rear seat speaker 410 and the rear seat microphone 420 may be turned off.

The drive unit 110 controls the shooting angle of the camera 300 in the rear seat conversation mode, so that the camera 300 photographs the first occupant (eg, Cn) making a call by rotating and tilting the camera 300. ) or a control signal for controlling zoom may be output to the camera 300 .

The determination unit 120 analyzes the video captured by the camera 300 or the conversation between the driver and the passenger inputted to the microphone, and performs a conversation with the driver D among the plurality of passengers C1 to Cn in the rear seat. 1 The occupant (eg, Cn) can be determined and specified.

Hereinafter, an example of a method for performing the rear seat conversation method through the rear seat conversation system according to the first embodiment of FIG. 1 will be described.

2 is a view for explaining an example of a rear seat conversation method performed by the rear seat conversation system according to the first embodiment of the present invention shown in FIG. 1, and FIG. 3 is a photograph of the first occupant Cn in FIG. 2 This is an example for explaining an example of a method in more detail.

An example of a rear seat conversation method performed by the rear seat conversation system according to the first embodiment of the present invention includes activating (S10), specifying a first occupant (S20), and photographing the first occupant (S30). and displaying an image (S40).

In FIG. 2, the case in which each step (S10 to S40) is sequentially performed is shown as an example, but the present invention is not necessarily limited thereto, and the step of specifying the first passenger (S20) is the step of photographing the first passenger. It may be performed simultaneously with (S30), or it is also possible that the activating step (S10) is performed after the step of specifying the first passenger (S20) and the step of photographing the first passenger (S30).

In the activating step S10 , the controller 100 may activate a conversation mode in the rear seat between the driver D and the first passenger (eg, Cn) using a microphone and a speaker installed in the vehicle.

In the activating step (S10), the controller 100 may activate the rear seat conversation mode in various ways.

For example, when the driver's seat conversation switch 240 is provided around the driver's seat DS and the driver's seat conversation switch 240 is turned on by the driver D, the controller 100 may activate the rear seat conversation mode. there is.

However, the present invention is not limited thereto, and the controller 100 may automatically activate the rear seat conversation mode by detecting a conversation between the driver and the first passenger.

For example, when the microphones provided in the vehicle (eg, the driver's seat (DS) microphone and the rear seat microphone) are turned on before the rear seat conversation mode is activated, the microphone is used between the driver (D) and the first occupant (Cn). of conversations can be detected.

When the controller 100 analyzes the detected conversation and determines that a rear seat conversation is attempted, the controller 100 automatically activates the rear seat conversation mode even if the driver D does not turn on the driver's seat conversation switch 240. It can be.

For example, as shown in FIG. 1 , when there are a plurality of microphones installed in a vehicle, in the step of activating (S10), a conversation between the driver D and the first occupant Cn is inputted into the plurality of microphones in the vehicle. It is possible to determine whether an attempt is made for conversation in the back seat by analyzing the position of the utterance.

More specifically, there may be a case where a driver's seat (DS) microphone and a plurality of rear seat microphones are provided in a vehicle, and the driver's seat (DS) microphone and the plurality of rear seat microphones are turned on even before the rear seat conversation mode is activated.

In this case, a conversation may be exchanged between the driver D and the first passenger Cn before the rear seat conversation mode is activated. At this time, the conversation between the driver D and the first passenger Cn may be input through the driver's seat (DS) microphone and the plurality of rear seat microphones, and the voice input through the driver's seat (DS) microphone and the plurality of rear seat microphones, respectively. Signals may have time differences.

The control unit 100 may calculate the ignition position by analyzing the time difference of the voice signal input through each microphone, and the ignition position is changed while going back and forth between the driver's seat DS and the rear seat of the first passenger Cn. If it is determined that it is true, the controller 100 may determine that the driver D and the first occupant Cn attempt a conversation in the rear seat, and activate the conversation mode in the rear seat.

As another example, when it is determined that a conversation in the rear seat is being attempted by analyzing a mouth shape of at least one passenger captured by a camera provided in the vehicle, the controller 100 may activate a conversation mode in the rear seat.

For example, when a conversation between the driver D and the first passenger Cn is detected by the microphone, the camera turns on and rotates, tilts, or zooms in the direction of the ignition position to capture at least one passenger. A photograph may be taken and the mouth of each of the at least one occupant may be analyzed.

When it is determined that there is a first passenger (Cn) whose mouth shape matches the detected conversation among the at least one passenger, the controller 100 determines that the first passenger (Cn) among the at least one passenger is the driver (D) and the driver (D) in the rear seat. When it is determined that a conversation is attempted, the rear seat conversation mode may be activated.

In the step of specifying the first occupant ( S20 ), the controller 100 may specify the first occupant Cn among at least one occupant in the rear seat. The step of specifying the first occupant ( S20 ) may be performed after the step of activating as shown in FIG. 2 , or may be performed together with the step of activating differently from FIG. 2 .

The step of specifying the first occupant (S20) may be variously performed by a camera and a microphone.

For example, by analyzing the mouth shape of each of the at least one passenger photographed by the camera, the first passenger (Cn ) can be specified.

As another example, when there are a plurality of microphones provided in the rear seat, the first passenger Cn may be specified by analyzing an ignition position from a conversation between the driver D and at least one passenger inputted by the plurality of microphones.

More specifically, from the conversation between the driver (D) and the passenger input by the microphone, the passenger's voice excluding the driver's (D) voice is analyzed, and the time delay of the passenger's voice input by the plurality of microphones is calculated to utter the passenger's voice. By calculating the position, the first occupant Cn may be specified.

As another example, although not shown in FIG. 1 , a conversation switch for activating a conversation mode for a rear seat may be provided for each rear seat in which a passenger sits. In this case, it is also possible to specify the first occupant Cn by analyzing the position of a turned-on conversation switch among a plurality of conversation switches provided for each rear seat. This will be described in more detail below in FIG. 4 .

In the step of photographing the first occupant (S30), a camera that is interlocked with the control unit 100 and photographs a rear seat may photograph the first occupant Cn. The step of photographing the first occupant ( S30 ) may include photographing the surrounding area and photographing the face.

In the step of photographing the surrounding area, the camera may primarily photograph the surrounding area of the first occupant Cn including the first occupant Cn by rotating, tilting, or zooming. The step of photographing the surrounding area may be performed together with the previous step of activating (S10).

In the step of photographing the face, the camera rotates, tilts, or zooms the face of the first occupant Cn, which is specified through image analysis of the surrounding area by the controller 100, and secondarily captures the face. can

In the step of displaying an image (S40), an image of the face of the first occupant Cn may be displayed on the image display device.

As described above, the present invention allows the driver D to more comfortably recognize the first passenger Cn by displaying the face of the first passenger Cn who is having a conversation with the driver D on the image display device. It is possible to have a back seat conversation.

In addition, according to the present invention, when a conversation between the driver D and the first occupant Cn is detected through a microphone provided in the vehicle, the rear seat conversation mode is automatically performed, thereby further enhancing the convenience of the driver D. can

So far, the case where the switch activating the rear seat conversation mode is provided only in the driver's seat DS has been described as an example, but the present invention is not limited thereto, and the switch activating the rear seat conversation mode may also be provided in the rear seat. This will be explained below.

In addition, in the following, the same parts as those described in FIGS. 1 to 3 are replaced with the previous contents, and other parts are mainly described.

4 is a diagram for explaining a second embodiment of a rear seat conversation system according to the present invention.

The rear seat conversation system according to the second embodiment of the present invention may further include a rear seat conversation switch. Accordingly, when the rear seat conversation mode is activated by turning on the switch, it may be activated when not only the driving conversation switch but also the rear seat conversation switch are turned on together.

However, the present invention is not necessarily limited thereto, and when the rear seat conversation mode is automatically activated by detecting a conversation or detecting a camera, the rear seat conversation switch may be activated even if the switch is not turned on.

The rear seat conversation switch 430 allows the passenger terminals 400k1 to 400kn to be connected to the driver terminal 200, and when the passengers C1 to Cn want a rear seat conversation mode, they are turned on by the passengers C1 to Cn. It can be. When the rear seat conversation switch 430 is turned on, the rear seat microphone 420 and the driver's seat speaker 230 may be turned on.

Hereinafter, an example in which the rear seat conversation method is performed by the rear seat conversation system of FIG. 4 will be described.

FIG. 5 is a diagram for explaining an example of a rear seat conversation method performed by the rear seat conversation system according to the second embodiment of the present invention shown in FIG. 4 .

An example of a rear seat conversation method performed by the rear seat conversation system according to the second embodiment of the present invention, as shown in FIG. 5 , includes turning on a first rear seat conversation switch (S100) and photographing a first occupant. (S200), displaying on the video display device (S300), and performing a back seat conversation mode (S400).

In FIG. 5 , a case in which the step S400 in which the rear seat conversation mode is performed after the step displayed on the video display device (S300) is immediately and continuously performed is illustrated as an example.

The present invention is not necessarily limited thereto, and other steps may be further included between the step of displaying on the video display device (S300) and the step of performing the rear seat conversation mode (S400).

In the step of turning on the first rear seat conversation switch ( S100 ), the first rear seat conversation switch 430 provided around the rear seat of the first occupant Cn in the vehicle 10 may be turned on for the rear seat conversation mode. .

The first rear seat conversation switch 430 is a rear seat conversation switch included in at least one first passenger terminal (eg, 400kn) that performs a rear seat conversation mode with the driver's seat terminal 200 among the plurality of passenger terminals 400k1 to 400kn. (430).

Hereinafter, for convenience of explanation, a case where the driver's seat terminal 200 and the passenger terminal performing the rear seat conversation mode are one will be described as an example.

However, the present invention is not limited thereto, and the driver's seat terminal 200 and the passenger terminal performing the rear seat conversation mode may be plural.

In addition, hereinafter, the rear seat conversation switch 430, the rear seat microphone 420, and the rear seat speaker 410 provided in the first occupant terminal 400kn are respectively referred to as the first rear seat conversation switch 430 and the first rear seat microphone 420. and a first rear seat speaker 410.

The first passenger Cn may refer to a passenger performing a conversation mode with the driver D among the plurality of passengers C1 to Cn in the vehicle 10. Hereinafter, the first passenger Cn Although the case of one person is described as an example, the present invention is not limited thereto, and the number of first passengers may be plural.

In the step of turning on the first rear seat conversation switch (S100), when the first rear seat conversation switch 430 is turned on by the first occupant Cn, the first rear seat microphone 420 and the driver's seat speaker 230 are turned on. It can be.

In the step of photographing the first occupant ( S200 ), the camera 300 provided in the vehicle 10 may photograph the first occupant Cn located around the first rear seat conversation switch 430 .

Unlike FIG. 4(a), when one passenger terminal 400k1 to 400kn is provided for each passenger C1 to Cn, the camera 300 uses the location of the passenger terminal 400k1 to 400kn. Thus, the first occupant Cn can be easily identified.

However, as shown in (a) of FIG. 4 , when one passenger terminal is provided for each rear seat of two passengers, it is not easy to specify which of the two passengers (C1 to Cn) wants the rear seat conversation mode. In this case, the determination unit 120 may analyze the image captured by the camera 300 to identify the first occupant Cn. This will be described later with reference to FIGS. 6 and 7 .

In the step of displaying on the image display device (S300), the image captured by the camera 300 may be displayed on the image display device 210 provided in the driver's seat DS in the vehicle 10.

Thereafter, when the driver D turns on the driver's seat conversation switch 240, the rear seat conversation mode is performed, and in step S400 in which the rear seat conversation mode is performed, the driver's seat microphone 220 provided around the driver's seat DS and the driver's seat The driver D and the first occupant Cn in the vehicle 10 are provided through the speaker 230 and the first rear seat microphone 420 and the first rear seat speaker 410 provided around the rear seat of the first occupant Cn. ) can be performed.

Accordingly, the present invention can cause the driver D to recognize the first passenger Cn, and during a period in which the rear seat conversation mode is performed between the first passenger Cn, the driver D is the first passenger Cn. Communication with the first occupant Cn is performed while viewing the face of Cn, so that communication between the driver D and the first occupant Cn can be more smoothly achieved.

Here, in the step of photographing the first occupant (S200), the control unit 100 differs depending on whether one occupant terminal (400k1 to 400kn) is provided for each rear seat of each occupant or one occupant terminal is provided for each rear seat of two occupants. ) may vary.

For example, unlike FIG. 4(a), when one passenger terminal is provided for each rear seat of a passenger, each position of the passenger terminal may be stored in advance in the control unit 100, and a first passenger terminal among a plurality of passenger terminals may be stored. When the first rear seat conversation switch 430 of the terminal is turned on, the control unit 100 may specify a passenger located around the first passenger terminal as the first passenger Cn by using the location of the first passenger terminal. there is.

Accordingly, the controller 100 may control the camera 300 to immediately photograph the first occupant Cn. At this time, the position of the face of the first occupant Cn may vary depending on whether the first occupant Cn is an adult or a child, but this can be controlled by adjusting the zoom angle of the e-cabin camera in detail.

However, as shown in (a) of FIG. 4, when one passenger terminal (400k1 to 400kn) is provided for each rear seat of two passengers (C1 to Cn), who among the two passengers (C1 to Cn) has a conversation in the rear seat? Since it is not easy to specify whether a mode is desired, the control unit 100 and the camera 300 may be operated differently from the above description. 6 and 7 below will be described as an example.

FIG. 6 is a diagram for explaining an example of a method of capturing a first occupant in FIG. 5 , and FIG. 7 is a diagram for explaining an example of a method for specifying a first occupant in FIG. 6 .

As described above with reference to (a) of FIG. 4 , when one passenger terminal 400k1 to 400kn is provided in the back seat of a plurality of passengers C1 to Cn, the step of photographing the first occupant (S200) is an example. As shown in FIG. 6 , it may include photographing the surrounding area (S210), specifying the first occupant (S220), and photographing the face of the first occupant (S230).

In step S210 of photographing the surrounding area, the driving unit 110 may control the camera 300 to primarily photograph the surrounding area of the first rear seat conversation switch 430 including the first occupant Cn.

That is, when the first rear seat conversation switch 430 is turned on, in the step of capturing the surrounding area (S210), the driving unit 110 uses the position of the first occupant terminal 400kn stored in advance in the control unit 100. The camera 300 is primarily controlled to photograph the area around the first rear seat conversation switch 430, so that the camera 300 captures the area around the first rear seat conversation switch 430, the first occupant Cn; The rear seats of the second occupant Cn-1 and the first and second occupants Cn and Cn-1 may be photographed.

An image of the surrounding area photographed by the camera 300 may be input to the determination unit 120 .

Thereafter, in the step of specifying the first occupant (S220), the determination unit 120 analyzes the image of the surrounding area, and at least one occupant (e.g., The motions of Cn and Cn-1) may be analyzed and a first passenger Cn among at least one passenger Cn and Cn-1 may be specified.

Specifically, in the step of specifying the first occupant ( S220 ), as shown in FIG. 7 , it is possible to determine whether there are several occupants around the first rear seat conversation switch 430 by analyzing the image of the surrounding area. If there is only one occupant around the first rear seat conversation switch 430, the determination unit 120 may specify one occupant as the first occupant.

However, as shown in (a) of FIG. 4 , when there are a plurality of passengers (eg, Cn, Cn-1) located in the area around the first rear seat conversation switch 430, the determination unit 120 determines the number of passengers (Cn). , Cn-1) It is possible to specify a first occupant among a plurality of occupants Cn and Cn-1 by analyzing an image of each motion or mouth shape.

In order to specify the first passenger Cn among the plurality of passengers Cn and Cn-1, the determination unit 120 determines the plurality of passengers Cn and Cn from the image of the area around the first rear seat conversation switch 430. -1) A contour line for at least one of each posture and mouth shape may be extracted.

Thereafter, the determination unit 120 may analyze the movement of each contour line of the plurality of occupants Cn and Cn-1 to determine whether they are the first occupant.

For example, the determination unit 120 determines that after the first rear seat conversation switch 430 is turned on, the upper bodies of the two occupants Cn and Cn-1 are tilted backward, and then the first rear seat microphone 420 moves forward. An occupant (eg, Cn) leaning forward may be determined as the first occupant Cn.

Or, as another example, the determination unit 120 determines that the upper bodies of the two passengers Cn and Cn-1 are tilted forward for a certain period of time (eg, for several seconds) after the first rear seat conversation switch 430 is turned on. If there is a passenger (eg, Cn) who is moving his or her mouth, the corresponding passenger (Cn) may be determined as the first passenger.

As another example, the determination unit 120 determines the movement of the mouth of each of the two passengers Cn and Cn-1 among the two passengers Cn and Cn-1 after the first rear seat conversation switch 430 is turned on. 1 The similarity between voices input through the rear seat microphone 420 may be determined, and an occupant having a high similarity (eg, Cn) may be determined as the first occupant.

To this end, the control unit 100 may further include a voice analyzer (not shown) capable of analyzing the voice of the first passenger Cn, and the voice analyzer (not shown) is configured to analyze the voice of the first passenger Cn. By analyzing the , it is possible to determine whether the content of the voice is a voice requesting a call with the driver D.

For example, if the content of the voice is calling the driver D's name or content such as "hello", the voice analysis unit (not shown) may determine that the content of the voice is a request for a call, and the passenger If the movement of the mouth of (Cn, Cn-1) has a high similarity to the content of the voice requesting a call, the corresponding passenger (eg, Cn) may be determined as the first passenger.

As described above, the determination unit 120 of the present invention may determine the first occupant by variously analyzing at least one of upper body movements and mouth-shaped movements of the plurality of occupants Cn and Cn-1.

Then, as shown in FIG. 6 , when the first occupant Cn is specified, in step S230 of photographing the face of the first occupant, the drive unit 110 causes the camera 300 to secondarily capture the first occupant Cn. It can be controlled to take a picture of the face of the user.

In this way, the facial image of the first occupant Cn photographed by the camera 300 may be displayed on the image display device 210 during the rear seat conversation mode period.

Hereinafter, an example in which the back seat conversation method according to the first embodiment of the present invention is applied to an actual situation will be described in detail.

In the following description of FIG. 8 , the content overlapping with the content described in FIGS. 4 to 7 is replaced with the content described in FIGS. 4 to 7 , and other parts are mainly described.

8 is a diagram for explaining a first modified example of a conversation method for a rear seat performed by a conversation system for a rear seat according to a second embodiment of the present invention.

The first modified example of the rear seat conversation method of the present invention shown in FIG. 8 is an example of a case where the driver D requests the rear seat conversation mode to a specific first passenger Cn among the plurality of passengers C1 to Cn. am.

A first modified example of the rear seat conversation method according to the present invention includes the steps of turning on the driver's seat conversation switch (S500), outputting the output through the entire rear seat speakers (S600), and turning on the first rear seat conversation switch (S100). , photographing the first occupant (S200), displaying on the video display device (S300), and performing a conversation mode in the rear seat (S400). Here, when the driver's seat conversation switch is turned on (S500) and the first rear seat conversation switch is turned on (S100), the rear seat conversation mode may be activated.

In the step of turning on the driver's seat conversation switch (S500), the driver's D may turn on the driver's seat conversation switch 240 provided around the driver's seat DS in the vehicle 10 for the rear seat conversation mode. Accordingly, the driver's seat microphone 220 may be turned on, and all of the plurality of rear seat speakers 410 may be turned on. Accordingly, the voice of the driver D can be transmitted through the entire plurality of rear seat speakers 410 provided in the vehicle 10 .

In step S600, when the driver D inputs a voice requesting a call with a specific first passenger Cn among the plurality of passengers C1 to Cn, the driver's seat microphone 220 is output through all rear seat speakers. The driver D's voice input through may be output through the entire plurality of rear seat speakers 410 provided on both sides of the vehicle 10 .

Then, in the step of turning on the first rear seat conversation switch (S100), the first rear seat conversation switch 430 may be turned on by the first occupant Cn, and accordingly, the first rear seat microphone 420 and the driver's seat speaker ( 230) can be turned on.

In addition, in the step of photographing the first occupant (S200), the camera 300 may specify the first occupant Cn and photograph the face of the first occupant Cn.

Accordingly, in the step of displaying on the image display device (S300), the face of the first passenger Cn is displayed on the image display device 210 provided in the driver's seat DS, and the rear seat conversation mode is performed (S400). , the driver D may recognize the face of the first passenger Cn and perform a conversation mode with the first passenger Cn in the rear seat.

Additional detailed description of the step of turning on the first rear seat conversation switch (S100), photographing the first occupant (S200), displaying on the video display device (S300), and performing the rear seat conversation mode (S400) Since is the same as the content described in FIGS. 4 to 7 above, it is replaced with the content described in FIGS. 4 to 7.

However, in the step of photographing the first occupant (S200), the camera 300 captures the second occupant Cn-1 sitting in the rear seat next to the first occupant Cn, not the first occupant Cn. In this case, the camera 300 may re-photograph the first occupant Cn by recognizing the gesture of the driver D for the incorrect photographing of the camera 300 .

To this end, a sensor (eg, a camera) capable of recognizing a gesture of the driver D may be provided on the front of the driver's seat DS, and a gesture capable of analyzing the gesture of the driver D may be provided in the controller 100. A determination unit (not shown) may be additionally provided.

Accordingly, in the step of photographing the first occupant (S200), when the camera 300 erroneously captures the second occupant Cn-1, a gesture determining unit (not shown) may cause the driver D to wave his hand. or shaking the head may be recognized, and the first occupant Cn right next to the second occupant Cn-1 may be photographed again.

In addition, in the step of performing the rear seat conversation mode (S400), the rear seat speakers 410 other than the first rear seat speaker 410 among the plurality of rear seat speakers 410 may be turned off while the rear seat conversation mode is being performed. The remaining rear seat microphones 420 other than the first rear seat microphone 420 may also remain turned off.

9 is a diagram for explaining a second modified example of a rear seat conversation method performed by the rear seat conversation system according to the second embodiment of the present invention.

A second modified example of the rear seat conversation method according to the present invention shown in FIG. 9 is for a case in which one of the plurality of passengers C1 to Cn requests the driver D to use the rear seat conversation mode. This is an example.

In the second modified example of the rear seat conversation method according to the present invention shown in FIG. 9, the first rear seat conversation switch is turned on (S100), the first passenger is photographed (S200), and the image display device displays the step (S200). It may include (S300), an output through the driver's seat speaker (S700), a driver's seat conversation switch being turned on (S800), and a rear seat conversation mode being performed (S400).

Here, when the first rear seat conversation switch is turned on (S100) and the driver's seat conversation switch is turned on (S800), the rear seat conversation mode may be activated.

In the step of turning on the first rear seat conversation switch (S100), the first rear seat conversation switch 430 may be turned on by the first occupant Cn, and accordingly, the first rear seat microphone 420 and the driver's seat speaker 230 can be turned on. Accordingly, the first rear seat microphone 420 may receive the voice of the first occupant Cn and output it through the driver's seat speaker 230 .

In addition, in the step of photographing the first occupant (S200), the camera 300 may specify the first occupant Cn and photograph the face of the first occupant Cn.

Accordingly, in the step of being displayed on the image display device (S300), the face of the first occupant Cn may be displayed on the image display device 210 provided in the driver's seat DS.

Here, more detailed descriptions of the step of turning on the first rear seat conversation switch (S100), the step of photographing the first occupant (S200), and the step of displaying on the image display device (S300) are described in FIGS. 4 to 7 above. Since it is the same as the description, it is replaced with the description in FIGS. 4 to 7.

In the step of outputting through the driver's seat speaker (S700), the voice of the first occupant Cn input through the first rear seat microphone 420 may be output through the driver's seat speaker 230.

The driver D can recognize the face of the first passenger Cn displayed on the image display device 210 and the call request of the first passenger Cn output through the speaker 230 in the driver's seat.

In the step of turning on the driver's seat conversation switch (S800), the driver's D may turn on the driver's seat conversation switch 240 provided around the driver's seat DS for the rear seat conversation mode.

Accordingly, in step S400 in which the rear seat conversation mode is performed, the driver D may recognize the face of the first passenger Cn and perform the rear seat conversation mode with the first passenger Cn.

Here, in the step of performing the rear seat conversation mode (S400), the rear seat speakers 410 other than the first rear seat speaker 410 among the plurality of rear seat speakers 410 provided on both sides of the vehicle 10 are turned off. can keep

Accordingly, in the rear seat conversation method according to an example of the present invention, when the first passenger Cn who wants to communicate with the driver D turns on the first rear seat conversation switch 430, the camera 300 provided in the vehicle 10 ) photographs the first occupant Cn and displays the face of the first occupant Cn on the image display device 210 of the driver's seat DS, the driver D and the first occupant Cn In between, the back seat conversation mode can be performed.

Accordingly, the present invention can further improve the convenience of the rear seat conversation mode.

The technical features disclosed in each embodiment of the present invention are not limited to the corresponding embodiment, and unless incompatible with each other, the technical features disclosed in each embodiment may be merged and applied to other embodiments.

Therefore, in each embodiment, each technical feature is mainly described, but each technical feature may be merged and applied to each other unless incompatible with each other.

The present invention is not limited to the above-described embodiments and accompanying drawings, and various modifications and variations will be possible from the viewpoint of those skilled in the art to which the present invention belongs. Therefore, the scope of the present invention should be defined by not only the claims of this specification but also those equivalent to these claims.

100: control unit 200: driver's seat terminal
300: camera 400: passenger terminal

Claims (15)

specifying, by a controller, a first occupant from among at least one occupant in the rear seat of the vehicle;
The control unit calculates the ignition position by analyzing the time difference between the voice signals input through the driver's seat microphone and the rear seat microphone, respectively, and the calculated ignition position is changed while moving back and forth between the driver's seat and the specified rear seat of the first passenger. if determined, activating a conversation mode for a rear seat supporting communication between the driver and the first occupant;
photographing the first occupant by a camera interlocked with the control unit and photographing the rear seat; and
Displaying an image of the first occupant by an image display device provided in a driver's seat in the vehicle and interlocked with the controller
including,
The step of photographing the first occupant
firstly photographing, by the camera, an area around the first occupant including the first occupant; and
and secondarily photographing, by the controller, the face of the first occupant identified by the camera through the image analysis of the surrounding area. delete delete delete According to claim 1,
The step of specifying the first passenger
The first occupant is identified by analyzing the mouth shape of each of the at least one occupant photographed by the camera;
The passenger's voice inputted through the driver's seat microphone and the rear seat microphone is analyzed and a time delay of the passenger's voice inputted through the plurality of microphones is calculated to analyze the driver's voice and the passenger's voice from the conversation between the driver and at least one passenger. A rear seat conversation method for specifying the first occupant by calculating an ignition position for delete delete delete delete delete delete delete delete delete delete

Images