KR102475071B1 - Method and apparatus for controlling contents in vehicle for a plurality of users - Google Patents

Abstract

A content control method in a vehicle for a plurality of users according to an embodiment of the present invention a) by detecting at least one of the distance between the screen of the display unit and the users and the number of users by a sensor unit, thereby enabling a user environment. determining; b) selecting one or more of the user's gaze, motion, gesture, voice, and touch sensed by the sensor unit based on the user environment as an interaction method; and c) controlling at least one of the display unit and the sound output unit to provide interactive content according to a detection signal of the sensor unit according to the interaction method.

Description

Method and apparatus for controlling contents in a vehicle for multiple users

The present invention relates to a content control method and a content control device in a vehicle for multiple users, and more particularly, to a content control method and content control device in which multiple users can smoothly manipulate content in a vehicle such as an unmanned shuttle bus. It relates to a content control method and content control device in a vehicle for a plurality of users, capable of providing appropriate content according to a method.

In recent years, the development and use of autonomous vehicles have been actively conducted. A content control method and a content control device for providing and controlling content such as infotainment for users are widely applied to such self-driving vehicles.

However, conventional content control methods and content control apparatuses in a vehicle perform interactions only with a manipulation method suitable for a vehicle for a single user, for example, a touch and/or a gesture. That is, since the conventional content control method and content control device in a vehicle are performed and applied by manipulation centered on the driver's seat, users manipulate content in a vehicle for a large number of users having a large space such as an unmanned shuttle bus. There are problems with which it is difficult.

In addition, since the conventional content control method and content control device in a vehicle provide content without recognizing the user environment, there is a problem in providing suitable content in a vehicle for a plurality of users.

An object of the present invention is, for example, content in a vehicle for multiple users, in which multiple users can smoothly manipulate content in a vehicle for multiple users with a large space such as an unmanned shuttle bus. It is to provide a control method and a content control device.

Another object of the present invention is to provide an in-vehicle content control method and content control device for multiple users, capable of providing content suitable for multiple users in-vehicle according to user environments.

In order to achieve the above object, according to a first feature of an embodiment of the present invention, a method for controlling content in a vehicle for a plurality of users is provided by a) controlling the screen of a display unit and the users by a sensor unit. determining a user environment by detecting at least one of a distance and the number of users; b) selecting one or more of the user's gaze, motion, gesture, voice, and touch sensed by the sensor unit based on the user environment as an interaction method; and c) controlling at least one of the display unit and the sound output unit to provide interactive content according to a detection signal of the sensor unit according to the interaction method.

According to a second feature of an embodiment of the present invention, a content control method in a vehicle for multiple users a) detects at least one of a distance between a screen of a display unit and the users and the number of users by a sensor unit. Determining the user environment by doing; b) determining whether an interaction using at least one of gaze, motion, gesture, voice, and touch of the users sensed by the sensor unit is necessary based on the user environment; and c) controlling at least one of the display unit and the sound output unit to provide one-way content according to whether the interaction is required.

According to a third feature of an embodiment of the present invention, an in-vehicle content control device for multiple users includes a display unit having a screen on which visual content is displayed; an audio output unit that outputs audio content; a sensor unit that detects at least one of a distance between the screen and the users and the number of users, and detects at least one of gaze, motion, gesture, voice, and touch of the users; And based on the user environment determined by at least one of the distance between the screen and the users and the number of users, selecting one or more of the users' gaze, motion, gesture, voice, and touch as an interaction method, and a control unit for controlling at least one of the display unit and the sound output unit to provide interactive content according to a detection signal of the sensor unit according to the interaction method.

According to a fourth feature of an embodiment of the present invention, an in-vehicle content control device for multiple users includes a display unit having a screen on which visual content is displayed; an audio output unit that outputs audio content; a sensor unit sensing at least one of a distance between the screen and the users and the number of users; and at least one of gaze, motion, gesture, voice, and touch of the users detected by the sensor unit based on the user environment determined by at least one of the distance between the screen and the users and the number of users. and a control unit that determines whether an interaction using , and controls one or more of the display unit and the sound output unit to provide one-way content according to whether or not the interaction is required.

Using the content control method and content control device in a vehicle for multiple users according to an embodiment of the present invention, the following effects are achieved.

1. For example, multiple users can smoothly manipulate content in a vehicle for multiple users with a wide space such as an unmanned shuttle bus.

2. According to the user environment, suitable content can be provided in the vehicle for multiple users.

Hereinafter, preferred embodiments of a content control method and content control device in a vehicle for multiple users according to the present invention will be described in detail with reference to the accompanying drawings.
1 is a block diagram of a content control system having an in-vehicle content control device for multiple users according to an embodiment of the present invention.
2 and 3 are perspective views illustrating a vehicle having a content control device in the vehicle for multiple users according to various embodiments of the present disclosure.
4 is a schematic diagram of a sensor unit and a display unit included in an apparatus for controlling content in a vehicle for multiple users according to an embodiment of the present invention.
5A to 5E are schematic diagrams illustrating a proximity user environment determined by a control unit of a content control device in a vehicle for a plurality of users and their interactions according to an embodiment of the present invention.
6A to 6E are schematic diagrams for describing a non-proximity user environment determined by a control unit of a content control device in a vehicle for multiple users and their interactions according to an embodiment of the present invention.
7A to 7C are schematic diagrams illustrating a group user environment determined by a control unit of a content control device in a vehicle for a plurality of users and content provision therefor according to an embodiment of the present invention.
8 is a flow chart of an in-vehicle content control method for multiple users according to an embodiment of the present invention.

Hereinafter, preferred embodiments of a content control method and content control device in a vehicle for multiple users according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of a content control system having an in-vehicle content control device for a plurality of users according to an embodiment of the present invention, and FIGS. 2 and 3 are a block diagram of a plurality of content control systems according to various embodiments of the present invention. 4 is a perspective view showing a vehicle having an in-vehicle content control device for users, and FIG. 4 is a sensor unit and a display unit included in the in-vehicle content control device for multiple users according to an embodiment of the present invention. it is a schematic

5A to 5E are schematic diagrams illustrating a proximity user environment determined by a control unit of a content control device in a vehicle for a plurality of users and their interactions according to an embodiment of the present invention. Specifically, FIGS. 5B to 5E illustrate interactions through touch, gesture, voice, and gaze, respectively.

6A to 6E are schematic diagrams for describing a non-proximity user environment determined by a control unit of a content control device in a vehicle for multiple users and their interactions according to an embodiment of the present invention. Specifically, FIG. 6b illustrates interaction through gaze, and FIGS. 6c and 6d illustrate interaction through motion.

7A to 7C are schematic diagrams illustrating a group user environment determined by a control unit of a content control device in a vehicle for a plurality of users and content provision therefor according to an embodiment of the present invention.

First, referring to FIGS. 1 to 7C , a content control device 100 in a vehicle C for multiple users according to an embodiment of the present invention will be described as follows.

1 to 3 , a vehicle (C; C1, C2) equipped with a content control device (100; 101, 102) is, for example, autonomous driving in which a plurality of users (U) can ride. It may be implemented by any one of a bus, a van, an SUV, and a train, but is not limited thereto, and various examples will be possible if it is a vehicle in which a number of users U can ride. In addition, in the present specification, "vehicles (C; C1, C2) for multiple users (U)" means a vehicle in which multiple users (U) can ride, and the screen of the display unit 110 It is not always necessary that there are a plurality of users U looking at the screen, and the number of users U looking at the screen of the display unit 110 may be one.

In addition, in this specification, content includes, for example, information, virtual reality (VR), augmented reality (AR), tourism, shopping, games, music, and/or video conferences. can include

A content control device 100 in a vehicle C for multiple users includes a display unit 110, a sound output unit 120, a sensor unit 130, and a control unit 140. In addition, the content control device 100 may further include a storage unit 150 and/or a communication unit 160 .

The display unit 110 is a component having a screen on which visual content is displayed. The display unit 110 may be implemented by, for example, a touchscreen panel display having a touchscreen 134 . Also, the display unit 110 may be a transparent display capable of displaying visual content while allowing the user U to see the outside of the vehicle C. Also, the display unit 110 may be a rollable display.

The audio output unit 120 is a component that outputs audio content. The sound output unit 120 may include a speaker and/or an earphone.

The sensor unit 130 is a component that senses at least one of the distance between the screen of the display unit 110 and the users U and the number of users U. Also, the sensor unit 130 may detect one or more of users U's gaze, motion, gesture, voice, and touch.

For example, the sensor unit 130 may include an RGB-Depth camera 131 , an infrared camera 132 , a microphone 133 , and/or a touch screen 134 . The RGB-D camera 131 may detect, for example, an operation of the user U, a distance between the screen of the display unit 110 and the user U, and/or the number of users U. . Also, for example, the RGB-D camera 131 may detect an operation of the user U in a wide space of the vehicle C by being composed of a plurality of RGB-D cameras 131 .

The infrared camera 132 may detect, for example, a gesture and/or gaze of the user U, or a distance between the screen of the display unit 110 and the user U and/or the number of users U. can Also, the microphone 133 may detect the user U's voice, and the touch screen 134 may detect the user U's touch.

In the present specification, “detecting gaze” may mean “detecting at least one of whether the user U is looking at the screen and a position of the user U looking at the screen”. For example, referring to FIG. 6B , the infrared camera 132 may detect that the users U1 and U2 are looking at the screen of the display unit 110 . Also, referring to FIG. 5E , the infrared camera 132 may detect a position on the screen of the display unit 110 that the user U is looking at. That is, gaze detection shown in FIG. 5E corresponds to gaze tracking.

In this specification, "sensing a motion" means "at least one of the presence or absence of a temporary motion of the user U at a position farther than a predetermined distance from the screen, the presence or absence of a continuous motion, the motion position, and the motion direction. Detected" may be. For example, referring to FIG. 6C , the RGB-D camera 131 may detect a temporary motion of the user U1 such as simply raising an arm. Also, referring to FIG. 6D , the RGB-D camera 131 may detect a motion position and a motion direction while detecting a continuous motion of the user U1 such as waving an arm. That is, motion detection shown in FIG. 6D corresponds to motion tracking.

In the present specification, “sensing a gesture” may mean “detecting one or more of a movement position and a movement direction of the user U within a predetermined distance from the screen”. For example, referring to FIG. 5C , the infrared camera 132 may detect one or more of the movement positions and directions of the hands or fingers of the users U1, U2, and U3.

In this specification, a “gesture” may be a smaller movement than a “motion”.

The control unit 140 determines the user's (U's) line of sight, motion, At least one of gesture, voice, and touch is selected as an interaction method, and at least one of the display unit 110 and the sound output unit 120 provides interactive content according to the detection signal of the sensor unit 130 according to the selected interaction method. control to do

In this specification, "interaction" may refer to an interaction between the user U and the sensor unit 130 of the content control device 100 after the user environment is determined by the control unit 140. have. Also, “interactive content” may refer to content using an interaction method through one or more of the user U's gaze, motion, gesture, voice, and touch. For example, “interactive content” may include games and/or video conferences.

In addition, when a plurality of detection signals by the interaction of a plurality of users U are input through the sensor unit 130, the controller 140 may integrate and process the plurality of detection signals. By integrated processing of such a plurality of detection signals, it is possible to smoothly provide contents without prioritizing user U interactions.

In addition, the controller 140 detects the user environment by the sensor unit 130 based on one or more of the distance between the screen of the display unit 110 and the users U and the number of users U. It is determined whether an interaction using one or more of gaze, motion, gesture, voice, and touch of the users (U) is required, and one or more of the display unit 110 and the sound output unit 120 is determined depending on whether the interaction is required. You can control to provide this one-way content. In this specification, “one-way content” may refer to content that does not require interaction through one or more of the user U's gaze, motion, gesture, voice, and touch. For example, “one-way content” may include listening to music and/or watching movies.

In addition, the controller 140 may determine the user environment according to whether the number of users U is less than or equal to a predetermined number. In addition, the controller 140 may determine the user environment according to whether the distance between the screen of the display unit 110 and the users U is equal to or less than a predetermined distance.

In this specification, 'the number of users U' may be, for example, the number of users U looking at the screen of the display unit 110 . 'Distance between the screen of the display unit 110 and the users U' refers to the user U who is the furthest away from the screen of the display unit 110 among users U looking at the screen of the display unit 110. and the distance from the screen'.

The user environment may include, for example, a 'proximity user environment', a 'non-proximate user environment', and/or a 'cluster user environment'.

Referring to FIG. 5A , 'proximity user environment' refers to, for example, when the number of users U is less than or equal to a predetermined number and the distance between the screen of the display unit 110 and the users U is less than or equal to a predetermined distance. It can be a user environment. 'Non-proximity user environment' refers to FIG. 6A , for example, when the number of users U is less than or equal to a predetermined number and the distance between the screen of the display unit 110 and the users U is a predetermined distance. may be a user environment that exceeds Referring to FIG. 7A , the 'group user environment' may be, for example, a user environment in which the number of users U exceeds a predetermined number. The 'predetermined number of people' may be, for example, three people, and the 'predetermined distance' may be, for example, 1 m, but is not limited thereto.

The storage unit 150 is a component that stores content. The storage unit 150 may store content in advance before the control unit 140 receives the detection signal from the sensor unit 130, or may store the content in advance according to the detection signal of the sensor unit 130 received by the control unit 140. Content may be received and stored from the server 200 .

The communication unit 160 is a component for communicating with the content server 200 and/or the terminal 300 . The communication unit 160 may perform wireless communication, for example. For example, the communication unit 160 may receive content from the server communication unit 220 of the content server 200, or transfer the contents stored in the storage unit 150 to the terminal communication unit 310 of the terminal 300. can also be sent.

Next, referring to FIG. 1, a content control system 1000 having a content control device 100 in a vehicle C for multiple users according to an embodiment of the present invention will be described as follows. .

The content control system 1000 includes a content control device 100 , a content server 200 , and a terminal 300 . Since the content control device 100 has been described above, only the content server 200 and the terminal 300 will be described below.

The content server 200 is a component that stores content and transmits the content to the content control device 100 and/or terminal 300 . The content server 200 includes a server storage unit 210, a server communication unit 220, and a server control unit 230. The server storage unit 210 stores content, and the server communication unit 220 transmits the content to the communication unit 160 of the content control device 100 and/or the terminal communication unit 310 of the terminal 300 . The server control unit 230 controls the server communication unit 220 to transmit content stored in the server storage unit 210 according to a signal received from the server communication unit 220 .

The terminal 300 is a component that can be carried by the user U and can receive content from the content control device 100 and/or the content server 200 . The terminal 300 includes a terminal communication unit 310, a terminal display unit 320, a terminal audio output unit 330, a terminal control unit 340, and an input unit 350. The terminal communication unit 310 receives content from the communication unit 160 of the content control device 100 and/or the server communication unit 220 of the content server 200 . The terminal display unit 320 displays the received visual content on the terminal 300 , and the terminal audio output unit 330 outputs the received audio content to the terminal 300 . The terminal controller 340 controls the content received by the terminal communication unit 310 to be displayed on the terminal display unit 320 and output to the terminal sound output unit 330 . In addition, the terminal control unit 340 may display the content on the terminal display unit 320 and output it to the terminal audio output unit 330 according to a signal input through the input unit 350 .

For example, referring to FIGS. 1 and 7C , among users U1 to U8 belonging to a cluster user environment, a user U5 who has difficulty viewing content displayed on the screen of the display unit 110 of the content control device 100 The terminal 300 of ) may display the same content as the content displayed on the screen of the display unit 110 of the content control device 100 through the terminal display unit 320 . The content provision of the terminal 300 may be performed by an input signal of the user U5 through the input unit 350, and the sensor unit 130 of the content control device 100 is able to provide the user U5 with difficulty viewing the content. It may be done by detecting

8 is a flow chart of an in-vehicle content control method for multiple users according to an embodiment of the present invention. Referring to FIGS. 1 to 8 , a content control method in a vehicle C for multiple users according to an embodiment of the present invention will be described as follows.

In step 410, the controller 140 determines the user environment by detecting at least one of the distance between the screen of the display unit 110 and the users U and the number of users U by the sensor unit 130. . For example, the controller 140 may determine whether the number of users U is less than or equal to a predetermined number. Also, the controller 140 may determine whether the distance between the screen of the display unit 110 and the users U is equal to or less than a predetermined distance.

In step 420, the controller 140 determines whether an interaction using one or more of the gaze, motion, gesture, voice, and touch of the users U detected by the sensor unit 130 is necessary based on the user environment. .

If the controller 140 determines that interaction using one or more of the gaze, motion, gesture, voice, and touch of the users U is necessary, in step 440, the controller 140 controls the sensor unit 130 based on the user environment. At least one of the gaze, motion, gesture, voice, and touch of the users U sensed by ) is selected as an interaction method.

For example, when the determined user environment is a 'proximity user environment' or a 'non-proximity user environment', the controller 140 performs an interaction using one or more of the users U's gaze, motion, gesture, voice, and touch. judge that this is necessary

Referring to FIGS. 5B to 5E , when the determined user environment is a 'proximity user environment', the controller 140 may, for example, perform touches and gestures of the users U detected by the sensor unit 130. , voice, and/or gaze may be selected as an interaction method.

Referring to FIGS. 6B to 6D , when the determined user environment is a 'non-proximity user environment', the controller 140 may, for example, determine the user U's gaze and / or the action can be selected as an interaction method.

Referring to FIG. 6E , the controller 140 may determine an interaction priority level regarding which user's interaction has priority by detecting gazes and/or motions of the users U1, U2, and U3. Specifically, when the motion of the first user U1 and the motion of the second user U2 are sensed at the same time, if the interaction of the first user U1 takes precedence over the interaction of the second user U2, the first user (U2) Content may be provided according to the operation of U1). However, the present invention is not limited thereto, and the control unit 140 detects a number of signals when a plurality of detection signals resulting from interactions of a plurality of users (U; U1, U2, and U3) are input through the sensor unit 130. It should be noted that signals may be integrated and processed.

When the controller 140 determines that interaction using one or more of the gaze, motion, gesture, voice, and touch of the users U is not necessary, in step 430, the controller 140 controls the display unit 110 and the sound output unit. One or more of (120) controls to provide one-way content.

For example, when the determined user environment is a 'group user environment', the controller 140 determines that interaction using one or more of the users U's gaze, motion, gesture, voice, and touch is not necessary, At least one of the display unit 110 and the sound output unit 120 is controlled to provide one-way content. For example, referring to FIG. 7B , the controller 140 determines the age and/or gender of the users U1 to U8 using the RGB-D camera 131, and determines the age and/or gender according to the determined age and/or gender. At least one of the display unit 110 and the sound output unit 120 may be controlled to provide customized content.

In step 450, the controller 140 controls one or more of the display unit 110 and the audio output unit 120 to provide interactive content according to the detection signal of the sensor unit 130 according to the interaction method.

For example, when the determined user environment is a 'proximity user environment', the controller 140 detects the sensor unit 130 according to interactions through the users U's touch, gesture, voice, and/or gaze. Depending on the signal, one or more of the display unit 110 and the audio output unit 120 may be controlled to provide interactive content. In addition, when the determined user environment is a 'non-proximity user environment', the control unit 140 displays the display unit 110 according to the detection signal of the sensor unit 130 according to the interaction through the gaze and/or motion of the users U. ) and one or more of the audio output unit 120 may be controlled to provide interactive content.

According to a first feature of an embodiment of the present invention, a content control method in a vehicle (C) for a plurality of users (U) is a) a screen of a display unit 110 by a sensor unit 130 and the users Determining a user environment by detecting at least one of a distance to (U) and the number of users (U); b) selecting one or more of the gaze, motion, gesture, voice, and touch of the users U detected by the sensor unit 130 based on the user environment as an interaction method; and c) controlling at least one of the display unit 110 and the audio output unit 120 to provide interactive content according to the detection signal of the sensor unit 130 according to the interaction method. Accordingly, a plurality of users (U) can smoothly manipulate content in a vehicle (C; C1, C2) for a plurality of users (U) having a wide space, such as, for example, an unmanned shuttle bus. have.

According to a second feature of the embodiment of the present invention, a content control method in a vehicle (C) for a plurality of users (U) is a) the screen of the display unit 110 by the sensor unit 130 and the users Determining a user environment by detecting at least one of a distance to (U) and the number of users (U); b) determining whether an interaction using one or more of the gaze, motion, gesture, voice, and touch of the users U detected by the sensor unit 130 is necessary based on the user environment; and c) controlling at least one of the display unit 110 and the sound output unit 120 to provide one-way content according to whether the interaction is required. Accordingly, it is possible to provide suitable content within the vehicle C for a plurality of users U according to the user environment.

Although the present invention has been shown and described with reference to the preferred embodiments of the accompanying exemplary drawings, it is not limited thereto, and those skilled in the art to which the present invention pertains within the scope of the technical idea of the present invention described in the claims below. Of course, it can be implemented in various forms.

100, 101, 102: content control device
110: display unit 120: sound output unit
130: sensor unit 131: RGB-D camera
132: infrared camera 133: microphone
134: touch screen 140: control unit
150: storage unit 160: communication unit
200: content server 210: server storage unit
220: server communication unit 230: server control unit
300: terminal 310: terminal communication unit
320: terminal display unit 330: terminal sound output unit
340: terminal control unit C, C1, C2: vehicle
U, U1~U8: User

Claims (10)

A content control method in a vehicle for multiple users,
a) detecting the number of users viewing the screen of the display unit by a sensor unit including an RGB-D camera and an infrared camera;
b) determining whether the number of users looking at the screen is less than or equal to a predetermined number;
c) when the number of users looking at the screen is less than or equal to a predetermined number, selecting the gaze of the users detected by the sensor unit in an interactive manner; and
d) controlling at least one of the display unit and the sound output unit to provide interactive content requiring interaction according to a detection signal of the sensor unit according to the interaction method;
Content control method in a vehicle for a plurality of users, characterized in that it comprises a. The method of claim 1, wherein in step b), when the number of users viewing the screen is less than or equal to the predetermined number, a distance between the screen of the display unit and the users viewing the screen is less than or equal to a predetermined distance. A method of controlling content in a vehicle for a plurality of users, comprising the step of determining whether or not the content is recognized. The method of claim 2, wherein in step c), when the number of users viewing the screen is less than or equal to a predetermined number and a distance between the screen of the display unit and the users viewing the screen is less than or equal to the predetermined distance , and further selecting one or more of the users' touches, gestures, and voices as the interaction method. 3 . The method of claim 2 , wherein in step c), the number of users viewing the screen is less than or equal to a predetermined number and a distance between the screen of the display unit and the users viewing the screen exceeds the predetermined distance. content control method in a vehicle for a plurality of users, characterized in that further selecting the actions of the users in the interaction method when doing so. In a content control device in a vehicle for multiple users,
a display unit having a screen on which visual content is displayed;
an audio output unit that outputs audio content;
a sensor unit including an RGB-D camera and an infrared camera that detects the number of users looking at the screen, and detects one or more of the users' gaze, motion, gesture, voice, and touch; and
It is determined whether the number of users looking at the screen is less than or equal to a predetermined number of users, and when the number of users looking at the screen is less than or equal to a predetermined number of users, the user's gaze detected by the sensor unit is used as an interaction method. and a control unit for controlling at least one of the display unit and the sound output unit to provide interactive content requiring interaction according to a detection signal of the sensor unit according to the interaction method.
An in-vehicle content control device for multiple users, comprising: The method of claim 5 , wherein the controller determines whether a distance between the screen of the display unit and the users viewing the screen is less than or equal to a predetermined distance when the number of users viewing the screen is less than or equal to the predetermined number of users. An in-vehicle content control device for multiple users, characterized in that for determining The method of claim 6, wherein the control unit, when the number of users viewing the screen is less than or equal to a predetermined number and a distance between the screen of the display unit and the users viewing the screen is less than or equal to the predetermined distance, A content control device in a vehicle for multiple users, characterized in that further selecting one or more of users' touches, gestures, and voices as the interaction method. The method of claim 6, wherein the control unit, when the number of users viewing the screen is less than or equal to a predetermined number and a distance between the screen of the display unit and the users viewing the screen exceeds the predetermined distance , In-vehicle content control device for multiple users, characterized in that the operation of the users is further selected by the interaction method. 6. The method of claim 5, wherein the vehicle is implemented by any one of an autonomous bus, a van, an SUV vehicle, and a train in which multiple users can ride. Device. The apparatus of claim 5 , wherein the display unit is implemented by any one of a touch screen panel display, a transparent display, and a rollable display.

Images