JP2023030054A - Sound apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it easy to specify an exact position.

SOLUTION: An acoustic apparatus 1 generates and acquires a three-dimensional space image in a passenger compartment generated on the basis of point group information acquired from a lidar 7 by a control unit 4 and outputs the three-dimensional space image to a display unit 6 in order to accept a designation of a localization position from a touch panel 5 that can accept the designation of the localization position.

SELECTED DRAWING: Figure 1

COPYRIGHT: (C)2023,JPO&INPIT

Description

The present invention relates to an acoustic device that localizes a sound image in a predetermined space formed by sounds emitted from a plurality of speakers at a specific position in the predetermined space.

For example, in an in-vehicle audio system, it is already known to control the localization position of a sound image by giving a delay time difference to each output of a plurality of speakers.

Japanese Patent Application Laid-Open No. 2002-200000 discloses an audio signal output to each speaker 3 that accepts designation of a localization position in accordance with a position input on a dashboard image 350 of a localization position reception window 300, and localizes a sound image to the accepted localization position. Setting the delay time is described. It is also described that designation of the localization position is performed by displaying a vehicle interior layout image 850, which is a layout image in the horizontal direction of the vehicle, to receive the designation of the localization position.

JP 2006-196941 A

In the case of the invention described in Patent Document 1, since the localization position is designated by a two-dimensional position, for example, one direction out of the width direction, the height direction, and the depth direction of the vehicle cannot be designated, and it cannot be specified accurately. It was difficult to specify the correct stereotactic position.

One example of the problem to be solved by the present invention is to facilitate designation of an accurate stereotactic position.

In order to solve the above-mentioned problems, the invention according to claim 1 is an acoustic device comprising a localization control unit that localizes a sound image in a predetermined space formed by sounds emitted from a plurality of speakers to a specific position in the predetermined space. an acquisition unit that acquires a three-dimensional spatial image of the predetermined space generated based on the three-dimensional information acquired from a sensor capable of acquiring three-dimensional information of the predetermined space; and a reception that can receive designation of the specific position. and an output unit configured to output the three-dimensional spatial image acquired by the acquisition unit to a display unit in order to accept designation of the specific position from the unit.

According to a seventh aspect of the present invention, an acquisition unit acquires a three-dimensional spatial image of the predetermined space generated based on three-dimensional information acquired from a sensor capable of acquiring three-dimensional information of the predetermined space; a display unit for displaying the three-dimensional spatial image acquired by the acquisition unit to accept the specification of the specific position from a reception unit capable of accepting the specification of the position; and the specific position received by the reception unit from multiple speakers. and an output unit for outputting information on a position for localizing a sound image formed by the emitted sound.

According to an eighth aspect of the invention, there is provided a sound control method executed by an acoustic device for localizing a sound image in a predetermined space formed by sounds emitted from a plurality of speakers at a specific position in the predetermined space, wherein an acquisition step of acquiring a three-dimensional spatial image of the predetermined space generated based on the three-dimensional information acquired from a sensor capable of acquiring three-dimensional information; and an output step of outputting the three-dimensional spatial image acquired in the acquisition step to a display unit in order to accept designation.

According to a ninth aspect of the invention, the acoustic control method according to the eighth aspect is executed by a computer.

1 is a functional configuration diagram of a system having an acoustic device according to a first embodiment of the present invention; FIG. 2 is a flowchart of the operation of the audio device shown in FIG. 1; FIG. 3 is a diagram of an example of a three-dimensional spatial image; FIG. 4 is a view showing the three-dimensional spatial image shown in FIG. 3 from a side view of the vehicle compartment; FIG. 4 is a view showing the three-dimensional space image shown in FIG. 3 from the viewpoint of the vehicle interior viewed from the front; FIG. 10 is a functional configuration diagram of a system having an acoustic device according to a second embodiment of the present invention; 7 is a flowchart of the operation of the audio device shown in FIG. 6; 9 is a flow chart showing the operation of the audio device according to the third embodiment of the present invention; It is a figure of the example which identified and displayed a person. FIG. 10 is an explanatory diagram of an operation of following a person; 14 is a flow chart showing the operation of the audio device according to the fourth embodiment of the present invention;

An acoustic device according to an embodiment of the present invention will be described below. An audio device according to an embodiment of the present invention acquires a three-dimensional spatial image of a predetermined space generated based on three-dimensional information acquired by an acquisition unit from a sensor capable of acquiring three-dimensional information of a predetermined space, and an output unit The three-dimensional spatial image acquired by the acquiring unit is output to the display unit in order to accept the specification of the specific position from the accepting unit capable of accepting the specification of the specific position. By doing so, it is possible to specify a specific position for localizing a sound image based on the three-dimensional spatial image displayed on the display unit, and it is easy to specify an accurate localization position by the three-dimensional spatial image that reproduces the predetermined space. can be

Also, the three-dimensional spatial image may be sequentially updated, and the output unit may output the updated three-dimensional spatial image. By doing so, it is possible to designate a specific position based on a three-dimensional spatial image corresponding to the entry and exit of a person in a predetermined space, the change in position, and the like.

Further, an identification unit for identifying a person in the three-dimensional space image acquired by the acquisition unit is further provided, the output unit outputs the three-dimensional space image in which the person is identified, and the reception unit is capable of specifying the person. Yes, when a person is specified by the reception unit, the localization control unit may localize the sound image at a position corresponding to the specified person. By doing so, for example, when a person is seated in a seat inside the vehicle, by designating the person, it is possible to localize the sound image at the position corresponding to the person.

The identification unit detects a change in the position of the person's head from a plurality of three-dimensional spatial images. The localization position of the sound image may be tracked by the head based on the change in position. By doing so, even if the orientation of the face, the angle of the seat, or the like changes, the localization position of the sound image can be followed accordingly.

Further, the sensor may measure the distance to an object by emitting electromagnetic waves into a predetermined space and receiving electromagnetic waves reflected by an object in the predetermined space. By doing so, it is possible to generate a three-dimensional spatial image based on the point group information acquired by a so-called lidar (LiDAR: Light Detection And Ranging).

Also, the audio device may be installed in a vehicle, and the predetermined space may be a vehicle interior space. By doing so, the sound image in the cabin space of the vehicle can be localized at a specific position.

Further, the terminal device according to an embodiment of the present invention acquires and displays a three-dimensional spatial image of a predetermined space generated based on three-dimensional information acquired by the acquisition unit from a sensor capable of acquiring three-dimensional information of the predetermined space. The three-dimensional spatial image acquired by the acquiring unit is displayed so that the unit can accept the specification of the specific position from the accepting unit that can accept the specification of the specific position in the predetermined space. Then, the output unit outputs the specific position received by the receiving unit as position information for localizing the sound image formed by the sounds emitted from the plurality of speakers. By doing so, it is possible to specify a specific position for localizing a sound image based on the three-dimensional spatial image displayed on the display unit, and to accurately localize the localization position from the terminal device using the three-dimensional spatial image that reproduces the predetermined space. specification can be facilitated.

Further, the acoustic control method according to one embodiment of the present invention acquires a three-dimensional spatial image of a predetermined space generated based on three-dimensional information acquired from a sensor capable of acquiring three-dimensional information of the predetermined space in the acquiring step, In the output step, the three-dimensional spatial image acquired by the acquiring unit for accepting the specification of the specific position from the accepting unit capable of accepting the specification of the specific position is output to the display unit. By doing so, it is possible to specify a specific position for localizing a sound image based on the three-dimensional spatial image displayed on the display unit, and it is easy to specify an accurate localization position by the three-dimensional spatial image that reproduces the predetermined space. can be

Also, the acoustic control method described above may be executed by a computer. By doing so, it is possible to use a computer to specify a specific position for localizing a sound image based on the three-dimensional spatial image displayed on the display unit, and it is possible to easily specify an accurate localization position. .

An acoustic device according to a first embodiment of the present invention will be described with reference to FIGS. 1 to 5. FIG. The audio device 10 is mounted in a vehicle such as an automobile, for example. FIG. 1 shows a functional configuration of an audio system having an audio device 10. As shown in FIG. The acoustic system includes an audio device 1, a delay adjustment unit 2 (2R, 2L), speakers 3 (3R, 3L), a control unit 4, a touch panel 5, a display unit 6, and a rider 7. there is Also, in this embodiment, the sound system is mounted on the vehicle and controls the sound inside the vehicle as a predetermined space.

The audio device 1 is a device, such as a CD player, a memory audio device, a radio tuner, or a receiving device for streaming distribution, which outputs an audio signal to be listened to by a user.

In this embodiment, the delay adjustment section 2 comprises a left channel delay adjustment section 2L and a right channel delay adjustment section 2R. The delay adjustment unit 2L delays the left-channel audio signal output from the audio device 1 to the left-channel speaker 3L by a set delay time. The delay adjustment unit 2R delays the right channel audio signal output from the audio device 1 to the right channel speaker 3R by a set delay time.

The controller 4 includes a generator 41 and a sound image controller 42 . The control unit 41 is composed of, for example, a microcomputer having a CPU. The generation unit 41 and the sound image control unit 42 may be implemented as functions of computer programs executed by the CPU.

The generator 41 models the point group information output from the lidar 7 with polygons or the like by a well-known method to generate a three-dimensional spatial image. That is, the control unit 4 functions as an acquisition unit that generates and acquires a three-dimensional spatial image based on the point cloud information. In addition, the generation unit 41 outputs the generated three-dimensional spatial image to the display unit 6 and causes the display unit 6 to display the image. That is, the control unit 4 (generation unit 41) functions as an output unit that outputs the three-dimensional spatial image to the display unit 6. FIG.

The sound image control unit 42 acquires the position coordinates touched on the touch panel 5, and specifies the three-dimensional coordinates (x, y, z) on the three-dimensional space image corresponding to the position coordinates. Then, the delay amounts of the delay adjustment units 2L and 2R are adjusted and set so that the sound image formed by the sound emitted from the speakers 3L and 3R is localized at the position in the real space corresponding to the specified three-dimensional coordinates. . That is, the control unit 4 (sound image control unit 42) functions as a localization control unit that localizes the sound image in the vehicle interior formed by the sounds emitted from the speakers 3L and 3R to a specific position.

The touch panel 5 is arranged so as to overlap the display surface of the display unit 6 . The touch panel 5 is a well-known input device that outputs two-dimensional coordinates of a position touched by a finger or the like. The display unit 6 is composed of, for example, a liquid crystal display. The display unit 6 displays a three-dimensional spatial image and the like generated by the control unit 4 . In this embodiment, the touch panel 5 is arranged so as to overlap the display surface of the display unit 6. However, the display unit 6 is arranged on the dashboard and can be used as a touch pad near the shift lever, the steering wheel, or the like. may be arranged separately, such as in the case of

The lidar 7 is installed in a vehicle interior as a predetermined space or at a position capable of scanning the interior of the vehicle. The lidar 7 is a sensor that recognizes an object existing in the surroundings scanned by the lidar 7, and is also referred to as LiDAR (Light Detection And Ranging). The lidar 7 emits an electromagnetic wave such as a laser beam, and discretely measures the direction and distance to an object existing within the scanning range based on the reflected wave (reflected light) of the electromagnetic wave, and determines the position, shape, etc. of the object. This is a known sensor that recognizes as a three-dimensional point cloud. Therefore, the point cloud recognized by the lidar 7 is output as point cloud information as three-dimensional information.

The number of riders 7 is not limited to one, and a plurality of riders may be installed. In the case of a vehicle, for example, the number and installation positions of the riders 7 may be appropriately determined according to the size of the vehicle interior space to be covered.

Here, the above-described delay adjustment unit 2 and control unit 4 function as the acoustic device 10 according to this embodiment.

Next, the operation (acoustic control method) of the acoustic device 10 having the above configuration will be described with reference to FIGS. 2 to 5. FIG. The flowchart shown in FIG. 2 is executed by the control unit 4 . Further, by configuring the flowchart shown in FIG. 2 as a program to be executed by the CPU of the control unit 4, the sound control program can be obtained.

First, in step S<b>11 , the generator 41 acquires point group information from the rider 7 . Next, in step S12, the generator 41 models the point group information acquired from the lidar 7 with polygons or the like by a well-known method to generate a three-dimensional spatial image, as described above. If there are multiple riders 7, a three-dimensional spatial image may be generated by synthesizing multiple pieces of point group information. An example of a three-dimensional spatial image is shown in FIG. FIG. 3 is a three-dimensional spatial image of the interior of the vehicle viewed from an oblique upper left viewpoint. Although FIG. 3 shows a viewpoint from diagonally upper left front, it is possible to change the viewpoint to an arbitrary viewpoint by operating the touch panel 5 to rotate, zoom in, zoom out, or the like.

Next, in step S13, the generation unit 41 causes the display unit 6 to display the three-dimensional spatial image generated in step S12.

Next, in step S<b>14 , the sound image control unit 42 obtains (accepts) a designation of a specific position (hereinafter simply referred to as a localization position) for localizing the sound image from the touch panel 5 . Here, a method of specifying a coordinate position indicating a localization position from a three-dimensional spatial image displayed on the display unit 6 will be described. In the three-dimensional space image of the viewpoint shown in FIG. 3, even if the position of the driver's seat is specified by touching, for example, the display on the display unit 6 is two-dimensional, so the correct three-dimensional coordinates ( x, y, z) are difficult to specify. Therefore, in this embodiment, first, the vehicle interior as shown in FIG. After having the user specify the position in the direction, the vehicle interior as shown in FIG. Ask the user to specify the position in . In this way, the three-dimensional coordinates of the specified localization position in the three-dimensional spatial image are identified.

Note that the display order of FIGS. 4 and 5 may be reversed. If the display order of FIGS. 4 and 5 is reversed, the user is asked to specify the position of the orientation position P in the width direction and height direction of the vehicle in the display of FIG. The user is asked to specify the position of P in the depth direction of the vehicle. Further, when displaying FIG. 5 after displaying FIG. 4, the position in the height direction specified in FIG. 4 may be displayed so that it can be referred to when specifying in FIG. The position of the localization position P in the height direction may be redesignated.

Furthermore, the view may be from above instead of from the side. In short, it suffices if positions in at least two directions out of the width direction, the height direction, and the depth direction of the vehicle can be specified.

That is, in step S13, the control unit 4 outputs the three-dimensional spatial image to the display unit 6 in order to receive the designation of the localization position from the touch panel 5 (reception unit) capable of receiving the designation of the specific position for localizing the sound image. In step S14, designation of a localization position is received from the touch panel 5. FIG.

Next, in step S15, the sound image control section 42 adjusts the delay adjustment sections 2L and 2R so that the sound image is localized at the localization position specified in step S14. Specifically, since the arrangement positions of the speakers 3L and 3R in the vehicle interior are known in advance, the position coordinates of the speakers 3L and 3R in the three-dimensional spatial image are also known. Then, the distance from the localization position specified in step S14 to the speaker 3L and the distance from the localization position specified in step S14 to the speaker 3R are obtained based on their respective coordinates. Then, the output from the speaker 3 with the shorter distance obtained is delayed according to the difference in distance from the speaker 3 with the longer distance.

Next, in step S16, the sound image control unit 42 causes the display unit 6 to display, for example, a message about whether or not readjustment is necessary. goes to step S17. On the other hand, if there is an input from the touch panel 5 indicating that readjustment is unnecessary (in the case of NO), the flow chart ends.

Next, in step S17, it is determined whether or not the three-dimensional spatial image needs to be regenerated. If necessary (YES), the process returns to step S11. If not necessary (NO), the process returns to step S14. . In step S17, for example, the display section 6 displays whether or not the three-dimensional spatial image needs to be regenerated, and the user is allowed to select it. If the user selects that it is necessary, the process returns to step S11 to generate a three-dimensional spatial image again. If the user selects that it is not necessary, the process returns to step S14 to acquire the localization position. In this manner, the three-dimensional spatial image is successively updated, and the updated three-dimensional spatial image is output to the display section 6 as necessary.

As is clear from the above description, step S12 functions as an acquisition process, and step S13 functions as an output process.

In addition, in the flowchart of FIG. 2, the three-dimensional spatial image is sequentially updated, but it may be generated once and then not updated. Also, the generated three-dimensional spatial image may be stored in a memory or the like within the control unit 4 . The storage destination is not limited to the control unit 4, and may be, for example, a detachable memory card or the like. If the images are stored in the external server, the next time the localization position is specified, steps S11 and S12 can be omitted, and steps S13 and subsequent steps can be executed after executing the step of acquiring the three-dimensional spatial image from the external server. .

According to this embodiment, the acoustic device 1 can generate and acquire a three-dimensional spatial image of the vehicle interior generated by the control unit 4 based on the point cloud information acquired from the lidar 7, and can accept designation of the localization position. A three-dimensional spatial image is output to the display unit 6 in order to receive designation of the localization position from the touch panel 5 . By doing so, it is possible to designate a specific position for localizing a sound image based on the three-dimensional space image displayed on the display unit 6, and to designate an accurate localization position by a three-dimensional space image that reproduces the vehicle interior space. can be facilitated.

Also, the three-dimensional spatial image is successively updated, and the control unit 4 outputs the updated three-dimensional spatial image. By doing so, it is possible to specify the localization position based on the three-dimensional spatial image corresponding to the boarding and alighting of the passenger in the vehicle compartment.

Next, an acoustic device 10A according to a second embodiment of the invention will be described with reference to FIGS. 6 and 7. FIG. The same reference numerals are given to the same parts as those of the first embodiment described above, and the description thereof will be omitted.

FIG. 6 shows a functional configuration of an audio system having an audio device 10A according to this embodiment. The acoustic system has an onboard device 20 , speakers 3 ( 3 L, 3 R), a rider 7 and a terminal device 30 . Here, the speakers 3 (3L, 3R) and the rider 7 are the same as in the first embodiment.

The in-vehicle device 20 is mounted in a vehicle, for example. The in-vehicle device 20 includes an audio device 1, a delay adjustment section 2 (2L, 2R), a control section 4A, and a communication section 8, as shown in FIG.

The audio equipment 1 and the delay adjustment section 2 are the same as in the first embodiment. The control unit 4A includes a generation unit 41A and a sound image control unit 42A.

The generation unit 41A generates a three-dimensional spatial image in the same manner as in the first embodiment. Different from the example.

The sound image control unit 42A adjusts the delay amounts of the delay adjustment units 2L and 2R so as to localize the sound image formed by the sounds emitted from the speakers 3L and 3R at the position in the real space corresponding to the calculated three-dimensional coordinates. is the same as in the first embodiment, but the coordinates of the position where the touch panel 33 is touched are acquired from the terminal device 30 via the communication unit 8, and a three-dimensional image on the three-dimensional space image corresponding to the position coordinates This embodiment differs from the first embodiment in that the coordinates are specified.

The communication unit 8 transmits the three-dimensional spatial image generated by the generation unit 41</b>A to the terminal device 30 and receives the coordinates of the position where the touch panel 33 is touched from the terminal device 30 . The communication unit 8 communicates with the terminal device 30 by wireless communication such as Bluetooth (registered trademark), Wi-Fi (registered trademark), or infrared communication.

The terminal device 30 is, for example, a portable electronic device such as a smartphone. The terminal device 30 includes a communication section 31 , a control section 32 , a touch panel 33 and a display section 34 .

The communication unit 31 transmits the coordinates of the position where the touch panel 33 is touched to the vehicle-mounted device 20 and receives the three-dimensional spatial image generated by the generation unit 41A from the vehicle-mounted device 20 .

The control unit 32 manages overall control of the terminal device 30 . The control unit 32 causes the display unit 34 to display the three-dimensional spatial image received by the communication unit 31 , acquires the position coordinates touched on the touch panel 33 , and transmits them to the vehicle-mounted device 20 via the communication unit 31 .

The touch panel 33 is arranged so as to overlap the display surface of the display unit 34 . The touch panel 33 is a well-known input device that outputs two-dimensional coordinates of a position touched by a finger or the like. The display unit 34 is configured by, for example, a liquid crystal display. The display unit 34 displays a three-dimensional spatial image or the like output from the control unit 32 .

Here, the delay adjustment unit 2, the control unit 4, and the communication unit 8 described above function as the acoustic device 10A according to the present embodiment.

In addition, the communication unit 31 of the terminal device 30 includes an acquisition unit that acquires a three-dimensional space image of the vehicle interior generated based on the point cloud information acquired from the rider 7, and a position that localizes the sound image at the specific position received by the touch panel 33 It functions as an output unit that outputs as information of In addition, the display unit 34 functions as a display unit that displays the three-dimensional spatial image received by the communication unit 31 in order to receive designation of a specific position from the touch panel 33 capable of receiving designation of a specific position for localizing a sound image inside the vehicle. do.

Next, the operation (acoustic control method) of the acoustic device 10A having the configuration described above will be described with reference to FIG. The flowchart shown in FIG. 7 is executed by the control unit 4A of the vehicle-mounted device 20 and the control unit 32 of the terminal device 30. FIG.

First, the operation of the in-vehicle device 20 will be described. In step S<b>21 , the control unit 4</b>A acquires point group information from the rider 7 . Next, in step S22, the control section 4A generates a three-dimensional spatial image for the point cloud information obtained from the lidar 7 by a well-known method.

Next, in step S23, the control unit 4A determines whether or not there is a request for a 3D spatial image by receiving request information for a 3D spatial image from the terminal device 30, and if there is no request (in the case of NO ) returns to step S21, and if there is a request (if YES), the communication unit 8 is caused to transmit the three-dimensional spatial image generated by the generation unit 41A to the terminal device 30 in step S24. If there is no request in step S23 and the process returns to step S21, the point group information is obtained again (step S21), and the three-dimensional spatial image is regenerated (step S22), thereby sequentially updating.

Next, in step S25, the control section 4A receives the localization position information from the terminal device 30 via the communication section 8. FIG. Then, in step S26, the control section 4A adjusts the delay adjustment sections 2L and 2R so that the sound image is localized at the localization position received in step S24, and returns to step S21. The adjusting method is the same as in the first embodiment.

When the process returns to step S21, the point group information is acquired again (step S21), and the three-dimensional spatial image is regenerated (step S22), thereby sequentially updating. Then, if there is request information for a regenerated three-dimensional spatial image from the terminal device 30, step S24 and subsequent steps are executed, and if there is no request information for a three-dimensional spatial image, the process returns to step S21.

Next, the operation of the terminal device 30 will be described. In step S<b>31 , the control unit 32 causes the communication unit 31 to transmit request information for a three-dimensional spatial image to the onboard device 20 . Next, in step S<b>32 , the control unit 32 receives the three-dimensional spatial image from the vehicle-mounted device 20 via the communication unit 31 .

Next, in step S33, the control unit 32 causes the display unit 34 to display the three-dimensional spatial image received in step S32.

Next, in step S<b>34 , the control unit 32 acquires (accepts) the designation of the localization position of the sound image from the touch panel 33 . The method of acquiring the stereotactic position is the same as in the first embodiment.

Next, in step S35, the control section 32 causes the communication section 31 to transmit the localization position acquired in step S34 to the vehicle-mounted device 20 as localization position information.

Next, in step S36, the control unit 32 causes the display unit 34 to display, for example, a message about whether or not readjustment is necessary. The process proceeds to step S37. On the other hand, if there is an input from the touch panel 33 indicating that readjustment is unnecessary (in the case of NO), the flowchart ends.

Next, in step S37, it is determined whether or not the three-dimensional spatial image needs to be regenerated. If necessary (YES), the process returns to step S31. If not necessary (NO), the process returns to step S34. . In step S37, for example, the display unit 6 displays whether or not the three-dimensional spatial image needs to be regenerated, and the user is prompted to select it. If the user selects that it is necessary, the process returns to step S31 to request the regenerated three-dimensional spatial image. If the user selects that it is not necessary, the process returns to step S34 to acquire the localization position.

According to this embodiment, the terminal device 30 obtains a three-dimensional spatial image of the vehicle interior generated by the control unit 32 based on the three-dimensional information obtained from the lidar 7, and the display unit 34 designates the localization position in the vehicle interior. A three-dimensional spatial image is displayed in order to receive the designation of the localization position from the touch panel 33 capable of receiving the . Then, the communication unit 31 outputs the localization position information. By doing so, it is possible to specify a localization position for localizing a sound image based on the three-dimensional spatial image displayed on the display unit 34, and it is possible to easily specify an accurate localization position from the terminal device 30. .

In addition, since the localization position can be specified from the portable terminal device 30, the localization position can be specified regardless of the installation position of the in-vehicle device 20. FIG.

Next, an acoustic device 10 according to a third embodiment of the invention will be described with reference to FIGS. 8 to 10. FIG. The same parts as those in the first and second embodiments described above are denoted by the same reference numerals, and descriptions thereof are omitted.

This embodiment has the same functional configuration as the first embodiment, but is partially different in the operation of the audio device 10 (sound control method). The operation (acoustic control method) of the acoustic device 10 according to the present embodiment will be described with reference to FIG. The flow chart shown in FIG. 8 is executed by the controller 4 .

First, in step S<b>41 , the generator 41 acquires point group information from the rider 7 . Next, in step S42, the generator 41 generates a three-dimensional spatial image for the point group information acquired from the rider 7, as described above.

In step S42, when generating a three-dimensional spatial image, the generation unit 41 recognizes a person (passenger) in the vehicle interior by object recognition or the like from point cloud information, and identifies the person in the three-dimensional spatial image. can be displayed as That is, the control unit 4 (generation unit 41) functions as an identification unit that identifies a person in the three-dimensional spatial image. FIG. 9 shows an example of a three-dimensional spatial image in which a person has been identified. FIG. 9 is obtained by adding a person to FIG. As shown in FIG. 9, the persons are given labels such as "A" and "B" so that they can be identified. It should be noted that different colors, patterns, or the like may be added to each person in place of the label.

Next, in step S43, the generation unit 41 causes the display unit 6 to display the three-dimensional spatial image generated in step S42.

Next, in step S<b>44 , the sound image control unit 42 acquires (accepts) the designation of the person whose sound image is to be localized from the touch panel 5 . For example, in FIG. 8, a person is specified by touching either "A" or "B". In this embodiment, the localization position is set to the front center of the face (head) regardless of whether the person touches the head or the body. The position coordinates of this localization position are automatically specified based on the position of the person in the three-dimensional spatial image. Also, the localization position that is automatically specified when a person is touched is not limited to the front center of the face, and may be another position such as an ear. Alternatively, for example, if the chest is touched, the chest may be the stereotactic position.

Next, in step S45, the sound image control unit 42 adjusts the delay adjustment units 2L and 2R so that the sound image is localized on the head of the person specified in step S44. The adjusting method is the same as in the first embodiment. That is, when a person is specified, the control unit 4 (sound image control unit 42) localizes the sound image at a position corresponding to the specified person.

Next, in step S<b>46 , the generator 41 acquires point cloud information from the rider 7 . This step S46 is performed for the determination of step S47. Then, in step S47, it is determined whether or not there is a change in the follow-up position. If there is a change, the process returns to step S45, and if there is no change, the process returns to step S46.

The follow-up position in step S47 is the head of the person whose sound image is localized in step S45. This is because the position of the person's head in the three-dimensional spatial image generated from the point cloud information acquired in step S46 has changed from the position of the person's head acquired in step S41 or previously executed step S46. If so, it is determined that the head has moved, and the process returns to step S45 to follow the movement of the head. Then, the sound image is localized at the position of the person's head in the three-dimensional space image generated from the point group information acquired in step S46.

An example of tracking is shown in FIG. FIG. 10 shows an example in which a person sitting on a seat tilts the backrest to change the angle of the backrest. In the case of FIG. 10, when the angle of the seat is changed, the orientation position P moves to the orientation position P' following the change in the position of the head.

On the other hand, when the position of the person's head in the three-dimensional space image generated from the point cloud information acquired in step S46 has not changed from the position of the person's head acquired in step S41 or the previously executed step S46 If the head does not move, step S45 is not executed, and point cloud information is acquired again in step S46.

In this embodiment, a three-dimensional spatial image may be generated (updated) based on the point cloud information acquired in step S46 and displayed on the display unit 6. FIG.

The flowchart shown in FIG. 8 is the operation when a person is specified. For example, in the three-dimensional spatial image shown in FIG. may be performed in the same manner as in the first embodiment. That is, according to the touch position (specified position), steps S44 and subsequent steps may be executed when a person is touched, and steps S14 and subsequent steps may be executed when a person other than a person is touched.

According to this embodiment, the generation unit 41 of the control unit 4 identifies a person in the three-dimensional space image, and outputs the three-dimensional space image in which the person is identified. Also, the touch panel 5 can specify a person, and when a person is specified on the touch panel 5, the delay adjustment unit 2 localizes the sound image at a position corresponding to the specified person. By doing so, for example, when a person is seated in a seat inside the vehicle, by designating the person, it is possible to localize the sound image at the position corresponding to the person.

In addition, the control unit 4 detects a change in the position of the person's head from a plurality of three-dimensional space images, and the delay adjustment unit 2 detects a change in the position of the person's head from the touch panel 5. The localization position of the sound image is made to follow the head based on the change in the position of the head. By doing so, even if the orientation of the face, the angle of the seat, or the like changes, the localization position of the sound image can be followed accordingly.

Next, an acoustic device 10A according to a fourth embodiment of the invention will be described with reference to FIG. The same parts as those of the first to third embodiments described above are denoted by the same reference numerals, and descriptions thereof are omitted.

This embodiment has the same functional configuration as the second embodiment, but is partially different in the operation (acoustic control method) of the audio device 10A. The operation (acoustic control method) of the acoustic device 10A according to this embodiment will be described with reference to FIG. The flowchart shown in FIG. 11 is executed by the control unit 4A of the vehicle-mounted device 20 and the control unit 32 of the terminal device 30. FIG.

First, the operation of the in-vehicle device 20 will be described. In step S<b>51 , the control unit 4</b>A acquires point group information from the rider 7 . Next, at step S52, the control section 4A generates a three-dimensional spatial image by a well-known method. It should be noted that the three-dimensional space image generated in step S52 can be displayed by identifying a person as in the third embodiment.

Next, in step S53, the control unit 4A determines whether or not there is a request for a 3D spatial image due to the reception of request information for a 3D spatial image from the terminal device 30. If there is no request (NO ) waits in this step, and if there is a request (in the case of YES), the communication unit 8 is caused to transmit the three-dimensional spatial image generated by the generation unit 41A to the terminal device 30 in step S54.

Next, in step S55, the control unit 4A receives from the terminal device 30 person designation information, which is information for designation of a person whose sound image is to be localized. Next, the delay adjusters 2L and 2R are adjusted so that the sound image is localized on the head of the person indicated by the person designation information received in step S55. The adjustment method is the same as in the first and third embodiments.

Next, in step S<b>57 , the generator 41 acquires point cloud information from the rider 7 . This step S57 is performed for the determination of step S58. Then, in step S58, it is determined whether or not there is a change in the follow-up position. If there is a change, the process returns to step S56, and if there is no change, the process returns to step S57. The follow-up method in step S58 is the same as in the third embodiment.

Also in this embodiment, a three-dimensional spatial image may be generated (updated) based on the point group information acquired in step S57 and displayed on the display unit 6. FIG.

Next, the operation of the terminal device 30 will be described. In step S<b>61 , the control unit 32 causes the communication unit 31 to transmit request information for a three-dimensional spatial image to the vehicle-mounted device 20 . Next, in step S<b>62 , the three-dimensional spatial image is received from the vehicle-mounted device 20 via the communication unit 31 .

Next, in step S63, the control unit 32 causes the display unit 34 to display the three-dimensional spatial image received in step S62.

Next, in step S<b>64 , the control unit 32 acquires (accepts) the designation of the person whose sound image is to be localized from the touch panel 5 . The method of obtaining the designation of a person is the same as in the third embodiment.

Next, in step S65, the control unit 32 causes the communication unit 31 to transmit the person specified in step S34 to the in-vehicle device 20 as person specifying information.

Next, in step S66, the control unit 32 causes the display unit 34 to display, for example, a message about whether or not readjustment is necessary. Return to step S64. On the other hand, if there is an input from the touch panel 33 indicating that readjustment is unnecessary (in the case of NO), the flowchart ends.

Also in the present embodiment, if there is an input indicating that readjustment is necessary in step S66, it may be determined whether or not the three-dimensional spatial image needs to be regenerated.

According to the present embodiment, the terminal device 30 acquires a three-dimensional space image in which a person is identified in the vehicle interior generated by the control unit 32 based on the three-dimensional information acquired from the rider 7, and the display unit 34 displays the image in the vehicle interior. A three-dimensional spatial image is displayed in order to accept the specification of a person to localize the sound image from the touch panel 33 capable of accepting the specification of the localization position. Then, the communication section 31 outputs the person designation information. By doing so, for example, when a person is seated in a seat in the passenger compartment of the vehicle, by designating the person from the terminal device 30, it is possible to localize the sound image at the position corresponding to the person. becomes.

In addition, the control unit 4A detects a change in the position of the person's head from a plurality of three-dimensional spatial images, and the delay adjustment unit 2 detects a change in the position of the person's head on the touch panel 5, and the control unit 4A detects The localization position of the sound image is made to follow the head based on the change in the position of the head. By doing so, even if the orientation of the face, the angle of the seat, or the like changes, the localization position of the sound image can be followed accordingly.

In the above-described embodiment, the lidar is used as the sensor, but other sensors may be used. For example, any sensor capable of acquiring three-dimensional information for generating a three-dimensional spatial image, such as a stereo camera, may be used.

In the above-described embodiment, the predetermined space is described as a vehicle compartment, but the predetermined space is not limited to the vehicle compartment, and may be an indoor space in a building. Further, although the above-described embodiment has been described with a two-channel sound system of right and left channels, it can be applied to a sound system that localizes a sound image using a plurality of speakers, such as a four-channel sound system consisting of four speakers.

Moreover, the present invention is not limited to the above embodiments. That is, those skilled in the art can carry out various modifications according to conventionally known knowledge without departing from the gist of the present invention. As long as the acoustic device of the present invention is still provided even with such modification, it is of course included in the scope of the present invention.

2 delay adjustment section (localization control section)
3 speaker 4 control unit (acquisition unit, output unit, identification unit)
4A Control unit (acquisition unit, identification unit)
5 Touch panel (reception part)
6 display unit 7 lidar 8 communication unit (output unit)
DESCRIPTION OF SYMBOLS 10, 10A Acoustic device 20 Vehicle-mounted device 30 Terminal device 31 Communication part (acquisition part)
32 control unit (output unit)
33 touch panel (reception part)
34 Display

Claims (16)

An acoustic device comprising a localization control unit that localizes a sound image in a predetermined space formed by sounds emitted from a plurality of speakers to a specific position in the predetermined space,
an acquisition unit for acquiring a three-dimensional spatial image of the predetermined space generated based on the three-dimensional information acquired from a sensor capable of acquiring three-dimensional information of the predetermined space;
an output unit for outputting to a display unit the three-dimensional spatial image acquired by the acquiring unit in order to accept the specification of the specific position from the accepting unit capable of accepting the specification of the specific position;
An acoustic device comprising: The three-dimensional spatial image is sequentially updated,
2. The acoustic device according to claim 1, wherein the output unit outputs the updated three-dimensional spatial image. further comprising an identification unit that identifies a person in the three-dimensional spatial image obtained by the obtaining unit;
The output unit outputs the three-dimensional spatial image in which the person is identified,
The reception unit is capable of specifying the person,
When the person is specified by the reception unit, the localization control unit localizes the sound image at a position corresponding to the specified person.
3. The acoustic device according to claim 1 or 2, characterized by: The identification unit detects a change in the position of the head of the person from the plurality of three-dimensional spatial images,
The localization control unit causes the head to follow the localization position of the sound image based on the change in the position of the head detected by the identification unit when the person is specified by the reception unit. 4. An acoustic device according to claim 3. 2. The sensor is capable of measuring the distance to the object by emitting electromagnetic waves into the predetermined space and receiving electromagnetic waves reflected by an object in the predetermined space. 5. The acoustic device according to any one of 4 to 4. 6. The acoustic device according to any one of claims 1 to 5, wherein the acoustic device is installed in a vehicle, and the predetermined space is a vehicle interior space. an acquisition unit that acquires a three-dimensional spatial image of a predetermined space generated based on three-dimensional information acquired from a sensor capable of acquiring three-dimensional information of the predetermined space;
a display unit for displaying the three-dimensional spatial image acquired by the acquiring unit for accepting the specification of the specific position from the accepting unit capable of accepting the specification of the specific position in the predetermined space;
an output unit that outputs the specific position received by the reception unit as information on a position to localize a sound image formed by sounds emitted from a plurality of speakers;
A terminal device comprising: A sound control method executed by an acoustic device for localizing a sound image in a predetermined space formed by sounds emitted from a plurality of speakers at a specific position in the predetermined space,
an acquisition step of acquiring a three-dimensional spatial image of the predetermined space generated based on the three-dimensional information acquired from a sensor capable of acquiring three-dimensional information of the predetermined space;
an output step of outputting the three-dimensional spatial image acquired in the acquisition step to a display unit in order to accept the designation of the specific position from a reception unit capable of accepting the designation of the specific position;
An acoustic control method comprising: A sound control program for causing a computer to execute the sound control method according to claim 8 . an acquisition unit that acquires a three-dimensional spatial image of a predetermined space generated based on three-dimensional information acquired from a sensor;
an output unit that outputs the three-dimensional spatial image acquired by the acquisition unit to a display unit;
a reception unit that receives designation of a specific position in the three-dimensional spatial image output to the display unit;
The output unit outputs to the display unit at least two images in which two directions out of a width direction, a height direction, and a depth direction in the predetermined space can be identified based on the three-dimensional space image,
The reception unit receives designation of the specific position for each of at least two images output to the display unit;
An input device characterized by: a first image of the two images is displayed on the display unit;
After receiving the specification of the specific position for the first image by the receiving unit, a second image is displayed on the display unit instead of the first image,
receiving the designation of the specific position for the second image by the receiving unit;
11. The input device according to claim 10, characterized by: At least two of the images output to the display unit are
An image in which one of the two directions constituting the image is different from each other,
12. The input device according to claim 10 or 11, characterized in that: The output unit outputs a selection regarding reproduction of the three-dimensional spatial image to the display unit;
regenerating the three-dimensional spatial image according to the selection made by the user;
13. The input device according to any one of claims 10 to 12, characterized in that: Of the at least two images output to the display unit, one image designates a position in two directions out of the width direction, the height direction, and the depth direction in the predetermined space, and the other image designates , a position in one remaining direction different from the two directions is specified. 15. The input device according to any one of claims 10 to 14, further comprising a localization control unit that localizes a sound image formed by sounds emitted from a plurality of speakers to the specific position in the predetermined space. 16. The input device according to any one of claims 10 to 15, wherein the reception unit can directly receive designation of the specific position for at least two of the images output to the display unit. .

Images