JP6710428B1 - Acoustic space generator - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a device capable of more easily performing an improvisational performance based on a user's motion. SOLUTION: A camera 20 for photographing an object, a light source 30 for illuminating the object, an input unit 61 for acquiring an image from the camera, a processing unit 62 for selecting a sound to be generated according to a frame of a video, and processing. And a plurality of speakers 50 that output the sound selected by the unit, and the processing unit compares the light amount of the frame with the light amount of the frame immediately before the frame, and based on a preset threshold value of the light amount. , Determines whether the object is in motion or in a stationary state, and when it is determined to be in motion, generates a motion detection signal, and selects a pitch to be output by a pre-stored music structure program. , The spatial structure program stored in advance assigns one of a plurality of speakers to each sound of the selected pitch, outputs the sound from the allocated speaker, and determines the motion state in the first unit time. After a first predetermined number of consecutive occurrences, the first predetermined number of consecutive occurrences has reached a second predetermined number of times, and has been stationary for a second unit time or longer When it is determined that there is a specific sound, and when a specific speaker is assigned to a predetermined sound, a sine wave is generated from the specific speaker by the music structure program, and the sine wave is generated in the first unit time. When the state of motion occurs for the first predetermined number of times in succession and the number of times of occurrence for the first predetermined number of times reaches the third predetermined number of times, after being in a stationary state for a second unit time or longer, When it is determined that the sound is in motion, and when a specific speaker is assigned to the selected sound, the music structure program causes the specific reverberant sound to be output from the specific speaker. When the motion state continuously occurs for the fourth predetermined number of times in the unit time, the second reverberation sound is output from the speaker in the predetermined order by the music structure program.

Description

The present invention relates to an acoustic space generation device.

2. Description of the Related Art There is known a musical sound generating apparatus which is capable of automatically creating musical tone data corresponding to the movement of a person operating in front of a camera and playing the musical tone data. For example, the following Patent Document 1 discloses a configuration of a musical sound generation device that reacts to the motion of a person who is a subject of a camera, creates musical sound data based on the position and the amount of change of the motion, and outputs this as sound. Has been done. According to this musical sound generation device, the performer does not need to acquire the knowledge and technique for operating the device for automatically playing music or playing the musical instrument, and only needs to exercise in front of the camera to improvise You can play easily.

Japanese Patent No. 3643829

As described above, there is known a device that outputs a sound that reacts to the movement of the subject. However, there is a device that can recognize the movement of the subject from a viewpoint different from that of the related art and play a new musical sound generated by the recognition. For example, it is possible to further enjoy the musical sound generated according to the movement of the subject. In view of such a point, in the present invention, the state of motion of an object is different from that of the related art so that the musical sound generated according to the state of motion of the object (for example, the motion of the player) can be further enjoyed. The main object of the present invention is to provide a device which recognizes from different viewpoints and produces a new musical sound generated by the recognition.

Further, in the musical sound generating apparatus described in Patent Document 1, it is necessary to both specify the position of movement of the camera subject and calculate the amount of change in the subject movement. Therefore, the musical sound generating apparatus has a possibility of increasing the load of data processing when playing. In view of such a point, it is another object of the present invention to provide a device that can more easily perform an improvisational performance based on the state of movement of an object.

The present invention focuses on simple movements such as "moving" and "stopping", and the temporal sensation of various movements such as physical expression, breathing method, "pause", "pause", and ON/OFF of each thing (interval). Is intended to be a sound or syllable. Then, in the present invention, a device that mainly focuses on repetition of "motion" and "stillness" of an object, and based on this, generates and organizes melodies, syllables, and acoustic spaces in a specific scale. I will provide a.

That is, the present invention includes a camera that captures an object, a light source that illuminates the object, an input unit that acquires an image from the camera, a processing unit that selects a sound to be generated according to a frame of the image , and a processing unit. A plurality of speakers that output the selected sound are provided, and the processing unit compares the light amount of the frame with the light amount of the frame immediately before the frame, and based on a preset threshold of the light amount, the object It is determined whether it is a motion state or a stationary state, and when it is determined that it is a motion state, a motion detection signal is generated, and a pitch to be output is selected by a pre-stored music structure program and stored in advance. According to the spatial structure program, one of a plurality of speakers is assigned to each sound of the selected pitch, the sound is output from the assigned speaker, and the movement state is the first in the first unit time. occurs in a predetermined number of times continuously, the occurrence count reaches the second predetermined number of times, after which was a second unit time than the stationary state, when it is determined that the motion of the state, and a predetermined When a specific speaker among a plurality of speakers is assigned to a sound, a sine wave is generated from the specific speaker by a music structure program, and a motion state continues for a first predetermined number of times in a first unit time. When the number of occurrences reaches a third predetermined number of times, it is determined to be in a moving state after being in a stationary state for a second unit time or longer, and the selected sound is generated. On the other hand, when a specific speaker is allocated, the music structure program causes the specific reverberant sound to be output from the specific speaker, and the movement state is the first predetermined number of times to the third predetermined time in the first unit time . When a specific speaker is assigned to the selected sound , the second reverberation sound is generated by the music structure program when the sound is generated continuously for the fourth predetermined number of times set in advance together with the predetermined number of times. Is an acoustic space generation device that outputs the sound from a specific speaker in a predetermined order.

Further, the present invention provides a sensor capable of detecting the state of an object, an input unit that acquires a signal from the sensor, a processing unit that selects a sound to be generated according to the signal, and a sound that is selected by the processing unit. A plurality of speakers, and the processing unit compares the signal with a signal immediately before the signal and determines whether the object is in a moving state or a stationary state based on a threshold value of the signal set in advance. When it is determined to be in the state of motion, a motion detection signal is generated, a pitch to be output is selected by a music structure program stored in advance, and a pitch is selected by a space structure program stored in advance. for each sound of the pitch, assigned one of a plurality of speakers, to output the sound from the assigned speakers, motion in a first unit time condition occurs at a first predetermined consecutive number of times, this occurs When the number of times has reached the second predetermined number of times and is determined to be in the moving state after being in the stationary state for the second unit time or longer, and when the predetermined sound is detected, the speaker is identified from the plurality of speakers. when the speaker is assigned, by the music structure program to generate a sine wave from a particular speaker, motion in a first unit time condition occurs at a first predetermined consecutive number of times, the number of occurrences is 3 When a predetermined number of times has been reached, a certain speaker is assigned to the selected sound when it is determined to be in the moving state after the stationary state for the second unit time or longer. Then, the first reverberation sound is output from the specific speaker by the music structure program, and the motion state is preset in the first unit time together with the first predetermined number of times to the third predetermined number of times . 4, when a specific speaker is assigned to the selected sound , the second reverberation sound is output from the specific speaker in a predetermined order by the music structure program. This is an acoustic space generation device.

In the acoustic space generation device of the present invention, each state of motion and stillness is determined based on the movement of an object (for example, a subject), a sound is selected according to the frequency, cycle, number of times, etc. of each state, and the selected sound is selected. Is output as a musical sound. Therefore, according to the present invention, it is possible to easily play a musical sound according to the mode of movement of an object, and it is possible to easily provide an acoustic space in which such a musical sound is played. Further, according to the present invention, it is possible to play a new syllable musical sound or sound generated based on the repetitive motion of the motion and rest of the object.

It is the schematic which shows an example of the acoustic space production|generation apparatus which concerns on embodiment, (a) is the figure seen from the front side, (b) is the figure seen from above. It is a block diagram which shows the principal part of the acoustic space production|generation apparatus of FIG. It is a figure which shows an example of the usage condition of the acoustic space production|generation apparatus of FIG. 3 is a flowchart showing an example of the operation of the acoustic space generation device in FIG. 1. It is a flow chart which shows processing of a processing part at the time of outputting the sound of a predetermined pitch. It is a figure which shows the audio file of a pitch. It is a figure for explaining an example of operation of a treating part. It is a flow chart which shows generation processing of a sine wave. It is a figure for explaining an example of operation of a treating part. It is a flow chart which shows the output processing of the 1st reverberation sound. It is a figure for explaining an example of operation of a treating part. It is a flow chart which shows output processing of the 2nd reverberation sound. It is a figure for explaining an example of operation of a treating part.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not limited to this. Further, in the drawings, in order to describe the embodiment, a part of the drawing is enlarged or emphasized, and the scale is appropriately changed. 1A and 1B are diagrams showing an outline of an example of the configuration of an acoustic space generation device 100 according to an embodiment, FIG. 1A is a view seen from the front side, and FIG. 1B is a view seen from above. Note that FIG. 1A shows a state where the front wall portion 11b is transparent, and FIG. 1B shows a state where the ceiling portion 11c is transparent.

The acoustic space generation device 100 according to the embodiment is a device that generates a melody or a syllable in a specific scale and generates the acoustic space 10 in which the melody or syllable is played. In the embodiment, the acoustic space 10 is a space where music is played.

In the embodiment, the acoustic space 10 is an internal space of the structure 11. The structure 11 is a container-shaped structure and is formed to be hollow and movable. For example, as shown in FIG. 1, the structure 11 has a substantially square floor surface 11a in plan view, a wall surface 11b standing up from each side of the floor surface 11a, and a floor surface 11a above the floor surface 11a so as to block the internal space. And a ceiling surface 11c which is disposed at. The acoustic space 10 is a space defined by the floor surface 11a, the wall surface 11b, and the ceiling surface 11c of the structure 11. Note that the acoustic space 10 is not limited to the internal space of a container-shaped structure such as the structure 11 described above, and any space capable of propagating sound can be applied. That is, the acoustic space 10 may be, for example, an internal space of a building such as a concert hall or an event hall, or an underground space. The acoustic space 10 is not limited to a closed space, and may be, for example, a space on an outdoor stage or a space on the ground surface. In this case, a camera 20, a light source 30, a stand 40, a plurality of speakers 50, and the like, which will be described later, are installed, for example, on an outdoor stage or on the ground surface.

The acoustic space generation device 100 includes a camera 20, a light source 30, a base 40, a plurality of speakers 50, and an information processing device 60 (see FIG. 2). Although the camera 20, the light source 30, and the base 40 are installed inside the acoustic space 10, they may be installed outside the acoustic space 10. For example, the camera 20 and the like may be installed in a place away from the acoustic space 10.

The camera 20 is a device capable of photographing the movement of a subject (object) (that is, the hand of the user U) H (see FIG. 3). The camera 20 is installed near the ceiling portion 11c so as to face downward. The camera 20 shoots the subject H from above and picks up a moving image of the subject H. The camera 20 is arranged in the central portion of the acoustic space 10 in plan view. The camera 20 is supported in the air, and is supported by, for example, a support fitting (not shown) extending in the horizontal direction from the wall portion 11b. The camera 20 is not limited to the above configuration, and the orientation, installation location, installation method, etc. of the camera 20 can be set arbitrarily. For example, the camera 20 is installed in an upward or horizontal direction. It may be installed in a fixed state with respect to the ceiling part 11c, or may be provided in a state of being suspended from the ceiling part 11c.

The camera 20 continuously captures the subject H and transmits the captured images to the information processing device 60. Specifically, the camera 20 shoots the subject H at a preset frame rate and inputs the image data for each frame into the input unit 61 (see FIG. 2) of the information processing device 60. The camera 20 is connected to the information processing device 60 so as to be able to perform data communication, and the image data is input from the camera 20 via the cable (for example, USB or Ethernet (registered trademark)) or wirelessly (for example, various radio wave communication or the Internet) to the input unit 61. Sent to. The frame rate of the camera 20 is set according to the speed of movement of the subject H, the performance of the information processing device 60, and the like. In the embodiment, the frame rate of the camera 20 is set to 40 fps (frames per second). In this case, the camera 20 shoots the subject H at a frequency of 40 times per second, and transmits the frame images acquired at a frequency of 40 per second to the information processing device 60. The frame rate of the camera 20 is not limited to 40 fps, and may be set to, for example, 25 fps, 30 fps, 50 fps, 60 fps.

The light source 30 is an instrument or device that emits light that illuminates the subject H. The light source 30 is arranged inside the acoustic space 10 near the ceiling portion 11c at the central portion in plan view. The light source 30 is, for example, an LED spotlight. The spotlight is a light that intensively illuminates a part of the acoustic space 10. The light source 30 is installed so as to face downward, and emits the light L downward. As a result, the light L is projected onto a partial region R (see FIG. 3) of the upper surface 41 of the table 40. The light source 30 is provided integrally with the camera 20. The light source 30 is not limited to the above configuration, and may be, for example, an arc lamp, an incandescent light bulb, a fluorescent light, or sunlight, or a light source that uniformly illuminates a wide range including the periphery of the table 40. May be. In addition, the direction of the light L to be emitted, the installation location, the installation method, and the like of the light source 30 can be arbitrarily set. For example, the light source 30 may be installed so as to emit the light L in the upward or horizontal direction. It may be installed separately from the camera 20.

The table 40 is installed on the floor 11 a below the light source 30. The height of the table 40 is set to a height corresponding to, for example, the waist position of the user (player) U in a standing posture (see FIG. 3 ). The pedestal 40 is installed substantially at the center of the surface of the floor 11a, and has a rectangular parallelepiped shape with a height of 90 cm (cm) and a length and width of 45 cm. The upper surface 41 of the pedestal 40 is planar and is arranged so as to include the entire region R irradiated with the emitted light L of the light source 30. The platform 40 is not limited to the above configuration, and its shape, size, arrangement, etc. can be changed as appropriate. Further, in the acoustic space generation device 100, whether or not to install such a table 40 is arbitrary.

As described above, the light L is emitted downward from the light source 30. The light L emitted from the light source 30 illuminates, for example, a circular region R on the upper surface 41 of the table 40 when it advances downward as it is. At this time, the substantially conical space having the light source 30 as the apex and the upper surface 41 of the base 40 as the bottom surface is partially illuminated by the light L in the acoustic space 10 set to be dark as a whole. Such a substantially conical space illuminated by the light L of the light source 30 is referred to as a light irradiation space S (see FIG. 3).

Five speakers 50 are installed in the acoustic space 10. These five speakers 50 output sounds based on the music data generated by the information processing device 60. Each of the first speaker 51, the second speaker 52, the third speaker 53, the fourth speaker 54, and the fifth speaker 55 has sound emitting portions 51a, 52a, 55a that emit sound in a predetermined direction.

The fifth speaker 55 is disposed in the acoustic space 10 at the central portion and the upper side in a plan view. The fifth speaker 55 is provided on the upper surface of the camera 20, and is installed with the sound emitting part 55a facing upward.

On the other hand, the first speaker 51, the second speaker 52, the third speaker 53, and the fourth speaker 54 are arranged at the bottom in the acoustic space 10, and are equally spaced on the same circumference centered on the fifth speaker 55. It is arranged so that. The first to fourth speakers 51 to 54 are installed at the four corners of the substantially square floor 11a. In addition, each of the first to fourth speakers 51 to 54 is in a state in which the sound emitting portions 51a, 52a, 55a are directed toward the central side of the acoustic space 10 and in a horizontal direction so that sound is emitted slightly upward. On the other hand, it is installed in a state of being inclined upward by about 5° to 25°.

Note that the plurality of speakers 50 included in the acoustic space generation device 100 are not limited to the configurations of the first to fifth speakers 51 to 55 described above. That is, the number of the speakers 51 and the like installed in the acoustic space 10, the arrangement of the speakers 51 and the like, the orientation of the sound emitting portion 51a, and the like can be changed as appropriate. Specifically, for example, the number of speakers included in the acoustic space generation device 100 may be two or more and four or less, or six or more. Further, for example, all the speakers 51 to 55 configuring the five speakers 50 may be arranged at the upper part, the bottom part, or the central part of the acoustic space 10. May be. Further, for example, the fifth speaker 55 may be installed separately from the camera 20 with the sound emitting portion 55a facing downward so as to emit sound downward, and the first to fourth speakers 51, etc. may be installed. It may be installed so as to emit sound in the horizontal direction. Further, the first to fifth speakers 51 and the like are all speakers having the same configuration, but some or all of them may be speakers having different configurations.

The information processing device 60 is composed of, for example, a computer. The information processing device 60 is communicatively connected to each of the first to fifth speakers 51 and the like and the camera 20 via wired or wireless communication. The information processing device 60 acquires an image of the camera 20 and outputs a predetermined sound from the first to fifth speakers 51 and the like. Although the information processing device 60 is installed outside the acoustic space 10, it may be installed inside the acoustic space 10. FIG. 2 is a block diagram showing a main part of the acoustic space generation device 100. As shown in FIG. 2, the information processing device 60 has an input unit 61, a processing unit 62, and a storage unit 63.

The input unit 61 acquires the image of the camera 20. The input unit 61 acquires a plurality of frames captured by the camera 20 as image data. The processing unit 62 performs a predetermined process based on the input image data, selects a pitch according to the operation state of the subject H, and causes the speaker 50 to output each sound of the selected pitch. The data processing and the like in the processing unit 62 will be described later. In addition, the processing unit 62 is realized by a configuration including a CPU, for example. The storage unit 63 stores image data input from the camera 20, data generated by the processing unit 62, and the like. The storage unit 63 also stores a program for executing the processing of the processing unit 62, and a music structure program and a space structure program described later. The storage unit 63 is realized by, for example, a memory or a hard disk.

Next, a method of using the acoustic space generation device 100 will be described. FIG. 3 is a diagram illustrating an example of a usage state of the acoustic space generation device 100. As shown in FIG. 3, the user U stands on the back side of the table 40 and moves or stops the hand H with the hand H in the light irradiation space S. The acoustic space generation device 100 is used in this way. At this time, the user U puts both the left and right hands H or one hand H in the light irradiation space S, and moves the hand H, for example, in the up-down direction, the left-right direction, rotates, or moves the finger. , Exercises such as expanding or closing the palm of the hand H, or temporarily stopping the movement of the hand H. A series of such movements of the hand H is captured by the camera 20. Then, the sound generated according to the motion of the hand H is output from the speaker 50.

Next, the operation of the acoustic space generation device 100 will be described. FIG. 4 is a flowchart showing an example of the operation of the acoustic space generation device 100. The operation of the acoustic space generation device 100 will be described below with reference to the flowchart of FIG.

First, the camera 20 captures a moving image of the subject (object) H (step S01). Then, the image data captured by the camera 20 is transmitted to the information processing device 60 (step S02). For example, in the use state shown in FIG. 3, the camera 20 continuously photographs the hand H in the light irradiation space S at a preset frame rate, and inputs image data for each frame as an input unit of the information processing device 60. Enter in 61.

When the image data (frame image) for each frame is input to the input unit 61, the processing unit 62 executes a predetermined process and a predetermined operation based on the image data (step S03). Then, a musical sound or the like is output from the speaker 50 (step S04). The sound output from the speaker 50 has a melody or syllable composed of a sound having a predetermined pitch selected by the processing unit 62.

The sound output from the speaker 50 includes a sine wave (sine wave) sound and a reverberation sound, in addition to such a melody and syllable sound, by satisfying a predetermined condition. The conditions under which the sine wave sound is output from the speaker 50 and the conditions under which the reverberation sound is output will be described later.

By the operation of steps S01 to S04 of the acoustic space generation device 100, a musical sound that is automatically tuned according to the movement of the subject H is played from the speaker 50, and the acoustic space 10 is generated.

Next, the content of the processing of the processing unit 62 in step S03 will be specifically described. FIG. 5 is a flowchart showing the processing of the processing unit 62 when outputting a sound of a predetermined pitch. The processing of the processing unit 62 is automatically executed based on, for example, the program stored in the storage unit 63.

As shown in FIG. 5, the processing unit 62 first acquires a frame image (step S11). Next, the processing unit 62 compares the light amount of the image data of the frame with the light amount of the image data of the frame acquired immediately before the frame (step S12). In step S12, the light amounts of all the regions of the frame image are compared, but instead of this, the light amounts of some predetermined regions in the frame image may be compared.

Then, it is determined whether or not the difference in light amount is equal to or more than a threshold value (step S13). In step S13, the processing unit 62 determines whether the difference between the light amount of the image data of the frame and the light amount of the image data of the frame acquired immediately before the frame is greater than or equal to the threshold value or less than the threshold value. The threshold value is a value of a predetermined light amount, which is set in advance and stored in the storage unit 63.

When the above-described difference in light amount is equal to or more than the threshold value (in the case of “YES” in step S13), the processing unit 62 determines that the subject H is in a “motion” state (step S14). Then, when the processing unit 62 determines that the subject H is in a moving state, it generates one motion sensing signal (step S15).

On the other hand, when the processing unit 62 determines that the above-described difference in light amount is less than the threshold value (in the case of “NO” in step S13), it determines that the subject H is in the “still” (stop) state (step S24). When the subject H determines that the subject H is stationary, the processing unit 62 may generate one pause/reverberation signal (step S25).

When one motion detection signal is generated in step S15, the processing unit 62 selects one pitch of the output sound based on the preset and stored music structure program (step S16). In step S16, for example, the pitch of the sound to be output from the file having the data shown in FIG. 6 is selected based on the music structure program. In this case, one pitch is selected from the data of 42 pitches (A1 to A42) shown in FIG. The pitch selected in step S16 may be selected from the recording material such as acoustic noise, instead of the data of 42 pitches (A1 to A42) shown in FIG. In step S16, a plurality of pitches may be simultaneously selected from the 42 pitch data (A1 to A42) shown in FIG.

Subsequent to step S16, the processing section 62 assigns a speaker to the sound of the selected pitch by the preset and stored spatial structure program (step S17). In step S17, one of the five speakers 51 to 55 is assigned to each sound of the selected pitch based on the spatial structure program. When a plurality of pitches are simultaneously selected in step S16, any one of the five speakers 51 to 55 is assigned to each of the selected pitches in step S17.

FIG. 6 is a diagram showing a pitch audio file. In FIG. 6, data relating to, for example, 42 pitches is shown. In FIG. 6, A1 to A42 are serial numbers of 42 pitches. These 42 pitches are pitches selectable in step S16. The MIDI number is also called a note number or a note number, and is a numerical value indicating the pitch and range of a MIDI (Musical Instrument Digital Interface) tone. The polyphony number means the number of sounds that can be output simultaneously, and is also a value corresponding to the number of layers. Further, the appearance frequency of each pitch when the motion detection signal occurs 127 times indicates how many times each pitch is selected when the motion detection signal occurs 127 times. Further, the output frequency of SP1/SP2/SP3/SP4/SP5 when the motion detection signal is generated 20 times means that the motion detection signal is generated 20 times and the sound with a predetermined pitch is selected 20 times. How many times is assigned to each of the first speaker 51 (SP1), the second speaker 52 (SP2), the third speaker 53 (SP3), the fourth speaker 54 (SP4), and the fifth speaker 55 (SP5). Is shown.

Numerical values relating to the frequency of appearance of each pitch and the frequency of allocation of each speaker to the sound of each pitch are preset. In steps S16 and S17, the pitch is selected and the speaker 50 is automatically assigned according to the numerical values regarding the appearance frequency and the assignment frequency.

Returning to FIG. 5, the processing unit 62 causes the assigned speaker 50 to output a sound of a predetermined pitch (step S18). In step S18, a sound of the selected pitch is output from the assigned speaker 50. When the speaker 50 outputs a continuous sound, the speaker 51 or the like assigned to each sound forming the continuous sound is changed (for example, a speaker 51 different for each sound forming the continuous sound is used). For example, the sound may be output from the speaker 50 so as to move vertically and horizontally in the acoustic space 10.

FIG. 7 is a diagram for explaining an example of the operation of the processing unit 62. By the above-described processing of the processing unit 62, as shown in FIG. 7, when the subject H is in the moving state, the sound of the pitch selected based on the music structure program is output from the speaker 50. On the other hand, when the subject H is in a stationary state, the speaker 50 temporarily becomes a rest state in which no sound is output, and as a result, the acoustic space 10 is in a state of producing no sound or lingering sound. Therefore, when the subject H continuously moves, the sound of the predetermined pitch is continuously output, so that the sound as the texture is generated. On the other hand, when the subject H appropriately sandwiches the stationary state in the movement state, the sound expresses the movement and the lingering sound can be heard. As a result, the acoustic space 10 in which a melody (melody) or a syllable is played is generated.

Further, in the processing of step S03 described above, the processing unit 62 causes the speaker 50 to generate a sine wave under predetermined conditions. This sine wave is output in a state of being synthesized with the waveform of the sound of the above-mentioned predetermined pitch. Therefore, the processing of the processing unit 62 relating to the generation of such a sine wave will be described.

FIG. 8 is a flowchart showing a sine wave generation process in the processing unit 62. The processing unit 62 executes the following processing. As shown in FIG. 8, it is determined whether or not the motion sensing signal is continuously generated for the first predetermined number of times in the first unit time (step S31). That is, it is determined whether or not the motion state has occurred continuously for the first predetermined number of times in the first unit time. When it is determined that the occurrence has occurred (in the case of “YES” in step S31), it is determined whether the number of occurrences of the first predetermined number of times in step S31 has reached the second predetermined number of times (step S32). When it is determined that it has arrived (in the case of "YES" in step S32), it is determined whether or not a motion sensing signal has been generated after the stationary state for the second unit time or longer (step S33). Then, when it is determined to be in the moving state after the second unit time or more for the stationary state (in the case of “YES” in step S33), the plurality of speakers 50 are further operated for a predetermined sound. When the specific speaker 51 or the like is allocated (when “YES” in step S34), a sine wave is generated from the specific speaker 51 or the like (step S35). In the above-described sine wave generation process, the determination of the stationary state may be made by recognizing the non-occurrence of the motion sensing signal or by detecting the pause/reverberation signal.

Specifically, it is as follows, for example. FIG. 9 is a diagram for explaining an example of the operation of the processing unit 62. The first unit time is, for example, 180 ms, the first predetermined number of times is, for example, 1 to 30 times, the second unit time is, for example, 500 ms, and the second predetermined number of times is, for example, 10 times. It Note that these set values are stored in the storage unit 63. As shown in FIG. 9, the processing unit 62 generates a motion sensing signal continuously for 1 to 30 times in a period of 180 ms or less, reaches a total of 10 times, and is still for 500 ms or more. When the motion sensing signal is detected, for example, the fifth speaker 55 is assigned and the first sine wave is generated from the fifth speaker 55 for 60 to 90 seconds. In addition, the processing unit 62 determines that the sound of the pitch selected by the generation of the motion sensing signal in step S33 is the sound of any of the pitches A19 to A25 of the 42 pitch data of FIG. Generates a second sine wave in addition to the first sine wave. The second sine wave is a combination of a third sine wave and a fourth sine wave, which will be described later, by a volume curve of 10 seconds. Here, the third sine wave is a sine wave having a frequency two octaves above the pitch selected by the generation of the motion detection signal in step S33. The fourth sine wave is a sine wave obtained by adding a frequency of 0.5 to 11 Hz to the third sine wave. The second sine wave is output from the fifth speaker 55 for 10 seconds, for example. In the fourth sine wave, when the numerical value is changed, the reached value is reached by using the time complement of 1500 to 4000 ms.

Further, in the process of step S03 described above, the processing unit 62 causes the speaker 50 to output the first reverberation sound under a predetermined condition. Therefore, subsequently, the processing of the processing unit 62 regarding the output of the first reverberation sound will be described.

FIG. 10 is a flowchart showing the output processing of the first reverberation sound. The processing unit 62 executes the following processing. As shown in FIG. 10, it is determined whether or not the motion sensing signal is continuously generated for the first predetermined number of times in the first unit time (step S41). When it is determined that the occurrence has occurred (in the case of “YES” in step S31), it is determined whether the number of occurrences of the first predetermined number of times in step S41 has reached the third predetermined number of times (step S42). When it is determined that it has arrived (in the case of “YES” in step S42), it is subsequently determined whether or not a motion detection signal has been generated after being in the stationary state for the second unit time or longer (step S43). Then, when it is determined to be in the moving state after being in the stationary state for the second unit time or longer (in the case of “YES” in step S43), the plurality of speakers 50 are further added to the selected sound. When a specific speaker is assigned (when “YES” in step S44), the specific reverberant sound is output from the specific speaker 50 (step S45).

Specifically, it is as follows, for example. FIG. 11 is a diagram for explaining an example of the operation of the processing unit 62. The first unit time, the first predetermined number of times, and the second unit time are set to the above values in advance, for example. Further, the third predetermined number of times is set to, for example, 3 to 5 times. Note that these set values are stored in the storage unit 63. As shown in FIG. 11, the processing unit 62 counts a cycle in which a motion sensing signal is continuously generated 1 to 30 times in a period of 180 ms or less as 1, and when the count reaches 3 to 5, further, 500 ms or more. When the motion sensing signal is detected after the stationary state, for example, the fifth speaker 55 is assigned and the first reverberation sound is output from the fifth speaker 55 for 60 to 90 seconds. At this time, if the speaker assigned to the sound of the pitch selected based on the motion sensing signal after resting for 500 ms or more is the fifth speaker 55, the first reverberation sound is output from the fifth speaker 55 to the above. Synthesize to pitch sound and output.

Furthermore, in the process of step S03 described above, the processing unit 62 causes the speaker 50 to output the second reverberation sound under predetermined conditions. Therefore, subsequently, the processing of the processing unit 62 regarding the output of the second reverberation sound will be described.

FIG. 12 is a flowchart showing the output processing of the second reverberation sound. The processing unit 62 executes the following processing. As shown in FIG. 12, it is determined whether or not the motion sensing signal has been generated a fourth predetermined number of times in the first unit time (step S51). When it is determined that it has occurred (in the case of “YES” in step S51), when a specific speaker is assigned to the selected sound among the plurality of speakers 50 (in the case of “YES” in step S52) The specific reverberant sound is output from the specific speaker 50 (step S53).

Specifically, it is as follows, for example. FIG. 13 is a diagram for explaining an example of the operation of the processing unit 62. In advance, the first unit time is set to the above value, for example. Further, the fourth predetermined number of times is set to 30 times, for example. Note that these set values are stored in the storage unit 63. As shown in FIG. 13, the processing unit 62 causes the speaker 50 to output the second reverberation sound when the motion sensing signal is generated 30 times in succession in a period of 180 ms or less. The processing unit 62 synthesizes the second reverberation sound with, for example, the sounds of the pitches A1 to A18 and A70 to A79 of the 42 pitches of data shown in FIG. 6, and then outputs the sound for 3 to 10 seconds. Let In addition, the processing unit 62 outputs the second reverberation sound from the first speaker 51 and the second speaker 52, for example, and also outputs the second reverberation sound from the third speaker 53 and the fourth speaker 54 after acoustic movement for one second.

As described above, the processing unit 62 causes the speaker 50 to output a sound of a predetermined pitch through the processing of steps S11 to S18. In addition, the processing unit 62 causes the speaker 50 to generate a sine wave by the processing of steps S31 to S35. Further, the processing unit 62 causes the speaker 50 to output the first reverberation sound through the processes of steps S41 to S45. Furthermore, the processing unit 62 causes the speaker 50 to output the second reverberation sound through the processes of steps S51 to S53.

As described above, in the acoustic space generation device 100, the input unit 61 (stored by the motion sensing signal issued from the processing unit 62 that determines the repetition of “moving/still” (variation of motion/rest time) through sensor input). The acoustic space 10 is generated through the music structure program and the space structure program implemented in the unit 63. Then, the acoustic space generation device 100 determines each of the motion and still state based on the motion of the subject H. However, a sound is selected according to the frequency, cycle, number of times, etc. of each state, and the selected sound is output as a musical sound.Accordingly, the acoustic space generation device 100 can easily generate a musical sound according to the mode of movement of an object. It is possible to easily provide the acoustic space 10 in which such a musical sound is played.According to the acoustic space generation device 100, a new acoustic space 10 is generated based on the repetitive motion of the motion and the stillness of the object. It is possible to play syllable musical tones and sounds, and to create a unique acoustic sound 10 including a sound generated by sine wave synthesis, a first reverberation sound, and a second reverberation sound under predetermined conditions. Can be provided.

Further, in the acoustic space generation device 100, the moving subject H is irradiated with the light L, so that the user U plays improvised music while watching the movement of the subject H illuminated by the light L. It is possible to appreciate. As a result, the acoustic space generation device 100 can provide the acoustic space 10 with a new entertainment property in which the user U can enjoy improvised music along with visual changes. Then, according to the acoustic space generation device 100, since the user U feels or hears the sound that is generated from the state of motion and stillness and that is related to the body expression such as yoga or welfare, the user U feels or hears the sound. Thus, not only can the music be experienced or heard, but it can also contribute to the improvement of the spirit of the music.

Next, a modified example of the acoustic space generation device 100 according to the above embodiment will be described. The acoustic space generation device 100 has a configuration in which the camera 20 for photographing the subject and the light source 30 for illuminating the subject H are provided, but the state of the object can be detected in place of both the camera 20 and the light source 30. It may be configured to include various sensors. Therefore, the configuration of the acoustic space generation device according to the modification will be specifically described below.

In the configuration of the acoustic space generation device according to the above modification, as the sensor capable of detecting the state of the object, for example, a distance sensor capable of detecting the distance to the object can be applied. The camera 20 is also one of the sensors capable of detecting the movement of the object.

The operations of the input unit and the processing unit of the acoustic space generation device according to such a modified example are the same as the operations of the input unit 61 and the processing unit 62 described above. However, the input unit according to the modification continuously acquires, for example, a signal corresponding to the distance from the sensor to the object from the sensor. Further, the processing unit of the modified example compares the acquired signal with the signal acquired immediately before, and if the difference is equal to or more than a threshold value, determines that the object is in a motion state. The threshold is a value related to a signal and is set in advance. It should be noted that the subsequent processing of the processing unit is similar to that of the processing unit 62 described above. That is, in the modified example, a sensor capable of detecting the state of the object, an input unit that acquires a signal from the sensor, a processing unit that selects a sound to be generated according to the signal, and a sound selected by the processing unit. A plurality of speakers 50 for outputting are provided, and the processing unit compares the signal with a signal immediately before the signal, and based on the threshold value of the signal set in advance, the object is in a moving state or a stationary state. If it is determined that it is in a state of motion, a motion detection signal is generated, a pitch to be output is selected by a music structure program stored in advance, and a spatial structure program stored in advance is selected. One of the plurality of speakers 50 is assigned to each sound of the selected pitch, and a sound is output from the assigned speaker. Other configurations of the acoustic space generation device according to the modification are the same as those of the acoustic space generation device 100 according to the above-described embodiment. The processing unit of the acoustic space generation device according to the modification includes, in addition to the sound of the above-mentioned pitch, a sine wave, a first reverberant sound, and a second reverberant sound according to the frequency, the cycle, the number of times, etc. of each state. Output from 50.

According to such a modification, the acoustic space generation device can be realized with an even simpler configuration. Therefore, for example, it is possible to configure the acoustic space generation device with a small PC with a video camera function. As a result, it becomes easy to apply the acoustic space generation device to music teaching materials for children.

Further, the acoustic space generation device 100 may be configured as below in addition to the above modification. Specifically, for example, the processing unit determines in advance whether the state where the sensor-mounted square is stepped on and the state where the square is removed from the square, or whether the buzzer switch is ON/OFF is determined in advance. Based on the threshold value of the set signal, it may be determined whether the object is moving or stationary. That is, for example, by using a sensor capable of detecting contact with the foot, different signals (for example, 1 when the foot is in contact and 1 when the foot is in contact When the buzzer switch is ON and OFF, a different signal (for example, ON) is output from the sensor to the processing unit or a sensor capable of detecting the ON state of the buzzer is used. When it is OFF, when it is OFF, 0) is output from the sensor to the processing unit. Then, the processing unit compares the signal acquired from the sensor with the signal acquired immediately before, and determines whether the difference is a threshold value or more (for example, 1 or more). With this, the processing unit determines whether the object is in a motion state (a state in which there is a change in motion) or a stationary state (a state in which there is no change in motion) (for example, when the difference is 1, the motion is performed). State, and if the difference is 0, it is determined to be a stationary state). When the processing unit determines that it is in a motion state, it generates a motion detection signal, selects a pitch to be output by a pre-stored music structure program, and selects it by a pre-stored spatial structure program. One of a plurality of speakers is assigned to each sound of the selected pitch, and the sound is output from the assigned speaker. In addition to the sound of the above pitch, a sine wave, a first reverberant sound, and a second reverberant sound are output from the plurality of speakers 50 according to the frequency, cycle, number of times, etc. of each state.

Further, in the acoustic space generation device 100, the processing unit, in the input voice information (for example, the voice input to the microphone), based on the threshold value of the signal set in advance, the attack (that is, the sound is determined to be strong and strong). It may be determined whether or not the object is in a moving state or a stationary state by determining whether or not the object is coming out, the sound is started, and the adult is rising. That is, for example, the acoustic space generation device 100 includes a microphone that is a voice recognition device for picking up a voice of an object, and the processing unit compares the voice signal acquired from the microphone that is the sensor with the voice signal acquired immediately before. Alternatively, it may be determined whether or not the difference (for example, the difference in sound volume) is equal to or greater than a preset threshold value, and whether the object is in a moving state or a stationary state may be determined.

Although the embodiments and modifications of the present invention have been described above, the technical scope of the present invention is not limited to the above-described embodiments. For example, the object whose movement is detected by a sensor such as a camera is not limited to the hand H, and may be, for example, the foot or the whole body of a person, or an object (for example, a stick) held by a person. It may be a shadow of a leaf or a wave. Further, the plurality of speakers 50 included in the acoustic space generation device 100 may be composed of a plurality of stationary speakers 51 and the like as in the embodiment, or a pair of speakers that directly output sound to both ears of the user U. It may be headphones or binaural earphones equipped with a speaker. Further, although the acoustic space 10 generated by the acoustic space generation device 100 is a real space, it may be a virtual space. That is, for example, when headphones or binaural earphones are applied as the speaker 50 of the acoustic space generation device 100, the acoustic space generation device 100 provides a virtual acoustic space to the user U who wears the speaker 50. To do. Further, in the above-described embodiment and modified example, processing such as determining a motion state (motion) and a stationary state (stop) with respect to the state of the object using a sensor capable of detecting the movement of the object (subject) is performed. Then, a sound corresponding to the result is generated, but the movement of the object (subject) is not limited to the movement of the actual object here. In other words, the acoustic space generation device of the present invention is, for example, a motion state of a motion of an object at a specific fictitious place in virtual reality (VR) or in a game (on an OpenGL (registered trademark) drawing, etc.). It is also possible to perform a process such as distinguishing between the stationary state and the stationary state (stop) to generate a sound according to the process. Further, the acoustic space generation device 100 stores a plurality of music structure programs having algorithms in which different conditions (thresholds, etc.) are set in the storage unit 63, and the process of step S16 is performed by the plurality of music structure programs. It is also possible to appropriately select from among these to execute and to execute a plurality of music structure programs at the same time. In this case, the plurality of music structure programs are stored in the storage unit 63, for example, in the state of being divided into different layers. As a result, the music structure program can be appropriately selected according to the usage environment of the acoustic space generation device 100 and the number of mounting locations, and the music structure program can be easily improved or added.

In addition, the acoustic space generation device of the present invention may be, for example, a system configuration in which a large number of speakers 51 and the like are randomly installed on a tree branch of a natural environment, a plane (for example, a painting, a book, a wall surface (for example, vertical A square-shaped wall with a horizontal length of 10 cm or 5 m, etc.)) and a product configuration with a plurality of small speakers 51 etc. interspersed with sensors, and the sensors and speakers 51 etc. are mounted on multiple streetlights that are also light sources 30. It may be realized by the configuration of the environment system described above. Moreover, the acoustic space generation device of the present invention may constitute a composite improvisational ensemble device/musical instrument. The acoustic space generation device in this case, for example, changes/assigns to 42 other numbers (A1 to A42) to which each pitch file shown in FIG. 6 is assigned, or is mounted in the storage unit 63. It may be configured to perform processing such as associating the operation of another created music structure program (layer). That is, for example, when the pitch number "A2" is selected in step S16, the noise recording material and the sound collection/output from the microphone are flowed in real time in the acoustic space, and then the pitch number "A2" or the like. When the number is selected once or a plurality of times, the sound collection output of the microphone may be turned off, and at the same time, another music structure program/layer may be operated. Note that the acoustic space generation device 100 of the above embodiment includes a plurality of speakers 50, but the number of speakers 51 and the like may be one.

H subject, hand (object)
20 camera (sensor)
30 light sources
50, 51, 52, 53, 54, 55 speaker 61 input unit 62 processing unit 100 acoustic space generation device

Claims (2)

A camera that shoots objects
A light source for illuminating the object,
An input unit that acquires an image from the camera,
A processing unit that selects a sound to be generated according to the frame of the image ,
A plurality of speakers that output the sound selected by the processing unit,
The processing unit is
Comparing the light quantity of the frame and the light quantity of the frame immediately before the frame, based on a preset threshold of the light quantity, it is determined whether the object is in a moving state or a stationary state,
When it is determined to be in a state of motion, a motion detection signal is generated, a pitch to be output is selected by a pre-stored music structure program, and each tone of the selected pitch is selected by a pre-stored spatial structure program. In contrast, one of the plurality of speakers is assigned, and the sound is output from the assigned speaker,
After the state of the motion in the first time unit first generates a predetermined number of times continuously, emitting raw count reaches a second predetermined number of times, was in a state of stationary second unit hours or more, the motion When it is determined that, and for a predetermined sound, a specific speaker of the plurality of speakers is assigned, by the music structure program, a sine wave is generated from the specific speaker,
The first of the movement per unit time state occurs in the first predetermined consecutive number of times, when the emitting raw count has reached the third predetermined number, a state of the stationary second unit hours or more After that, when it is determined that the sound is in motion, and the specific speaker is assigned to the selected sound, the music structure program causes the first reverberant sound from the specific speaker. To output
When the state of motion occurs in the first unit time in succession for the fourth predetermined number of times together with the first predetermined number of times to the third predetermined number of times, and the selected sound is generated. On the other hand, when the specific speaker is assigned , the acoustic space generation device that causes the specific reverberation sound to be output from the specific speaker in a predetermined order by the music structure program. A sensor that can detect the state of the object,
An input unit for obtaining a signal from the sensor,
A processing unit that selects a sound to be generated according to the signal,
A plurality of speakers that output the sound selected by the processing unit,
The processing unit is
By comparing the signal and the signal immediately before the signal, based on a threshold value of the signal set in advance, it is determined whether the object is in a moving state or a stationary state,
When it is determined to be in the state of movement, a motion sensing signal is generated, and the pitch to be output is selected by the pre-stored music structure program, and each note of the selected pitch is selected by the pre-stored spatial structure program. In contrast, one of the plurality of speakers is assigned, and the sound is output from the assigned speaker,
After the state of the motion in the first time unit first generates a predetermined number of times continuously, emitting raw count reaches a second predetermined number of times, was in a state of stationary second unit hours or more, the motion When it is determined that, and for a predetermined sound, a specific speaker of the plurality of speakers is assigned, by the music structure program, a sine wave is generated from the specific speaker,
The first of the movement per unit time state occurs in the first predetermined consecutive number of times, when the emitting raw count has reached the third predetermined number, a state of the stationary second unit hours or more After that, when it is determined that the sound is in motion, and the specific speaker is assigned to the selected sound, the music structure program causes the first reverberant sound from the specific speaker. To output
When the state of motion occurs in the first unit time in succession for the fourth predetermined number of times together with the first predetermined number of times to the third predetermined number of times, and the selected sound is generated. On the other hand, when the specific speaker is assigned , the acoustic space generation device that causes the specific reverberation sound to be output from the specific speaker in a predetermined order by the music structure program.

Images