JP2014228750A - Performance recording system, performance recording method and instrument - Google Patents

Abstract

PROBLEM TO BE SOLVED: To start recording performance automatically.SOLUTION: An instrument 100 includes: an operation section 110 for accepting performance operation; a generation section 120 for generating performance information indicating the accepted performance operation; a detection section 130 for detecting a portable terminal; and a control section 170 for starting processing, when the portable terminal is detected, for recording at least one of the performance information generated by the generation section and a video including a scene imaged in which the performance operation is performed, on at least one of the instrument and the portable terminal.

Description

  The present invention relates to performance recording.

  There is an electronic musical instrument having a function of storing the contents of a performance performed by a performer as data and reproducing the contents of the performance later (see Patent Document 1).

JP 2011-81364 A

Conventionally, in order to start recording a performance, it is necessary to perform a predetermined operation prior to the performance. For example, when an electronic musical instrument having a performance recording function is used, the predetermined operation is an operation on a remote control terminal attached to the musical instrument or a switch provided on the electric musical instrument. Or it is operation for starting recording with respect to the recording device, after connecting a musical instrument and a recording device.
If there is no assistant for recording the performance, the player's confidence needs to perform the above operation. However, since the performer before the performance is usually in a tension state, it tends to forget such an operation and start the performance. In addition, forcing a player to perform such an operation before playing may feel a psychological burden depending on the performer.

  An object of the present invention is to automatically start recording performances.

In one aspect, the present invention detects an operation unit that receives a performance operation, a generation unit that generates performance information representing the received performance operation, a detection unit that detects a portable terminal, and the portable terminal. And a process of recording at least one of the performance information generated by the generation unit and a video of a scene where the performance operation is performed on at least one of the instrument and the portable terminal. A musical instrument is provided having a controller for starting.
In a preferred aspect, the control unit stops the processing when the mobile terminal is no longer detected by the detection unit.
In a preferred aspect, when the portable terminal is detected, the control unit performs performance information generated by the generation unit, sound information collected by the sound collection unit, and the performance operation. A process of recording at least one of the images of the scenes recorded on at least one of the musical instrument and the portable terminal is started.
In a preferred aspect, the musical instrument further includes an interface for communicating with the detected portable terminal, and outputs an instruction to start photographing the video by the imaging unit to the detected portable terminal via the interface. To do.
In a preferred aspect, the musical instrument further includes a storage unit and an interface for communicating with the detected portable terminal, and outputs the performance information recorded in the storage unit to the portable terminal after the performance ends.
In another aspect of the present invention, in a performance recording system having a musical instrument and a portable terminal, a step of receiving a performance operation with the musical instrument and a step of generating performance information representing the received performance operation with the musical instrument A step in which the musical instrument detects a portable terminal; and a scene in which the generated performance information and the performance operation are performed in at least one of the musical instrument and the portable terminal when the portable terminal is detected. And a step of starting a process of recording at least one of the images obtained by shooting the performance.
In still another aspect of the present invention, in a performance recording system having a musical instrument and a portable terminal, a storage unit, a reception unit that receives a performance operation, and a generation unit that generates performance information representing the received performance operation. An imaging unit that captures the scene where the accepted performance operation is performed, a detection unit that detects that the portable terminal and the instrument are in a predetermined positional relationship, and the portable terminal is detected. There is provided a performance recording system comprising: means for starting a process of recording at least one of performance information generated by the generation means and video captured by the imaging means in the storage means. .

  According to the present invention, performance recording is automatically started.

1 is a conceptual diagram of a performance recording system 1. FIG. 1 is an external view of a musical instrument 100. FIG. 1 is a configuration diagram of a musical instrument 100. FIG. 2 is a functional configuration diagram of a mobile terminal 200. FIG. It is a figure showing the structural example of the data D1. 3 is a functional configuration diagram of a server 300. FIG. It is a figure showing the structural example of the data D2. 3 is an operation example of the performance recording system 1. It is a block diagram of the musical instrument 100A. 10 is another operation example of the performance recording system 1. It is an example of the image displayed on the screen of the portable terminal 200.

  FIG. 1 is a diagram schematically showing a performance recording system 1 according to an embodiment of the present invention. The performance recording system 1 includes a musical instrument 100, a portable terminal 200, a server 300, a communication network 410, and a relay device 420. The musical instrument 100 is a musical instrument played by a user. In this embodiment, the musical instrument 100 is a keyboard instrument such as a piano or an electronic piano having at least a function of generating performance information performed by a player. In the following description, a general piano that emits sound in response to a performance operation will be described. However, in the musical instrument according to the present invention, a sound emitting function is not essential, and a sound emitting function such as a silent piano (registered trademark) is provided. An instrument that does not have The external form of the musical instrument is not limited to a keyboard musical instrument, and may be a wind instrument, a percussion instrument, or the like.

  The mobile terminal 200 is a portable information communication terminal such as a mobile phone, a smartphone, or a tablet PC. The portable terminal 200 has a function of exchanging information with the server 100 by communicating with the relay device 420 connected to the communication network 410 by radio and exchanging information with the musical instrument 100. It has the function to do. The server 300 has a function of storing data received from the mobile terminal 200 via the relay device 420 and the communication network 410 and transmitting the stored data in response to a request from the mobile terminal 200. The performance recording system 1 may include a plurality of musical instruments 100 and a plurality of portable terminals 200. In this case, one musical instrument 100 can be connected to a plurality of portable terminals 200, and one portable terminal 200 can be connected to a plurality of communications. For convenience of explanation, FIG. 1 shows only one musical instrument 100 and one portable terminal 200.

  In one aspect of the present invention, the server 300, the communication network 410, and the relay device 420 may be omitted from the performance recording system 1. That is, the performance recording system according to the present invention need not have a function of communicating with the server 300. In this case, the data stored in the portable terminal 200 regarding the performance is used (reproduced, edited, etc.) in the portable terminal 200.

  FIG. 2 shows the appearance of the musical instrument 100. The musical instrument 100 includes the same mechanism as that of a general acoustic piano, such as an action mechanism that strikes a string according to the movement of a key on the keyboard and a damper that suppresses vibration of the string according to the movement of the pedal. Yes. When the musical instrument 100 has an automatic performance function, the musical instrument 100 includes a sensor for detecting key movement (none of which is omitted in FIG. 1), musical tone data from an external audio device, or performance information stored in the musical instrument 100. A communication interface such as a connection terminal for outputting to the external device, and a user interface such as a touch panel for inputting information on settings and confirming information and informing the performer.

  The musical instrument 100 is provided with a pedestal 132 for installing the portable terminal 200. A terminal for connecting to the portable terminal 200 is provided on the upper surface of the pedestal 132. When the user installs the portable terminal 200 on the pedestal 132, the musical instrument 100 is fixed and information can be exchanged between the musical instrument 100 and the portable terminal 200 via the terminal. Note that. The pedestal 132 may be formed of a deformable material such as a flexible tube. In this case, the fixed position of the portable terminal 200 with respect to the musical instrument 100 can be changed according to the user's preference. In addition, not only the position but also the orientation of the mobile terminal 200 can be adjusted. Further, the position where the pedestal is provided is not limited to the position near the left edge of the music stand of the musical instrument 100 as shown in FIG. However, it is preferable that the position is convenient for shooting a performance scene and is within the reach of the user so as not to interfere with the performance.

  FIG. 3 is a functional configuration diagram of the musical instrument 100. The musical instrument 100 includes an operation unit 110, a drive mechanism 121, a generation unit 120, a detection unit 130, a storage unit 140, a sound generation unit 160, and a control unit 170. The operation unit 110 is an input mechanism including a keyboard 111, a pedal 112, and the like, and accepts a performance operation by a performer. The generation unit 120 includes a sensor 122 and a signal processing unit 123, and generates performance information representing the performance operation accepted by the operation unit 110. The drive mechanism 121 is an action mechanism that strikes a string in accordance with the movement of a key on the keyboard or a damper mechanism that stops the vibration of the string, and transmits mechanical information reflecting the performance operation to the sound generator 160. In addition, the musical instrument 100 includes a mechanism (not shown) such as an actuator that drives the operation unit 110, and has a function of realizing automatic performance by operating the operation unit 110 based on a control signal from the control unit 170. You may have. That is, the musical instrument according to the present invention may or may not have an automatic performance function.

  The sensor 122 is provided for each key / hammer of the keyboard or for each string, detects at least one of the movement of the keyboard and the vibration of the string, and outputs the detected information to the signal processing unit 123. The signal processing unit 123 is realized by a signal processing processor, and generates performance information corresponding to the operation content of the operation unit 110 based on the information supplied from the sensor 122. The performance information has, for example, a MIDI (Musical Instrument Digital Interface) format. In this case, the performance information includes information (note on / off) indicating the timing at which the operation was performed, the pitch (note number) of the sound, volume and strength (velocity), and the like. In short, the performance information is information reflecting the performance operation (action) of the performer, and may be any information that can reproduce the contents of the performance based on the performance information. The generated performance information is output to the control unit 170. The signal processing unit 123 performs signal processing such as processing / editing of required performance information, changing the data format, and the like.

  The sound generating unit 160 emits a sound corresponding to the performance operation received by the operation unit 110. Specifically, the sound generation unit 160 includes a string 161 and a soundboard 162, and generates and generates an acoustic wave based on the mechanical force of the operation unit 110 transmitted through the drive mechanism 121. An acoustic wave is transmitted to a space outside the instrument 100. The detection unit 130 includes a terminal 131, a pedestal 132 (see FIG. 2), and a signal circuit (not shown), and functions as a cradle for installing the mobile terminal 200. In addition, the detection unit 130 detects the mobile terminal 200. Specifically, the mechanical contact between the terminal on the mobile terminal 200 side and the terminal 131 is detected based on the change in the electrical signal. The detection result is output to the control unit 170. The detection unit 130 outputs the information supplied from the mobile terminal 200 via the terminal 131 to the control unit 170 under the control of the control unit 170, and supplies the information supplied from the control unit 170 to the mobile terminal 200 side. To do.

  The storage unit 140 is realized by a semiconductor memory such as RAM or ROM, a magnetic storage medium such as a hard disk, or an optical storage medium such as a DVD, and can be obtained by editing performance information and performance information generated by the signal processing unit 123. Performance data is stored. In addition, a terminal ID for identifying the mobile terminal 200 in which information necessary for exchanging information with the mobile terminal 200 is stored is stored. Furthermore, a device ID for identifying the musical instrument 100, which is information provided to the portable terminal 200 and accompanying the performance, is stored. Further, a program executed by the control unit 170 to control the musical instrument 100 is stored. This program may be stored in the storage unit 140 in advance, or may be stored afterwards by being stored in a storage medium or downloaded from a communication network.

  The control unit 170 is realized by a processor such as a CPU, and controls the operation unit 110, the generation unit 120, and the detection unit 130 using information acquired from the storage unit 140 and the detection unit 130. Specifically, when the mobile terminal 200 is detected, at least one of the performance information generated by the generation unit and the video of the scene where the performance operation is performed is displayed on the instrument and the mobile phone. The process of recording in at least one of the terminals is started. In a preferred embodiment, when the mobile terminal 200 is detected, the control unit 170 performs performance information generated by the generation unit 120, sound information collected by the sound collection unit, and the performance operation. A process of recording at least one of the images of the scenes recorded on at least one of the musical instrument and the portable terminal is started. In the present embodiment, the imaging function and the recording function provided in the portable terminal 200 are used for video and sound collection.

This recording process may be executed immediately after the mobile terminal 200 is detected, or may be executed at a timing depending on the recording preparation status, such as receiving a recording start notification from the mobile terminal 200. . In any case, the performer of the musical instrument 100 does not need to perform an operation for starting performance recording other than installing the portable terminal 200 at a predetermined position.
In addition, the control unit 170 determines the end timing of performance recording. Specifically, when the mobile terminal 200 is no longer detected by the detection unit 130, the recording process ends. Alternatively, when an instruction to stop recording is input from the input unit 202 to the portable terminal 200 by a performer or another person, a signal requesting to stop recording is output from the portable terminal 200 to the musical instrument 100, and the musical instrument 100 outputs the signal. When received, the recording process may be terminated. Furthermore, the control unit 170 instructs the mobile terminal 200 to start shooting on the mobile terminal 200. In addition to an instruction to start imaging, an instruction to end imaging may be transmitted from the musical instrument 100 to the portable terminal 200.

  When recording is performed by the instrument 100, performance information reflecting the operation is stored in the storage unit 140 every time a performance operation is detected. In a preferred embodiment, the performance information may be finally stored in the mobile terminal 200 as performance data. For example, the generated performance information is extracted from the storage unit 140 one by one or at a predetermined timing and transferred to the mobile terminal 200 via the detection unit 130. The predetermined timing may be determined based on a transfer rate between the musical instrument 100 and the portable terminal 200, for example. Alternatively, the performance information accumulated in the storage unit 140 after the end of a series of performances (a performance of one song) may be output to the mobile terminal 200 collectively.

  FIG. 4 is a functional configuration diagram of the mobile terminal 200. The mobile terminal 200 includes a display unit 201, an input unit 202, a sensor 203, a recording unit 204, an imaging unit 205, a second communication unit 206, a sound emitting unit 207, a first communication unit 208, a storage unit 209, and a control unit 210. . The display unit 201 is a display device such as a liquid crystal display, and displays information necessary for the user when recording and capturing performances and storing information in the server 300. The input unit 202 is realized by a numeric keypad, a touch panel, or the like, and accepts information input from a user when recording or capturing performances or storing information in the server 300. The sensor 203 includes a sensor such as an acceleration sensor, a gyro sensor, and an illuminance sensor, and a processing circuit that analyzes the sensor result, and detects and determines a shooting environment.

The recording unit 204 includes a microphone and a sound processing circuit, and collects sound. Note that the microphone provided in the recording unit 204 may be used, or may be provided in the musical instrument 100 although not shown in FIG. Alternatively, the musical instrument 100 may be introduced into the performance recording system 1 as an independent device from the portable terminal 200. The imaging unit 205 includes a lens, an image sensor, and a video signal processing circuit, and performs imaging. In the following, “shooting” and “recording” mean that both acquisition of video data by the imaging unit 205 and sound collection by the recording unit 204 are performed at the same time unless otherwise specified. In other words, when the subject to be imaged by the imaging unit 205 is a performer, an image of the performer (and piano image, surrounding landscape, etc.) is acquired, and the performance sound of the musical instrument 100 (and the performer at the same time as the performer). When singing, singing sounds and environmental sounds (other noises) are collected.
The second communication unit 206 is realized by an antenna and a radio signal processing circuit, and performs voice communication and data communication with the mobile terminal 200 of the other party via the relay device 420 according to a wireless communication standard such as 3G, LTE, or IEEE, and a communication network. 410 and server 300 are connected, and data is transmitted to and received from server 300. The sound emitting unit 207 is a speaker, which is used for voice communication and notifies the user of various information by voice.

  The first communication unit 208 includes a terminal 2081 for connecting to the detection unit 130 of the musical instrument 100 and detects the connection with the musical instrument 100. A terminal ID for identifying the portable terminal 200 is provided to the musical instrument 100 via the terminal 2081, and information such as a device ID of the musical instrument 100 and musical tone data is provided from the musical instrument 100 to the portable terminal 200. The storage unit 209 includes a user ID for identifying the user, a terminal ID, performance information received from the musical instrument 100, performance information edited from the performance information, and accompanying information related to the performance (device ID, date and time information, etc.) ) Is memorized. In addition, the storage unit 209 stores a basic program and an application program that are executed by the control unit 210 and control the control unit 210. These programs may be stored in a storage medium or downloaded from a server device via a communication network.

  The control unit 210 includes a processor such as a CPU, and includes a display unit 201, an input unit 202, a sensor 203, a recording unit 204, an imaging unit 205, a second communication unit 206, a sound emitting unit 207, a first communication unit 208, and a storage unit. 209 is controlled. Specifically, in addition to the function of a general information communication terminal, the control unit 210 executes a predetermined application to recognize the connected musical instrument 100 via the terminal 2081, and transmits information to the musical instrument 100. , A function for receiving a request from the musical instrument 100, and a function for controlling the imaging unit 205, the sensor 203, the display unit 201, the second communication unit 206, and the like according to the received request.

  FIG. 5 shows an example of the structure of the data D1 stored in the storage unit 209. D1 stores a file ID for identifying each performance, the date and time when the performance was performed, the device ID of the musical instrument on which the performance was performed, and performance information (MIDI data) in association with each other. The control unit 210 adds a user ID to the read D1 information and transmits it to the server 300 automatically or when an instruction from the user is given.

  When the server 300 is omitted from the configuration of the performance recording system 1, the storage unit 209 has the function of the server 300. In this case, it is not necessary to access the server 300 using the second communication unit 206 in order to reproduce or process data related to the performance. In addition, since the recorded information is only about the user and there is no need to distinguish other people's information from their own information, there is no possibility that the data structure will be complicated for identifying and authenticating the data creator. .

  FIG. 6 is a functional configuration diagram of the server 300. The server 300 includes a control unit 301, an input / output unit 302, a storage unit 303, and a communication unit 304. The control unit 301 is a processor such as a CPU, and controls the input / output unit 302, the storage unit 303, and the communication unit 304. Specifically, in response to a request from the mobile terminal 200, the performance data and video data are received from the mobile terminal 200 and stored in the database DB. In response to a request from the mobile terminal 200, performance data recorded for a performance performed by the user of the mobile terminal 200 is provided, and a file designated by the mobile terminal 200 is transmitted to the mobile terminal 200. At this time, the server 300 may perform billing processing. A communication unit 304 is a communication interface, and exchanges data with the mobile terminal 200 via the communication network 410. In data exchange, user authentication is always performed based on the user ID. The input / output unit 302 is an input device such as a keyboard and a mouse and a display device. The storage unit 303 is a storage device such as a hard disk, and stores a database DB stored from the portable terminal 200 in association with performance data and recorded data.

  7A shows an example of the structure of a data record D2 stored in the database DB. (B) and (c) show configuration examples of the databases DB1 and DB2 constructed for different users. In this way, a database unique to the user is constructed. When accessing the database DB, the control unit 301 performs an authentication process using a user ID, thereby permitting access to only the database corresponding to the user.

  FIG. 8 shows an operation example of the performance recording system 1. First, the user carrying the mobile terminal 200 makes advance preparations (S100). Specifically, an application for recording in cooperation with the musical instrument 100 is activated. Further, the device ID of the portable terminal 200 is registered in the musical instrument 100 in advance. The user sits in front of the musical instrument 100 and installs the musical instrument 100 in the detection unit 130. Then, the musical instrument 100 detects the portable terminal 200 (S102). Specifically, the following processing is performed (not shown). At the same time, the portable terminal 200 also recognizes the connection with the musical instrument and outputs the terminal ID to the musical instrument 100. When receiving the terminal ID from the portable terminal 200, the musical instrument 100 determines whether the ID is registered in the storage unit 140. If it is not registered, the process is stopped. Alternatively, a process for prompting new registration may be executed by outputting a signal requesting new registration to the mobile terminal 200 and displaying a predetermined registration guidance message on the screen of the mobile terminal 200.

  When the registration fact is confirmed, the musical instrument 100 reads the device ID of the musical instrument 100 from the storage unit 140. The read device ID is included in the recording request Req1 and transmitted to the portable terminal 200 (S104). The mobile terminal 200 that has received the recording request Req1 displays a predetermined message on the screen or makes an audio announcement to notify the performer that recording related to the performance is started (S106). Subsequently, the mobile terminal 200 activates the imaging unit 205 to start recording (S108). The portable terminal 200 transmits a notification Res1 that recording has been started to the musical instrument 100 (S110). At this time, if the musical instrument 100 has a function of notifying the user of a message by video or audio, the recording is started and the performance recording is ready by flashing the light emitting means in a predetermined pattern. This may be notified to the user to prompt the start of performance. Thereafter, when the performance operation is detected by the sensor 122, the musical instrument 100 determines that the performance has started, and continues to detect the performance information corresponding to all performance operations detected thereafter. During this time, it is recorded in the storage unit 140 (S114). During this time, a message such as “Performance recording is in progress. Please do not remove the terminal.” May be displayed on the portable terminal 200 to prompt the terminal to maintain the connection. The recorded emotional performance information is stored in the storage unit 140. Instead of being stored in the storage unit 140 or in addition to the storage unit 140, the acquired performance information may be sequentially output to the mobile terminal 200 and stored in the mobile terminal 200.

The user interrupts or ends the performance, and removes the mobile terminal 200 from the detection unit 130. Then, the musical instrument 100 recognizes that the portable terminal 200 is no longer detected (S115A; NO), and the musical instrument 100 ends the performance recording (S116). On the other hand, when the portable terminal 200 detects that the connection with the musical instrument 100 is interrupted (S115B; NO), the portable terminal 200 ends the recording (S122).
When the performance information is sequentially transferred, the musical instrument 100 connects the performance information recorded from the start to the end of the performance, and generates one MIDI data representing a series of performances (S120). The generated MIDI data is transmitted to the portable terminal 200 together with the date / time information and the musical instrument ID (S120). At this time, since the connection between the musical instrument 100 and the portable terminal 200 is interrupted, data cannot be transferred as it is. Therefore, for example, when the portable terminal 200 determines that the connection has been disconnected before receiving the MIDI data, the portable terminal 200 prompts the user to connect to the musical instrument 100 again by displaying a predetermined message on the screen of 200. When the connection is made again, the MIDI data is transferred.
On the other hand, when the performance information is sequentially transferred, the mobile terminal may generate one MIDI data by connecting the performance information received at that time without prompting reconnection (S120). That is, the generation of MIDI data is performed by the portable terminal 200 instead of the musical instrument 100.
When the mobile terminal 200 receives these data, the mobile terminal 200 associates the data with the captured video and audio data,
The performance data related to one performance is stored in the storage unit 209 (S126). Thereafter, when instructed by the user, the portable terminal 200 reads the performance data from the storage unit 209 and uploads it to the server 300 as necessary together with the user ID and the shooting data (S128).

  Note that the performance interruption or termination determination may be performed by inputting a recording end instruction to the mobile terminal 200 in addition to an action of the user removing the mobile terminal 200 from the detection unit 130. Alternatively, when the instrument 100 is provided with information input / output means, it is determined that the performance has ended when there is an input indicating that the performance has ended, or when a performance operation has not been detected for a predetermined time. Good.

  According to the present embodiment, the recording of performance is automatically started only by the user performing a simple act of installing the mobile terminal 200 on which a predetermined application is executed at a predetermined position. That is, since it is not necessary to perform a switch operation for recording, the psychological burden on the performer is small. In addition, recording on the mobile terminal 200 is automatically started without the performer operating the mobile terminal 200. Therefore, it is freed from the trouble of preparing in advance such as connecting the recording apparatus and the musical instrument 100 in advance.

The above-described embodiment may be modified as follows. The following modifications may be combined with each other.
Moreover, the detection of the portable terminal 200 by the musical instrument 100 may be performed without contact. Hereinafter, an example in which the electronic piano and the mobile terminal perform wireless communication will be described. Needless to say, the electronic piano is an example of a musical instrument that performs, and may be an acoustic piano or other musical instruments.

  FIG. 9 shows a configuration of a musical instrument 100A having a function of a general electronic piano. The musical instrument 100A omits the drive mechanism 121, and includes a point using the detection unit 130A and the sound generation unit 160A in place of the sound generation unit 160 and the detection unit 130, and an input / output unit 150 and a sound collection unit 155. In this respect, the functional configuration of the musical instrument 100 is different. The sound generator 160A includes a speaker 163 and an amplifier 164. The speaker 163 emits sound based on the performance information. The amplifier 164 is a signal amplification circuit, amplifies the signal supplied from the signal processing unit 123 and supplies the amplified signal to the speaker 163. When the keyboard 111 detects the movement of the key, it outputs it to the sensor 122. The generation unit 120 generates performance information. The generated performance information is supplied to the sound generator 160A together with the controller 170. Thereby, the detection unit 130A includes the antenna 114 and the signal processing circuit 114, and performs wireless communication with the musical instrument 100A according to a communication standard such as Bluetooth (registered trademark) or WiFi (registered trademark). In a preferred embodiment, a short-range wireless communication standard such as FeriCa (registered trademark) or ISO / IEC14443 is used. The input / output unit 150 includes display means such as a liquid crystal screen and liquid crystal drive, information output means such as a light emitting element and a speaker, and input means such as buttons and knobs. The user inputs setting information necessary for controlling the musical instrument 100A. Or, it is used to notify the control contents or to prompt the user to perform an operation. The sound collection unit 155 is composed of a microphone and a signal processing circuit, collects sounds including the sound emitted by the musical instrument 100A (singer's singing sound and other environmental sounds), and an acoustic signal such as analog / digital conversion. Processing is performed to generate information representing the collected sound. In the following operation example, the musical instrument 100 grasps that the mobile terminal 200 has a recording function when connected to the mobile terminal 200, and collects sounds including performance sounds by the sound collection unit 155. Assume that you decide not to use the function.

  Detection of the portable terminal 200 by the musical instrument 100A is performed by determining whether the musical instrument 100A and the portable terminal 200 are within a predetermined distance. Specifically, when the intensity of the radio signal received by the musical instrument 100A is greater than or equal to a predetermined value, it is determined that the musical instrument 100A has entered a predetermined distance from the portable terminal 200. However, even if the mobile terminal 200 is within a predetermined distance from the musical instrument 100A, preparation for photographing is not always completed. For example, the mobile terminal 200 may be stored in a pocket, the user (scheduled performer) may be far away from the instrument 100A, or the instrument 100A is moving around the instrument 100 and the instrument 100A is stationary. May not have. From this point of view, the portable terminal 200 recognizes the presence (approach) of the musical instrument 100A, and then determines whether the musical instrument 100A is ready for photographing (whether the position and orientation of the musical instrument 100A are suitable for photographing) from the sensor 203. It is preferable to use this information or information from an image acquired by the imaging unit 205.

  For example, the mobile terminal 200 first determines ambient brightness with the illuminance sensor of the sensor 203, and activates the imaging unit 205 when the illuminance is greater than or equal to a predetermined value. Next, the image captured by the image capturing unit 205 is analyzed to determine whether the target registration of the musical instrument 100A, the performer, or the like is shown, or whether the camera lens is in focus. In this determination, for example, the musical instrument 100A is photographed in advance and the video data thereof is registered in the storage unit 209, and the obtained photographing data is compared with the registered photographing data. Then, the mobile terminal 200 starts recording when it is determined that the shooting target is captured. Alternatively, the user of the musical instrument 100A who is going to start playing adjusts the position and orientation of the mobile terminal 200 one by one while confirming the captured image, and carries an instruction to start recording when the adjustment is completed. You may input into the terminal 200. FIG.

FIG. 10 shows an operation example when using the musical instrument 100A and the portable terminal 200 provided with a function (not shown) for performing wireless communication in accordance with a communication standard such as Bluetooth or WiFi instead of omitting the terminal. Indicates. FIGS. 11A to 11D show examples of screens displayed on the mobile terminal 200 in the operation example shown in FIG.
In FIG. 10, the processing from S100 to S106 is the same as the operation described with reference to FIG. 8 except that the detection process in S102 is performed in a non-contact manner, and thus the description thereof is omitted. Upon receiving the recording request Req1, the portable terminal 200 displays a message shown in FIG. 11A on the screen, thereby notifying that the portable terminal 200 and the musical instrument 100 can communicate with each other, and the portable terminal 200. The user is prompted to adjust the position and orientation of the camera to a state suitable for shooting (S201).

  A user who wishes to record selects the “Continue” soft button displayed on the screen, installs the mobile terminal 200 at a position where he / she wants to shoot, and adjusts the orientation (posture) of the musical instrument 100A so that the imaging unit faces the shooting target. To do. Further, detailed adjustment (setting of zoom magnification, focus, etc.) may be performed. Subsequently, the portable terminal 200 activates the imaging unit 205 to acquire shooting data, and sequentially acquires sensor information (acceleration, illuminance, etc.) using the sensor 203 (S203). Then, the mobile terminal 200 determines whether the preparation for shooting is complete using at least one of the analysis result of the video data and the sensor information from the imaging unit 205. This determination process is repeated at a predetermined cycle. When it is determined that the preparation for shooting is not complete (for example, when nothing is shown, when the projected video is too dark, or when the mobile terminal 200 is not stationary), the mobile terminal 200 is shown in FIG. A screen as shown is displayed to prompt the user to change the shooting environment. If it is determined that shooting preparation is complete, the video acquired by the imaging unit 205 is displayed and a message as shown in FIG. 11C is displayed to prompt the user to confirm shooting preparation (S205). As for a method for determining whether the imaging conditions are satisfied, for example, the musical instrument 100 or an image around the musical instrument 100 is registered and photographed by imaging the musical instrument 100 with the portable terminal 200 in advance. The similarity between the image data and the registered image data is calculated using a known resolution algorithm, and the determination is made based on the similarity.

When the user sees the video and is satisfied with the current shooting environment, the user selects the “OK” soft button (S207: YES). Then, the portable terminal 200 starts recording in this photographing environment (S108), and notifies that the recording is in progress as shown in FIG. When the notification Res1 is transmitted to the musical instrument 100 (S110), the user is notified with an image or sound that the performance recording is ready using the input / output unit 150 (S112).
The process of recording a performance is the same as the operation shown in FIG. However, in this example, the performance information is sequentially stored only by the musical instrument 100A, and the portable terminal receives MIDI data from the musical instrument 100 after the performance is completed (S120 and S124). The difference from the operation shown in FIG. 8 is that the musical instrument 100A transmits a recording stop request to the mobile terminal 200 when it is determined in step S116 that the performance has ended (S118). This is the point (S122) to end.

  According to this modification, it is not necessary to bring a device that has a recording function and is necessary to start recording a performance into contact with the musical instrument. As a result, the restriction on the shooting position is greatly relaxed, and the degree of freedom of video expression is improved. In addition, since the recording is started after the user confirms the shooting state, the recording may automatically start in a state that is not suitable for shooting, for example, the server 300 remains in the user's pocket. Is prevented. If the portable terminal 200 is provided, the player before the performance is notified that the musical instrument 100 and the portable terminal 200 are ready to record the performance sound and perform the recording of the performer. There is less risk of forgetting to record or record a performance.

  The performance information may be recorded in a format other than MIDI data (also referred to as MIDI message). For example, the instrument 100 is provided with a microphone to collect the sound emitted from the sound generation unit 160, and the collected sound is digitized together with or in place of the MIDI data to at least the instrument 100 and the portable terminal 200. Either of them may be stored. Further, the data obtained by recording only needs to include at least video data, and may include only video data or video data and audio data.

  The information recorded on the performance includes (a) performance information and shooting data (specifically (b) video data and (c) performance sound data (including environmental sound, singing sound, etc. depending on circumstances)) A total of three types of data are included, but a process for determining which data among (a) to (c) to be recorded or all to be stored may be introduced. For example, when the mobile terminal does not have an imaging function, only performance sound data is recorded. In addition, when the storage capacity or the signal processing speed is low, recording of a specific type of data may not be performed in the performance recording system. This determination is performed, for example, when a musical instrument and a portable terminal are connected.

  Further, the mobile terminal 200 may automatically adjust the shooting environment. For example, a drive mechanism for changing the orientation of the camera is provided in the imaging unit 205, and the control unit 210 controls the drive mechanism so as to realize an optimum photographing state. That is, the portable terminal used in the present invention starts recording when it is at a predetermined position with respect to the musical instrument, and communicates with the musical instrument 100 in a contact state or a non-contact state to represent the performance content performed by the musical instrument. What is necessary is just to have at least the function to acquire information.

Regarding the selection of whether the performance information or MIDI data recording destination is the musical instrument 100 (100A) or the portable terminal 200, the recording destination may be fixedly determined in advance, or each time the portable terminal 200 is detected. It may be determined. Further, when the recording destination is always set to the portable terminal 200, the musical instrument 100 does not need to be provided with the storage unit 140. In this case, the musical sound data is sequentially transferred to the portable terminal 200. As a specific determination method of the recording destination, for example, when the portable terminal 200 is detected, the instrument 100 (100A) and the portable terminal 200 perform a predetermined negotiation. As an example, information indicating the priority of the recording location is stored in advance in each of the musical instrument 100 (100A) and the portable terminal 200, and the device to which the higher priority is set by referring to this information is the storage destination. It is determined that
Alternatively, the recording destination may be determined according to the musical instrument 100, the portable terminal 200, and the functional configuration. For example, when the performance information generation function, recording function, and sound collection function are provided on both the instrument side and the portable terminal side, the performance information is generated, recorded, and collected on the higher device. Sound may be carried.
Alternatively, the user may input information specifying a recording destination to the musical instrument 100 (100A) or the portable terminal 200. Note that recording and sound collection may be executed simultaneously (in parallel) on both the musical instrument 100 (100A) and the portable terminal 200.

In short, the performance recording system of the present invention is a performance recording system having a musical instrument and a portable terminal, a storage means, a reception means for receiving a performance operation, and a generation means for generating performance information representing the received performance operation. An imaging unit that captures a scene in which the accepted performance operation is performed, a detection unit that detects that the portable terminal and the instrument are in a predetermined positional relationship, and the portable terminal is detected. And at least means for starting a process of recording at least one of the performance information generated by the generating means and the video imaged by the imaging means in the storage means.
The musical instrument according to the present invention detects an operation unit that receives a performance operation, a generation unit that generates performance information representing the received performance operation, a detection unit that detects a portable terminal, and the portable terminal. , And starts processing to record at least one of the performance information generated by the generation unit and the video of the scene where the performance operation is performed on at least one of the instrument and the portable terminal It is only necessary to have a control unit to perform.

DESCRIPTION OF SYMBOLS 100, 100A ... Musical instrument, 110 ... Operation part, 120 ... Generation part, 130 ... Detection part, 140 ... Memory | storage part, 150 ... Input / output part, 155 ... Sound collection Part, 160 ... sound generation part, 170 ... control part, 121 ... drive mechanism, 122 ... sensor, 123 ... signal processing part, 111 ... keyboard, 112 ... pedal, 161 ... string, 162 ... soundboard, 163 ... speaker, 164 ... amplifier, 300 ... server, 301 ... control unit, 302 ... input / output unit, 303 ... Storage unit 304 ... Communication unit 200 ... Mobile terminal 201 ... Display unit 202 ... Input unit 203 ... Sensor 204 ... Recording unit 205 ... Imaging unit 206 ... 2nd communication part, 207 ... Sound emission part, 208 ... 1st communication part, 09 ... storage unit, 210 ... controller, 134 ... wireless communication module, 131 ... terminal, 132 ... pedestal, 1081,2081 ... terminal

Claims (7)

An operation unit for receiving performance operations;
A generating unit for generating performance information representing the received performance operation;
A detection unit for detecting a mobile terminal;
When the portable terminal is detected, at least one of the performance information generated by the generation unit and a video of a scene in which the performance operation is performed is selected from at least the musical instrument and the portable terminal. And a control unit that starts a process of recording in one of the instruments. The musical instrument according to claim 1, wherein the control unit stops the processing when the mobile terminal is no longer detected by the detection unit. When the portable terminal is detected, the control unit photographs the performance information generated by the generation unit, the sound information collected by the sound collection unit, and the scene where the performance operation is performed. The musical instrument according to claim 1 or 2, wherein a process of recording at least one of the images on at least one of the musical instrument and the portable terminal is started. An interface for communicating with the detected mobile terminal;
The musical instrument according to any one of claims 1 to 3, wherein an instruction to start capturing the video by the imaging unit is output to the detected portable terminal via the interface. A storage unit;
An interface for communicating with the detected mobile terminal;
The musical instrument according to any one of claims 1 to 4, wherein performance information recorded in the storage unit is output to the portable terminal after the performance is completed. In a performance recording system having a musical instrument and a portable terminal,
Receiving a performance operation on the instrument;
Generating performance information representing the performance operation accepted by the instrument;
The instrument detecting a portable terminal;
When the portable terminal is detected, at least one of the generated performance information and a video of the scene where the performance operation is performed is captured in at least one of the musical instrument and the portable terminal. A performance recording method comprising: a step of starting a recording process. In a performance recording system having a musical instrument and a portable terminal,
Storage means;
A reception means for receiving a performance operation;
Generating means for generating performance information representing the accepted performance operation;
Imaging means for photographing the scene where the accepted performance operation is performed;
Detecting means for detecting that the portable terminal and the musical instrument are in a predetermined positional relationship;
Means for starting a process of recording at least one of the performance information generated by the generating means and the video imaged by the imaging means in the storage means when the portable terminal is detected; A performance recording system.

Images