WO2019087331A1 - Dispositif et procédé de détection de position de référence - Google Patents

Dispositif et procédé de détection de position de référence Download PDF

Info

Publication number
WO2019087331A1
WO2019087331A1 PCT/JP2017/039578 JP2017039578W WO2019087331A1 WO 2019087331 A1 WO2019087331 A1 WO 2019087331A1 JP 2017039578 W JP2017039578 W JP 2017039578W WO 2019087331 A1 WO2019087331 A1 WO 2019087331A1
Authority
WO
WIPO (PCT)
Prior art keywords
keyboard
image data
reference position
position detection
unit
Prior art date
Application number
PCT/JP2017/039578
Other languages
English (en)
Japanese (ja)
Inventor
大地 渡邉
Original Assignee
ヤマハ株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ヤマハ株式会社 filed Critical ヤマハ株式会社
Priority to PCT/JP2017/039578 priority Critical patent/WO2019087331A1/fr
Publication of WO2019087331A1 publication Critical patent/WO2019087331A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H1/00—Details of electrophonic musical instruments
    • G10H1/18—Selecting circuits

Definitions

  • the present invention relates to a reference position detection device and a reference position detection method for detecting a reference position on a keyboard.
  • Patent Document 1 describes a key area dividing device capable of automatically changing the key area dividing position according to the playing state of the performer.
  • the entire key area of the keyboard is imaged by two cameras. The images obtained by each camera are analyzed, and the type of hand of the player's right hand or left hand etc. based on the arrangement of each finger, hand size, finger length, finger thickness, skin color etc. And the position of the hand is detected.
  • the key area division position is dynamically changed based on the detected hand type and hand position.
  • An object of the present invention is to provide a reference position detection device and a reference position detection method capable of detecting a reference position on a keyboard with simpler processing and at low cost.
  • a reference position detection device includes a keyboard and an acquisition unit for acquiring image data representing an image obtained by imaging an object on the keyboard, and a white key not including a black key from the acquired image data. And an identification unit for identifying an object on the keyboard based on the luminance value of the extracted image data, and a predetermined position on the keyboard with respect to the identified object. And a detection unit that detects a reference position having a relationship.
  • the object may be the player's hand.
  • the hand may be both hands, and the reference position may be a position between the hands.
  • the reference position may be a key area division position that divides the keyboard into a plurality of key areas to which individual tone control characteristics are assigned.
  • the reference position detection device may further include a resolution reduction unit that reduces the resolution of the extracted image data, and the identification unit may identify an object on the keyboard based on the luminance value of the image data after the resolution reduction.
  • the reference position detection device may further include a blurring processing unit that performs blurring processing on the extracted image data, and the identification unit may identify an object on the keyboard based on the luminance value of the image data after the blurring processing.
  • the reference position detection device may further include a keyboard and an imaging device for imaging an object on the keyboard, and the acquisition unit may acquire image data from the imaging device.
  • the imaging device is a camera having a fisheye lens
  • the reference position detection device further includes a correction unit that corrects the acquired image data so that the keyboard has a rectangular shape, and the extraction unit extracts a black key from the corrected image data.
  • the image data corresponding to the area not including white and including the white key may be extracted.
  • the acquisition unit may acquire image data indicating an image including the entire keyboard.
  • a reference position detection method comprises: obtaining an image data representing an image obtained by imaging a keyboard and an object on the keyboard; and a white key not including a black key from the acquired image data Having a predetermined positional relationship on the keyboard with respect to the identified object, the steps of extracting image data representing an area including A, identifying the object on the keyboard based on the brightness value of the extracted image data, and Detecting a reference position.
  • FIG. 1 is a block diagram showing the configuration of an electronic music apparatus including a reference position detection apparatus according to an embodiment of the present invention.
  • FIG. 2 is a block diagram mainly showing a functional configuration of the reference position detection device of FIG.
  • FIG. 3 is a flowchart showing a reference position detection method in the reference position detection device of FIG.
  • FIG. 4 is a diagram showing the keyboard and the hands of the player.
  • FIG. 5 is a schematic view showing an example of an image subjected to resolution reduction processing.
  • FIG. 6 is a schematic view showing an example of the image subjected to the blurring process and the gray scale process.
  • FIG. 7 is a schematic view showing an example of a binarized image.
  • FIG. 8 is a schematic view showing an example of a histogram.
  • FIG. 9 is a flowchart showing the area identification process of step S10 of FIG.
  • FIG. 10 is a flowchart showing the key area division position detection process of step S11 of FIG.
  • FIG. 1 is a block diagram showing the configuration of an electronic music device including a reference position detection device according to an embodiment of the present invention.
  • the electronic music apparatus 1 of FIG. 1 is, for example, an electronic keyboard instrument.
  • the electronic music apparatus 1 includes an imaging device 30, a keyboard 2, a setting operation unit 3, and a display 4.
  • the keyboard 2 has a plurality of black keys and white keys and is connected to the bus 14.
  • the keyboard 2 may be an image of the keyboard displayed on the screen of the touch panel described later.
  • the imaging device 30 is disposed to capture an image of the keyboard 2 and outputs image data representing an image of the keyboard 2 to the bus 14.
  • the imaging device 30 is a single camera having a fisheye lens.
  • the imaging device 30 is disposed above the central portion of the keyboard 2 or obliquely rearward above the central portion, and images the entire keyboard 2. As a result, image data indicating the entire image of the keyboard 2 is output from the imaging device 30.
  • the setting operation unit 3 includes an operation switch that is turned on and off, an operation switch that is rotated, and an operation switch that is slid and is connected to the bus 14.
  • the setting operation unit 3 is used to perform various instructions, various adjustments, various settings, and power on / off.
  • the display 4 includes, for example, a liquid crystal display, and is connected to the bus 14.
  • the display 4 displays a music title, a score, or other various information.
  • the display 4 may be a touch panel display. In this case, a part or all of the keyboard 2 or the setting operation unit 3 may be displayed on the display 4. The user can instruct various operations by operating the display 4.
  • the electronic music apparatus 1 includes a sound source unit 5 and a sound system 6.
  • the sound source unit 5 is connected to the bus 14 and generates audio data (sound signal) based on tone data etc. reproduced by the operation of the keyboard 2. Audio data is sampling data indicating a sound waveform.
  • the sound system 6 includes a digital analog (D / A) conversion circuit, an amplifier and a speaker. The sound system 6 converts audio data supplied from the sound source unit 5 into an analog sound signal, and generates sound based on the analog sound signal.
  • the electronic music apparatus 1 further includes a storage device 7, a CPU (central processing unit) 8, a timer 9, a RAM (random access memory) 10, a ROM (read only memory) 11, and a communication I / F (interface) 12. .
  • the storage device 7, the CPU 8, the RAM 10, the ROM 11 and the communication I / F 12 are connected to the bus 14.
  • the timer 9 is connected to the CPU 8.
  • An external device such as the external storage device 13 may be connected to the bus 14 via the communication I / F 12.
  • the storage device 7 includes a storage medium such as a hard disk, an optical disk, a magnetic disk or a memory card.
  • the storage device 7 stores white key area coordinate information, note number correspondence information, and key area division position information described later. Further, the storage device 7 stores a computer program such as a reference position detection program. Furthermore, the storage device 7 stores performance data indicating a performance sound of a musical instrument and automatic accompaniment data for automatic accompaniment.
  • the tone data includes performance data and automatic accompaniment data.
  • the musical tone data may be audio data or MIDI (Musical Instrument Digital Interface) data.
  • the RAM 10 is, for example, a volatile memory and is used as a work area of the CPU 8 and temporarily stores various data.
  • the ROM 11 is, for example, a non-volatile memory, and stores a control program.
  • the ROM 11 may store a computer program such as a reference position detection program.
  • the CPU 8 executes a reference position detection program stored in the storage device 7 or the ROM 11 to perform a reference position detection method described later.
  • the timer 9 provides the CPU 8 with time information indicating time lapse.
  • the CPU 8, the timer 9, the RAM 10 and the ROM 11 constitute a reference position detection device 100.
  • the reference position detection program may be provided in a form stored in a computer readable recording medium, and may be installed in the storage device 7 or the ROM 11. Also, the reference position detection program may be stored in the external storage device 13. Furthermore, when the communication I / F 12 is connected to the communication network, a reference position detection program distributed from a server connected to the communication network may be installed in the storage device 7 or the ROM 11.
  • FIG. 2 is a block diagram mainly showing the functional configuration of the reference position detection device 100 of FIG.
  • the reference position detection device 100 includes an acquisition unit 101, a correction unit 102, an extraction unit 103, a resolution reduction unit 104, a blurring processing unit 105, and a gray scale processing unit 106.
  • the reference position detection device 100 further includes a binarization unit 107, a generation unit 108, an identification unit 109, a detection unit 110, a presentation unit 111, and a tone control characteristic setting unit 112.
  • the function of each component (101 to 112) of the reference position detection device 100 is realized by the CPU 8 of FIG. 1 executing a reference position detection program stored in the storage device 7 or the ROM 11.
  • the imaging device 30 images the keyboard 2 and the hands of the player on the keyboard 2 and outputs image data representing an image including the entire keyboard 2 and a part of the hand.
  • the image data is color image data representing a color image.
  • Color image data is composed of a plurality of pixels having luminance values of a color image.
  • the acquisition unit 101 acquires image data output from the imaging device 30.
  • the imaging device 30 is a camera having a fisheye lens
  • the image of the keyboard 2 represented by the image data is distorted such that the central portion is enlarged compared to the both ends.
  • the correction unit 102 corrects the image data acquired by the acquisition unit 101 so that the image of the keyboard 2 has a rectangular shape.
  • the extraction unit 103 does not include a black key from the image data corrected by the correction unit 102 based on the white key region coordinate information acquired from the storage device 7 and includes a white key (hereinafter referred to as a white key region). Extract the image data corresponding to.
  • the white key area coordinate information includes coordinates for specifying a white key area in the entire image of the keyboard 2.
  • the resolution reduction unit 104 performs resolution reduction processing to reduce the resolution of the image data extracted by the extraction unit 103.
  • the blurring processing unit 105 performs blurring processing on the image data having the resolution reduced by the resolution reducing unit 104.
  • the grayscale processing unit 106 converts the color image data into grayscale image data by performing grayscale processing on the image data subjected to the blurring processing.
  • the luminance value of each pixel of gray scale image data represents a gradation value.
  • the binarization unit 107 performs binarization processing to convert the luminance value of the image data obtained by the gray scale processing unit 106 into binary pixel values. Thereby, binary image data is generated. Binary image data is composed of a plurality of pixels having a value of “0” or “1”.
  • the generation unit 108 performs a histogram generation process of generating a histogram based on each pixel value of the image data obtained by the binarization unit 107. The histogram generation process will be described later.
  • the identification unit 109 identifies the hand on the keyboard 2 based on the histogram generated by the generation unit 108.
  • the detection unit 110 detects the note number of the reference position having a predetermined positional relationship with the hand identified by the identification unit 109 based on the note number correspondence information acquired from the storage device 7. Details of operations of the identification unit 109 and the detection unit 110 will be described later.
  • the predetermined positional relationship is the central position between the left hand and the right hand
  • the reference position is the key area division position (split point).
  • the note number correspondence information indicates the correspondence between a plurality of pixels constituting the image of the keyboard 2 and the note number.
  • the note number of the key area division position detected by the detection unit 110 is stored in the storage device 7 as key area division position information. Thereby, the key area division position is set to the key having the note number indicated by the key area division position information.
  • the presentation unit 111 presents the key area division position detected by the detection unit 110 to the player using the guide lamp 41, the key light emission unit 42 or the display 4.
  • the guide lamp 41 is composed of a plurality of light emitting diodes disposed behind the plurality of keys of the keyboard 2.
  • the presentation unit 111 presents the key area division position to the player by lighting the light emitting diode disposed behind the key for which the key area division position is set.
  • the presentation unit 111 causes the key area division by emitting the key emission unit 42 provided for the key for which the key area division position is set.
  • the position may be presented to the player.
  • the presentation unit 111 may cause the display 4 to display the image of the keyboard 2 and display the key area division position on the image of the keyboard 2.
  • the musical tone control characteristic setting unit 112 sets individual musical tone control characteristics in two key areas divided left and right at the key area division position of the keyboard 2 based on the key area division position information stored in the storage device 7 Do.
  • the individual tone control characteristics are, for example, an individual tone, an individual tone range or an individual function.
  • the tone color of the base may be set in the left key range
  • the tone color of the piano may be set in the right key range.
  • a range two octaves higher than the original range may be set in the left key range, and the original range may be set in the right range.
  • an automatic accompaniment function by chords may be set in the left key range, and a normal performance function may be set in the right key range.
  • FIG. 3 is a flowchart showing a reference position detection method in the reference position detection apparatus 100 of FIG.
  • FIG. 4 shows the keyboard 2 and the hands of the player.
  • FIG. 5 is a schematic view showing an example of an image subjected to resolution reduction processing.
  • FIG. 6 is a schematic view showing an example of the image subjected to the blurring process and the gray scale process.
  • FIG. 7 is a schematic view showing an example of a binarized image.
  • FIG. 8 is a schematic view showing an example of a histogram.
  • the reference position detection method of FIG. 3 is performed by the CPU 8 of FIG. 1 executing a reference position detection program stored in the storage device 7 or the ROM 11.
  • the keyboard 2 has an array of a plurality of black keys BK and white keys WH.
  • a plurality of black keys BK and white keys WH are assigned note numbers that increase in order from left to right. For example, when the keyboard 2 has 88 keys, note numbers “21” to “108” are assigned in order from the leftmost key to the rightmost key.
  • a linear gap 21 exists between each two adjacent white keys WH.
  • the left hand LH and the right hand RH of the player exist on the keyboard 2.
  • a key range division position (split point) SP is set between the left hand LH and the right hand RH by a method described later.
  • the white key area 22 is a rectangular area not including the black key BK in the keyboard 2 but including the white key WH.
  • the acquisition unit 101 of FIG. 2 acquires image data representing the entire image of the keyboard 2 from the imaging device 30 (step S1).
  • the imaging device 30 is a single camera having a fisheye lens
  • the image of the keyboard 2 represented by the image data is enlarged so that the central portion is larger than both ends. It is distorted. Therefore, the correction unit 102 corrects the image data acquired by the acquisition unit 101 so that the image of the keyboard 2 has a rectangular shape (step S2).
  • the extraction unit 103 acquires white key area coordinate information from the storage device 7 (step S3), and extracts image data corresponding to the white key area 22 from the image data based on the acquired white key area coordinate information (step S4). ).
  • white key area coordinate information indicating the position of the white key area 22 on the image is stored in advance in the storage device 7.
  • the white key area coordinate information may be automatically calculated by image processing based on the luminance difference between the outer periphery of the keyboard 2 and the white key WH and the luminance difference between the black key BK and the white key WH in the image.
  • white key area coordinate information may be generated by the user designating four corners or the outer periphery of the white key area 22 of the keyboard 2 on the image displayed on the display 4. Furthermore, white key area coordinate information may be generated by the user inputting the coordinates of the white key area 22 on the image. Coordinates in the horizontal direction (left and right direction) and the vertical direction (front and rear direction) are assigned to a plurality of pixels forming the image data of the white key area 22.
  • the resolution reduction unit 104 performs resolution reduction processing on the extracted image data (step S5).
  • the resolution reduction process for example, the number of pixels forming the image data is reduced by averaging the luminance values of adjacent constant pixel groups. Thereby, as shown in FIG. 5, the black straight line representing the gap 21 between the white keys WH in FIG. 4 is blurred on the image of the white key area 22.
  • the luminance values of the left hand portion IL and the right hand portion IR corresponding to the left hand LH and the right hand RH in the image of the white key area 22 are lower than the luminance values of other portions corresponding to the white key WH.
  • the width of the gap between the white keys WH is enlarged at the central portion of the white key area 22 on the image.
  • a thick vertical line 21a representing the gap 21 between the white keys WH remains at the center of the image of the white key area 22.
  • the blurring processing unit 105 performs blurring processing on the image data having the resolution reduced by the resolution reducing unit 104 (step S6).
  • the gray scale processing unit 106 performs gray scale processing on the image data on which the blurring process has been performed (step S7). Thereby, image data representing a gray scale image is obtained.
  • the vertical line 21a of FIG. 5 representing the gap 21 between the white keys WH is eliminated by being equalized to the color of the white key WH.
  • the luminance values (gradation values) of the left hand portion IL and the right hand portion IR become lower than the luminance values (gradation values) of the other portions.
  • the binarization unit 107 performs a binarization process of converting the luminance value of the image data obtained by the gray scale processing unit 106 into a binary pixel value (step S8).
  • binary image data representing a binary image is generated.
  • Binary image data is composed of a plurality of pixels having a value of “0” or “1”.
  • the value of a pixel having a luminance value higher than a predetermined luminance threshold value is converted to "0”
  • the value of a pixel having a luminance value smaller than the luminance threshold value is "1". It is converted.
  • the brightness threshold value in the binarization process is adjusted by the setting switch of the setting operation unit 3 in accordance with the brightness around the keyboard 2 or the like.
  • the pixel values of the left hand portion IL and the right hand portion IR are “1” (white), and the pixel values of the other portions are “0” (black).
  • the left hand part IL and the right hand part IR are clearly distinguished from other parts.
  • the generation unit 108 performs histogram generation processing based on binary image data (step S9). Specifically, in the binary image data, the generation unit 108 calculates the sum of pixel values for each pixel row arranged parallel to the longitudinal direction (front-back direction) of each white key WH of the keyboard 2 A histogram is generated which indicates the sum of pixel values of a pixel row (referred to as a total pixel value) as a frequency. Each section on the horizontal axis of the histogram HG in FIG. 8 is each pixel row, and the frequency on the vertical axis is a total pixel value.
  • the histogram HG shows the distribution of total pixel values of a plurality of pixel columns. The higher the total pixel value of each pixel column, the higher the probability that a hand is present. Further, in the histogram HG, a predetermined determination threshold TH is set.
  • the identification unit 109 performs region identification processing for identifying a hand on the keyboard 2 based on the histogram HG (step S10). Details of the area identification process will be described later. Thereafter, based on the hand area identified by the identification unit 109, the detection unit 110 performs key area division position detection processing for detecting a key area division position, which is a reference position (step S11). The keyboard 2 is divided into a left key area and a right key area around the detected key area division position. Details of the key area division position detection process will be described later.
  • the musical tone control characteristic setting unit 112 assigns individual musical tone control characteristics to the left key area and the right key area (step S12). Also, the presentation unit 111 presents the key area division position to the player using the guide lamp 41, the key light emission unit 42 or the display 4 (step S13).
  • the detection unit 110 determines whether the end of the reference position detection process has been instructed (step S14). When the end of the reference position detection process is not instructed, the detection unit 110 returns to step S1. Thus, the processes of steps S1 to S14 are performed. In this way, when the player's hand moves on the keyboard 2, the key area division position moves dynamically along with the movement of the hand. When the end of the reference position detection process is instructed in step S14, the reference position detection process ends.
  • an instruction to end the reference position detection process is generated, for example, by the end of the music performance or the operation of the operation switch of the setting operation unit 3.
  • FIG. 9 is a flowchart showing the area identification process of step S10 of FIG.
  • the identification unit 109 extracts a pixel sequence having a total pixel value equal to or greater than the determination threshold TH in the histogram HG generated by the generation unit 108 (step S101), and selects one or more adjacent pixel sequences among the extracted pixel sequences.
  • An area composed of pixel columns is extracted (step S102).
  • the user can operate the setting switch of the setting operation unit 3 in advance to display the histogram HG on the display 4 and adjust the determination threshold TH.
  • the identification unit 109 determines whether the width of the extracted area is equal to or greater than a predetermined value (step S103).
  • the predetermined value is preset based on the width of the human hand. If the width of the extracted area is equal to or greater than the predetermined value, the identification unit 109 identifies the extracted area as a hand area (step S104).
  • the position of the hand area in the white key area 22 is represented by the coordinates in the horizontal direction (left and right direction) in the white key area 22.
  • step S103 when the width of the extracted area is smaller than the predetermined value, the identification unit 109 determines that the extracted area is not a hand area (step S105). Thereby, the area due to noise is excluded from the hand area.
  • the identification unit 109 determines whether the extraction of all the regions in the image data is completed (step S106). If the extraction of all the areas has not been completed, the identification unit 109 returns to step S102 to extract another area. When the extraction of all the areas is completed in step S106, the identification unit 109 ends the area identification process, and the process proceeds to step S11 in FIG.
  • FIG. 10 is a flowchart showing the key area division position detection process of step S11 of FIG.
  • the detection unit 110 determines whether the identification unit 109 has identified two hand regions (step S111). If two hand areas are identified, the detection unit 110 calculates the coordinates of the center position of each hand area (step S112), and the coordinates of the center position between the coordinates of the center positions of the two hand areas are keyed The coordinates of the area division position are calculated (step S113). In the example of FIG. 8, two hand regions R1 and R2 are identified.
  • the coordinates of the central position P1 of the hand region R1 and the coordinates P2 of the central region P2 of the hand region R2 are calculated, and the coordinates of the central position P0 between the coordinates of the central positions P1 and P2 of the two hand regions R1 and R2 is a key Calculated as the coordinates of the area division position.
  • the detection unit 110 acquires note number correspondence information from the storage device 7 (step S114), and assigns a note number to the key area division position based on the note number correspondence information (step S115). After that, the detection unit 110 stores key area division position information indicating the note number of the key area division position in the storage device 7 (step S116). As a result, the key area division position is set to the keyboard 2. When the key area division position information is already stored in the storage device 7, the detection unit 110 updates the key area division position information stored in the storage device 7. Thereafter, the detection unit 110 proceeds to step S12 in FIG.
  • step S111 the detection unit 110 proceeds to step S12 in FIG. 3 without storing the key area division position information in the storage device 7 (step S117).
  • step S117 the detection unit 110 proceeds to step S12 without updating the key area division position information of the storage device 7.
  • the hand on the keyboard 2 is identified based on the value. In this case, the hand on the keyboard 2 is identified with high accuracy from other parts by a simple process based on the luminance difference between the white key WH and the hand. Further, the key area division position is detected by a simple process as a reference position having a predetermined positional relationship on the keyboard 2 with respect to the identified hand. Therefore, it becomes possible to detect the key area division position on the keyboard 2 with simpler processing and at low cost.
  • the player's hand since the player's hand is identified as an object on the keyboard 2, it is possible to set a key area division position which dynamically changes in accordance with the movement of the hand on the keyboard 2. Further, in the present embodiment, since the key area division position between both hands is detected as the reference position, the key area division position dynamically changing according to the movement of the player's hands may be set between the both hands. it can.
  • the vertical line due to the gap 21 between the white keys WH can be removed from the image of the white key area 22. Thereby, the hand on the keyboard 2 can be more accurately identified from other parts.
  • the blurring process is performed on the image data by the blurring processing unit 105, the vertical line 21a due to the gap 21 between the white keys WH remaining in the image of the white key area 22 can be removed.
  • the entire keyboard 2 can be imaged by a single camera. Further, since the image of the keyboard 2 is corrected to have a rectangular shape by the correction unit 102, it is possible to extract image data that accurately represents the rectangular white key area 22. Furthermore, in the present embodiment, since image data representing the entire image of the keyboard 2 is obtained by the imaging device 30, the hand on the keyboard 2 can be more accurately identified from other parts.
  • the player's hand is identified as an object on the keyboard 2, but the object on the keyboard 2 is not limited to the hand. It may be any object of
  • the key area division position of the center position between the player's hands is detected as the reference position having a predetermined positional relationship with the object, but the reference position is limited to the key area division position I will not.
  • the left end, the right end, or both ends of each player's hand may be detected as the reference position.
  • information indicating the octave corresponding to the position of the hand on the keyboard on the teaching score displayed on the screen for example, the pitch played by the player with respect to the octave to which the pitch represented in the score belongs.
  • Information indicating whether the value is high or low may be displayed.
  • the pitch played by the performer is distributed to the treble staff and the bass staff based on the position of the hand on the keyboard 2. May be displayed.
  • the played pitch is the pitch that should normally be displayed on the bass staff, if it is inferred from the position of the player's right hand that the pitch is played with the right hand, then it will be played.
  • the pitch may be displayed on the treble staff using an additive line.
  • the pitch played was normally to be displayed on the treble staff, if it was inferred from the position of the player's left hand that the pitch was played with the left hand, The indicated pitch may be displayed on the bass staff using an additive line.
  • the key area division position may be calculated and updated based on one hand area.
  • the key area division position may be calculated based on the position of the left end, the right end, or the center of one detected hand area. Specifically, note numbers separated by a predetermined number of pitches from the note numbers corresponding to the positions of the end of the detected hand area or the positions of the end may be calculated as the key area division position .
  • one key area division position is detected as the reference position, but a plurality of reference positions may be detected.
  • the keyboard 2 may be divided into three or more key areas by detecting a plurality of key area division positions as a plurality of reference positions.
  • the imaging device 30 is a camera having a fisheye lens, but the present invention is not limited to this, and the imaging device 30 may be a camera having no fisheye lens. Also, the imaging device 30 may include a plurality of cameras. Furthermore, in the above embodiment, the coordinates of the key area division position are calculated after the image data obtained by the imaging device 30 is corrected, but the correction is not performed on the image data obtained by the imaging device 30. Coordinates of the area division position may be calculated, and the calculated coordinates may be converted into note numbers in consideration of distortion of the image due to the fisheye lens.
  • image data representing the entire image of the keyboard 2 is obtained by the imaging device 30.
  • image data representing a partial image of the keyboard 2 may be obtained by the imaging device 30.
  • the imaging range of the keyboard 2 by the imaging device 30 is determined in advance, it is possible to specify the note number corresponding to each part of the image.
  • an index or the like for specifying a note number is attached to a part of keys of the keyboard 2, it is possible to specify a note number corresponding to each part of the image.
  • the user may be able to set the imaging range (key area) of the imaging device 30 by the setting operation unit 3.
  • the imaging range (key area) may be automatically set by the angle of the imaging device 30.
  • the acquisition unit 101 acquires image data representing a color image from the imaging device 30.
  • the acquisition unit 101 may acquire image data representing a monochrome image from the imaging device 30.
  • the gray scale processing unit 106 may not be provided.
  • the generation unit 108 generates a histogram indicating the sum of the number of “0s” of each pixel column as a frequency.
  • the reference position detection apparatus and the reference position detection method of the present invention are applicable not only to electronic keyboard instruments but also to electronic devices such as smart phones, tablet terminals, personal computers and the like.
  • the keyboard may be displayed on the screen, or the keyboard may be connected to the electronic device.
  • the reference position detection device 100 of FIG. 2 is realized by a computer such as the CPU 8 and a reference position detection program, but the reference position detection device 100 of FIG. 2 is realized by hardware such as an electronic circuit. It may be done.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

L'invention concerne un dispositif de détection de position de référence équipé d'une unité d'acquisition, d'une unité d'extraction, d'une unité d'identification, et d'une unité de détection. L'unité d'acquisition acquiert des données d'image représentant une image obtenue par imagerie d'un clavier et d'un objet sur le clavier. L'unité d'extraction extrait des données d'image, correspondant à une zone qui comprend des touches blanches mais pas de touches noires, à partir des données d'image acquises par l'unité d'acquisition. L'unité d'identification identifie l'objet sur le clavier sur la base de la valeur de luminance des données d'image extraites par l'unité d'extraction. L'unité de détection détecte, pour l'objet identifié par l'unité d'identification, une position de référence qui présente une relation de position prescrite sur le clavier.
PCT/JP2017/039578 2017-11-01 2017-11-01 Dispositif et procédé de détection de position de référence WO2019087331A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/JP2017/039578 WO2019087331A1 (fr) 2017-11-01 2017-11-01 Dispositif et procédé de détection de position de référence

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2017/039578 WO2019087331A1 (fr) 2017-11-01 2017-11-01 Dispositif et procédé de détection de position de référence

Publications (1)

Publication Number Publication Date
WO2019087331A1 true WO2019087331A1 (fr) 2019-05-09

Family

ID=66331585

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2017/039578 WO2019087331A1 (fr) 2017-11-01 2017-11-01 Dispositif et procédé de détection de position de référence

Country Status (1)

Country Link
WO (1) WO2019087331A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111695499A (zh) * 2020-06-09 2020-09-22 小叶子（北京）科技有限公司 一种琴键识别方法、装置、电子设备及存储介质
CN112818981A (zh) * 2021-01-15 2021-05-18 小叶子（北京）科技有限公司 一种乐器演奏键位提示方法、装置、电子设备及存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007240929A (ja) * 2006-03-09 2007-09-20 Yamaha Corp 鍵域分割位置決定装置
JP2009122753A (ja) * 2007-11-12 2009-06-04 Sharp Corp 画像処理装置
JP4389841B2 (ja) * 2005-05-31 2009-12-24 ヤマハ株式会社 鍵域分割装置及びプログラム
JP2014063265A (ja) * 2012-09-20 2014-04-10 Casio Comput Co Ltd 画像処理装置、画像処理方法及びプログラム

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4389841B2 (ja) * 2005-05-31 2009-12-24 ヤマハ株式会社 鍵域分割装置及びプログラム
JP2007240929A (ja) * 2006-03-09 2007-09-20 Yamaha Corp 鍵域分割位置決定装置
JP2009122753A (ja) * 2007-11-12 2009-06-04 Sharp Corp 画像処理装置
JP2014063265A (ja) * 2012-09-20 2014-04-10 Casio Comput Co Ltd 画像処理装置、画像処理方法及びプログラム

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111695499A (zh) * 2020-06-09 2020-09-22 小叶子（北京）科技有限公司 一种琴键识别方法、装置、电子设备及存储介质
CN111695499B (zh) * 2020-06-09 2024-04-16 小叶子（北京）科技有限公司 一种琴键识别方法、装置、电子设备及存储介质
CN112818981A (zh) * 2021-01-15 2021-05-18 小叶子（北京）科技有限公司 一种乐器演奏键位提示方法、装置、电子设备及存储介质
CN112818981B (zh) * 2021-01-15 2023-12-12 小叶子（北京）科技有限公司 一种乐器演奏键位提示方法、装置、电子设备及存储介质

Similar Documents

Publication Publication Date Title
KR101524279B1 (ko) Ｒｇｂ-ｄ 카메라를 이용한 피아노 교육 시스템 및 그 방법
US9747876B1 (en) Adaptive layout of sheet music in coordination with detected audio
AU6735300A (en) Method and apparatus for selecting harmonic color using harmonics, and method and apparatus for converting sound to color or color to sound
US8067682B2 (en) Music score recognition method and system thereof
KR102306765B1 (ko) 인공지능 프로세싱을 이용한 악보 인식 방법
WO2019087331A1 (fr) Dispositif et procédé de détection de position de référence
US8744136B2 (en) Sound reproduction method and device
JP2019159145A (ja) 情報処理方法、電子機器及びプログラム
KR101896193B1 (ko) 이미지를 음악으로 변환하는 방법
US20080168893A1 (en) Sound Generating Method
CN111052220B (zh) 和弦信息提取装置、和弦信息提取方法及存储装置
JP6746892B2 (ja) 画像処理装置、画像処理方法およびプログラム
CN114419634B (zh) 一种基于特征规则的曲谱解析方法与装置
CA2885712A1 (fr) Procedes et systemes pour transcription musicale visuelle
US9418637B1 (en) Methods and systems for visual music transcription
JP2010204450A (ja) 楽譜表示装置および楽譜表示方法を実現するためのプログラム
JP3643829B2 (ja) 楽音生成装置、楽音生成プログラムおよび楽音生成方法
KR20070094207A (ko) 영상을 소리로 변환하는 방법 및 장치
Deb et al. An image analysis approach for transcription of music played on keyboard-like instruments
JP6496998B2 (ja) 演奏情報編集装置および演奏情報編集プログラム
KR101965694B1 (ko) 휴대 단말기가 광고 이미지 또는 동영상을 제공하는 방법 및 그 휴대 단말기
KR102089207B1 (ko) 이미지에 기초한 음악 재생 방법 및 이를 수행하기 위한 프로그램을 제공하는 서버
KR102241864B1 (ko) 이미지 분석에 기반한 콘텐츠 제공 장치, 이를 포함하는 콘텐츠 제공 시스템 및 이의 콘텐츠 제공 방법
WO2019132126A1 (fr) Dispositif opérationnel pour service de composition à base de contenu graphique
JPH06332443A (ja) 楽譜認識装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17930340

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17930340

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP