CN1142509C - Method of converting image information into music - Google Patents

Abstract

The present invention relates to a method for converting image information into music. The method has the following steps: firstly, various kinds of information of selected images are read to be respectively stored in data files; secondly, the data is processed by a principle of being faithful to original image information; then, the processed image data is converted into music symbols which are stored in an MIDI(musical notation format) text file; finally, the MIDI(musical notation format) text file is converted into MIDI(musical notation format) music to be outputted so as to make people perceive an image effect from auditory sensation when the people appreciate the images. The present invention helps the people to understand the relation between the images and the music by a '1/f' wave theory.

Description

A kind of method that image information is converted to music

(1) technical field:

The present invention a kind ofly converts image information the method for music to, belongs to signal processing technology.

(2) background technology:

In recent years, multimedia technology had obtained development at full speed, and increasing people begins to pay close attention to and be devoted to the research in this field, makes this field that flourishing vitality arranged.Though the present invention is to be research object with image in the multimedia and these two kinds of most important medium of sound, the applicant thinks that the present invention is the signal Processing field that belongs to relevant with the perception science, rather than multimedia technology field.Multimedia is the media of being convenient to the information that transmits that these information carriers of image, sound, literal and computer data are mixed.And the present invention is according to certain science, art rule, extracts the raw information of image, and the raw information with image is mapped to the music information space again, thereby makes people experience the pleasantly surprised of " listening figure " for the first time.Because the present invention is to the mapping relations in music information space and the exploration of mapping principle to the image information space.So its originality and novelty are obvious.The present patent application person do not have to find relevant therewith background technology after consulting a large amount of documents, but the inventor thinks that the knowwhy of the form (mainly be MIDI＜music format〉standard) of form by understanding image file, audio files, general music theory knowledge and signal Processing can have very great help to understanding the present invention.

The present invention is a kind of method that image information is converted to music, its objective is: under the prerequisite of faithful to original image information, image information is converted into music information, then music is come out, make the enough auditory perception visual effects of people's energy, the help people comprehend the inner link between image and the music: dull image contains dull melody, mixed and disorderly image contains mixed and disorderly melody, meet " 1/f fluctuation " beautiful harmonious image of rule and then must contain the melody of beautiful harmony, realize science and artistic perfect adaptation veritably.

(3) summary of the invention:

A kind ofly convert image information the method for music to, it is characterized in that: at first, the various information of image are deposited in separately the data file; Then these data are handled in line with the principle of faithful to image raw information; And then the view data after will handling is converted to music symbol and deposits in the MIDI file; At last the MIDI file is converted into the output of MIDI melody;

Now details are as follows with this method step:

One, reading images information:

Image adopts be 24 very color, with BMP document storage image information, begin to read the data of BMP file from end-of-file, be decimal data with the hexadecimal RGB grayvalue transition of being read, and deposit in respectively in three interim text data files;

Because what store in the BMP file is gray-scale value, thus utilize transformational relation between HSV model and the RGB model can obtain HSV three property values of color, and the HSV value is also deposited in respectively in three ephemeral data files;

Two, image information is handled:

Music is corresponding with the characteristic of image, it is the emphasis of whole proposal, the present invention takes image vertically to be principal direction, input data file is one of top six data files that produce, we are 14 parts with image division, its method is: view data is being divided on the basis of 14 parts, merging first earlier asks on average as first column data, then with 1,2 two parts merge on average as the secondary series data, again with 1,2,3 three parts merge on average as the 3rd column data, by that analogy, till all parts have all been merged, the result after handling is deposited in the data output file;

Three, view data is converted into the note text:

Comprise high, medium and low three ranges of sound in the music, each range of sound comprises the note of three octaves, and each note comprise its rise semitone, in, flats, adding up to total (7 * 3) to plant tone * 3 octaves * 3 ranges of sound=189 note can be by correspondence; Because the tone of regulation too much has 189 kinds, and to have much be repetition, so with reference to the keyboard of piano the number of tone is reduced to 88 here, then one group of complete sound group just adds that by " 1,2,3,4,5,6,7 " seven whole tones " #1, #2, #4, #5, #6 " five semitones form;

Rhythm is the key factor of music, by the playing time of each note being set the condition that produces rhythm to satisfy; Obtain 11 kinds of duration of a sound through statistics, replace with 11 kinds of symbols; Difference with adjacent two elements of every column data is a research object, finds out maximum, the minimum value of every capable difference, and to correspond to the changing value of data on 11 kinds of duration of a sound be rhythm tempo;

Because the rhythm and the duration of a sound and beat have very confidential relation, so beat employing is in the present invention determined the algorithm that the duration of a sound carries out statistic of classification;

For the pairing musical instrument of passage, by midi standard 128 kinds of musical instrument sounds are divided into 16 groups according to the musical instrument kind, every group has defined 8 kinds of musical instrument sounds; MIDI is defined in 16 passages, and except that 11,12 for not with the passage, remaining 14 passage all will be assigned to the musical instrument sound that compositor can produce, musical instrument picked at random in corresponding kind that each passage reality is adopted;

Per minute beat number, the variation speed of promptly playing speed and every column data also has much relations; Adjacent data by adding up all data rows poor found out maximum difference, the difference that minimal difference and number are maximum;

In order to make the MIDI file of compliant, regulation MIDI information text is made form;

Consider people's ability to accept, transfer so artificial unified regulation tone mark is C that beat 8/8 is clapped, per minute beat number is 180;

Four, from " note " file, receive data after, it is translated into the MIDI document code, then according to the regulation of MIDI file, add file header and track head, writing in the melody file, form the MIDI file, can be the MIDI melody with " note " file conversion like this.

A kind of method that image information is converted to music, it is characterized in that: better understand getting in touch between image and the music in order to help people, " " wave theory as can be known: it also is dull that dull image is converted into music to 1/f, and it also is melodious that harmonious image is converted into music in its utilization.

The present invention is a kind of to convert image information the method for music to, and its beneficial effect is: the invention belongs to new ideas, new design, new exploration, is the mapping relations of setting up between a kind of image and the music.Have application prospect, it not only can state works with showpiece or the U.S.A such as education of science facilities such as science and technology centers, converts the form of expression of music to, allows people experience the visual effect of scientific and technological showpiece and work of fine arts with the sense of hearing; Can also provide thinking to the computer composing expert system; And the melody that Protean beautiful image transitions becomes can evoke composer's creation inspiration; Moreover, the image encryption of the encrypted music of Image Intelligence and the music information good cryptographic means of all can yet be regarded as.

(4) description of drawings:

Fig. 1 the inventive method step block scheme.

Fig. 2 hardware configuration block diagram of the present invention.

(5) embodiment:

The present invention is a kind of method that image information is converted to music, the steps include: one, at first, read selected digital image various information (red as the R of three primary colors RGB＜wherein representative, the G representative is green, B represent orchid gray-scale value, face three attributes of color HSV＜wherein on behalf of tone, S, H represent saturation degree, V to represent lightness), they are deposited in respectively in separately the data file; Two, in line with the principle of faithful to image raw information these data are handled; Three, the view data after will handling is converted to music symbol and deposits in MIDI (note form) text; Four, at last MIDI (note form) text is converted into the output of MIDI (note form) melody;

Now details are as follows with the concrete grammar step:

One, reading images information:

What image adopted is 24 very coloured silks, file layout is deposited image information for the BMP file layout, the process of depositing is that computing machine reads in the BMP file header earlier, by file mark " BM ", judge whether it is the BMP file, again by the color zone bit, judge whether to be 24 rgb images, then read in the length and width data of file and the side-play amount of image data area, the length and width of documents data are deposited in the file, begin because the BMP image file is last column by image, from the bottom to top, carry out data storage from left to right, so we according to length and width value, begin to read the data of BMP file utilizing after side-play amount finds image data area from end-of-file, with hexadecimal RGB (the red green orchid) grayvalue transition of being read is decimal data, and deposits in respectively in three interim text data files;

Because what store in the BMP file is gray-scale value, so utilize the relation between HSV (wherein: on behalf of tone, S, H represent saturation degree, V to represent lightness) model and RGB (the red green orchid) model can obtain face three attributes of color (H, S, V) value, and three attributes (H, S, V) value also deposited in respectively in three ephemeral data files;

Two, image information is handled:

Music is corresponding with the characteristic of image, it is the emphasis of whole proposal, because base class medium when music is, note must could be formed melody by a time duration direction, and the custom of most of people when appreciating picture is to browse from top to down, so the present invention takes image vertical (from top to bottom) for handling principal direction (being the music working direction), input data file is one of top six data files that produce, and is that example describes with tone (H) value below; Consider the transitionality of image each several part tone and the ambiguity of tone transition, be the correlativity between the adjacent image point and the uncertainty of image boundary, and 14 available standard MIDI (form of music) passage, we cut apart view data and extract, with image division is 14 parts: view data is being divided on the basis of 14 parts, merging first earlier asks on average as first column data, then with 1,2 two parts merge on average as the secondary series data, again with 1,2,3 three parts merge on average as the 3rd column data, by that analogy, till all parts have all been merged, the result after handling is deposited in the data output file;

Three, view data is converted into the note text:

Comprise high, medium and low three ranges of sound in music, each range of sound comprises the note of three octaves, and each note comprise its rise semitone, in, flats, adding up to total (7 * 3) to plant tone * 3 octaves * 3 ranges of sound=189 note can be by correspondence; At first, regulation is represented seven musical alphabets of bass area with " c, d, e, f, g, a, b ", seven musical alphabets with " 1,2,3,4,5,6,7 " expression middle register, seven musical alphabets with " C, D, E, F, G, A, B " expression treble, "/" expression before musical alphabet rises octave, " octave falls in \ " expression, and blank is then represented not rise also not fall; Immediately following " b " behind musical alphabet expression flats, " # " expression rises semitone, because the tone of regulation is (189 kinds) too much, and to have much be repetition, so with reference to the keyboard of piano the number of tone is reduced to 88 here, still be high, medium and low three ranges of sound, be divided into nine sound groups such as two groups of big words, one group of big word, big word group, small character group, one group of small character, two groups of small characters, three groups of small characters, four groups of small characters, five groups of small characters from low to high; Except two groups of big words only contain three kinds of tones and small character only contains a kind of tone for five groups, all the other seven groups all contain seven basic announcements (the white key of piano) and five semitone 12 kinds of tones such as (black keys of piano); Again since some sound as " c ", its flats " bd " that rise the sound " d " that semitone " #c " is adjacent are identical; Stipulate that all represent with the semitone that rises of previous sound so we are unified; Then one group of complete sound group just adds that by " 1,2,3,4,5,6,7 " seven whole tones " #1, #2, #4, #5, #6 " five semitones form;

Rhythm is the key factor in the music, and it relates to all factors relevant with " time ", is sound Protean tissue morphology aspect strong and weak and long and short two, and we are by setting the condition that produces rhythm to satisfy to the playing time of each note; Through 11 kinds of duration of a sound that statistics obtains, we replace with 11 kinds of symbols: "-" expression minim (2 clap), ". " expression dotted note (

Clap), " " expression crotchet (1 claps), " _. " dot of expression quaver band (

Clap), " _ " expression quaver ( Clap), "=. " dot of expression semiquaver band ( Clap), "=" expression semiquaver ( Clap), " " dot of expression demisemiquaver band (

Clap), ": " expression demisemiquaver (

Clap), " dot of expression hemidemisemiquaver band (

Clap), "; " the expression hemidemisemiquaver (

Clap).So we are research object with the difference of adjacent two elements of every column data, find out maximum, the minimum value of every capable difference, correspond on 11 kinds of duration of a sound (rhythm tempo) by the changing value of following formula tone data; $S=\frac{\mathrm{Gl}-\mathrm{ml}}{\mathrm{Ml}-\mathrm{ml}}\×10(\mathrm{Ml}\≠\mathrm{ml}\≠0)$

S＝0 (Ml＝ml)

S-rhythm tempo (Speed), Gl-tone value difference

The maximal value of Ml-tone value difference, the minimum value of ml-tone value difference

Because the rhythm and the duration of a sound and beat have very confidential relation, so beat employing is in the present invention determined the algorithm that the duration of a sound carries out statistic of classification; The note of all band dots is included into its main classes, can be included into minim as the note of being with dot in two minutes and add up; Choose the denominator of the duration of a sound of the maximum the sort of representative of quantity as beat, even the quantity of minim is maximum, and then the denominator of beat is 2; Here molecule temporarily uses the non-zero random number less than denominator to replace;

For the pairing musical instrument of passage, we have adopted some standards of MIDI (note form).General MIDI (note form) standard is divided into 16 groups with 128 kinds of musical instruments according to the musical instrument kind, and every group has defined 8 kinds of musical instrument sounds; These 16 groups of musical instruments are respectively: piano musical instrument, half-dozen percussion instrument, organ musical instrument, guitar musical instrument, bass, stringed musical instrument, (instrumental) ensemble musical instrument, brass instrument, spring musical instrument, wind instrument musical instrument, syntheticly induce, synthetic voiceless sound, synthetic effect, national musical instruments, percussion instrument, sound effect etc.; MIDI (note form) is defined in 16 passages, except that 11,12 for not with the passage, remaining 14 passage all will be assigned to the musical instrument sound that compositor can produce, if grow up to be a useful person as elementary sum, then give three kinds of orchestra sound with the 13-15 channel assigning, and allow to produce 6 kinds of notes simultaneously, and give percussion instrument sound with 16 channel assignings, allow to produce simultaneously 3 kinds of notes; For extend synthesizer, then give 9 kinds of orchestra sound the 1-9 channel assigning, can allow to produce 16 notes simultaneously; 10 passages are used for percussion instrument sound, allow to produce simultaneously 16 notes, musical instrument picked at random in corresponding kind that each passage reality is adopted;

Per minute beat number, the variation speed of promptly playing speed and every column data also has much relations; So we add up adjacent data poor of all data rows, find out maximum difference (MaxSpeed), the difference (Flag) that minimal difference (MinSpeed) and number are maximum; Adopt following formula to change:

$S=\frac{(F-\mathrm{MinS})\×(200-120)}{\mathrm{MaxS}-\mathrm{MinS}}+120$

S-rhythm tempo (Speed), the difference that the F-number is maximum (Flag)

The difference of MaxS-maximum (MaxSpeed), the difference of MinS-minimum (MinSpeed)

In order to make the MIDI file of compliant, it is as follows to make form at this regulation MIDI information text:

[MIDI]

＜tone mark 〉,＜beat 〉,＜per minute beat number〉and,＜track number 〉

[1]

Pn

............

[n]

Pn

... ... illustrate:

Tone mark: take a character, be necessary for A, B, C, D, E, F, G, transfer otherwise be considered as C;

Beat: value is as follows, and 2/4,3/4,4/4,3/8,6/8

Per minute beat number: represent the beat sum that every public clock is played, value otherwise is considered as 120 between 40-200;

The track numberRepresent this song Part.As three parts, can it be set to 3.Correspondingly, n=3;

[n] back be the music information of expression track, transform good music symbol above being;

Pn: expression is provided with tone color, and value is between 0-127;

Other character is considered as unallowable instruction digit;

Consider people's ability to accept, artificial unified regulation tone mark is that C transfers, and beat 8/8 is clapped, and per minute beat number is 180; The music symbol that conversion is good deposits in interim MIDI (note form) text " MIDIout.txt ";

Four, with " note " file conversion be MIDI (note form) melody:

Because MIDI (note form) file comprises a piece and track piece two parts; Its form is generally as follows:

MIDI (note form) file header structure struct MH char Midild[4]; MIDI file mark MThd long length; Block structure message length int format; The form int ntracks that deposits; Track number int PerPaiNum; Every joint counter value }; Track header structure struct TH char Trackld[4]; Track flag MTrk long length; Message length }

After from " note " file, receiving data, it is translated into MIDI (note form) document code, according to the regulation of MIDI (note form) file, add file header and track head then, write in the melody file, form MIDI (note form) file;

The present invention is except above-mentioned software section, and necessary image input, audio output device are absolutely necessary.

In order to help people to understand getting in touch between image and the music, " " wave theory as can be known: it also is dull that dull image is converted into music to 1/f, and it also is melodious that harmonious image is converted into music in its utilization.

Its design philosophy mainly is the Changing Pattern that meets " 1/f fluctuation (or claiming " 1/f variation ", " 1/f noise ") ", and image information is transformed into musical composition;

Since ancient times, all things on earth in the Nature just is among the organic whole, and anything all is among continuous contact and the continuous development, has so just formed the Nature of perfect harmony.In the Nature, the power spectrum density and the frequency of ubiquitous fluctuation (noise) are inversely proportional on logarithmic coordinate, and we are called such fluctuation " 1/f fluctuation (or claiming " 1/f variation ", " 1/f noise ") "." 1/f fluctuates (noise) " in the universe has the vital role of maintaining overall situation balance harmony.So when the mankind are washed in the Nature, will feel that body and mind loosens.External scientists is through discovering, why " 1/f fluctuation " make us feeling comfortable, be because of the people when peace and quiet the α E.E.G and the fluctuation pattern of heart beat cycle match with " 1/f fluctuation " rule well.

Image and music are a pair of twin sisters always, original and different artist sketches the contours of the beautiful picture scroll that a width of cloth width of cloth meets " 1/f fluctuation " rule with magical paintbrush before people, from the emotion that visually stimulates people, people are being drunk among the ink and painting.Simultaneously, different picture scrolls also can excite the different imagination of people.For example, as if when appreciating the picture scroll of " Pentium the Yellow River ", people can hear the growl of the Yellow River; When appreciating the picture scroll of " mountain forest song in morning ", people have seemed to hear the sound of singing that bird is narrated in a specious manner that moves one to listen in the chatter, woods in intermountain brook again.

The present invention just is based on above-mentioned idea proposition: the information that reads from image is converted into music then and it is played back, and makes people from acoustically experiencing the effect of vision, realizes science and artistic perfect adaptation veritably.The present invention mainly is divided into four parts as shown in Figure 1.At first, read the various information (as the gray-scale value of three primary colors RGB, face three attributes of color HSV etc.) of selected digital image, they are deposited in respectively in separately the data file; In line with the principle of faithful to image raw information these data are handled then; View data after handling is converted to music symbol and deposits in the MIDI text; At last the MIDI text is converted into the MIDI melody.

Claims (2)

1, a kind ofly converts image information the method for music to, it is characterized in that: at first, the various information of image are deposited in separately the data file; Then these data are handled in line with the principle of faithful to image raw information; And then the view data after will handling is converted to music symbol and deposits in the MIDI file; At last the MIDI file is converted into the output of MIDI melody;

Now details are as follows with this method step:

One, reading images information:

Image adopts be 24 very color, with BMP document storage image information, begin to read the data of BMP file from end-of-file, be decimal data with the hexadecimal RGB grayvalue transition of being read, and deposit in respectively in three interim text data files;

Because what store in the BMP file is gray-scale value, thus utilize transformational relation between HSV model and the RGB model can obtain HSV three property values of color, and the HSV value is also deposited in respectively in three ephemeral data files;

Two, image information is handled:

Music is corresponding with the characteristic of image, it is the emphasis of whole proposal, the present invention takes image vertically to be principal direction, input data file is one of top six data files that produce, we are 14 parts with image division, its method is: view data is being divided on the basis of 14 parts, merging first earlier asks on average as first column data, then with 1,2 two parts merge on average as the secondary series data, again with 1,2,3 three parts merge on average as the 3rd column data, by that analogy, till all parts have all been merged, the result after handling is deposited in the data output file;

Three, view data is converted into the note text:

Comprise high, medium and low three ranges of sound in the music, each range of sound comprises the note of three octaves, and each note comprise its rise semitone, in, flats, adding up to total (7 * 3) to plant tone * 3 octaves * 3 ranges of sound=189 note can be by correspondence; Because the tone of regulation too much has 189 kinds, and to have much be repetition, so with reference to the keyboard of piano the number of tone is reduced to 88 here, then one group of complete sound group just adds that by " 1,2,3,4,5,6,7 " seven whole tones " #1, #2, #4, #5, #6 " five semitones form;

Rhythm is the key factor of music, by the playing time of each note being set the condition that produces rhythm to satisfy; Obtain 11 kinds of duration of a sound through statistics, replace with 11 kinds of symbols; Difference with adjacent two elements of every column data is a research object, finds out maximum, the minimum value of every capable difference, and to correspond to the changing value of data on 11 kinds of duration of a sound be rhythm tempo;

Because the rhythm and the duration of a sound and beat have very confidential relation, so beat employing is in the present invention determined the algorithm that the duration of a sound carries out statistic of classification;

For the pairing musical instrument of passage, by midi standard 128 kinds of musical instrument sounds are divided into 16 groups according to the musical instrument kind, every group has defined 8 kinds of musical instrument sounds; MIDI is defined in 16 passages, and except that 11,12 for not with the passage, remaining 14 passage all will be assigned to the musical instrument sound that compositor can produce, musical instrument picked at random in corresponding kind that each passage reality is adopted;

Per minute beat number, the variation speed of promptly playing speed and every column data also has much relations; Adjacent data by adding up all data rows poor found out maximum difference, the difference that minimal difference and number are maximum;

In order to make the MIDI file of compliant, regulation MIDI information text is made form;

Consider people's ability to accept, transfer so artificial unified regulation tone mark is C that beat 8/8 is clapped, per minute beat number is 180;

Four, from " note " file, receive data after, it is translated into the MIDI document code, then according to the regulation of MIDI file, add file header and track head, writing in the melody file, form the MIDI file, can be the MIDI melody with " note " file conversion like this.

2, image information according to claim 1 converts the method for music to, it is characterized in that: understand getting in touch between image and the music in order to help people, " " wave theory as can be known: it also is dull that dull image is converted into music to 1/f, and it also is melodious that harmonious image is converted into music in its utilization.

Images