CN108492347A - Image generating method, device and computer readable storage medium - Google Patents
Image generating method, device and computer readable storage medium Download PDFInfo
- Publication number
- CN108492347A CN108492347A CN201810340384.XA CN201810340384A CN108492347A CN 108492347 A CN108492347 A CN 108492347A CN 201810340384 A CN201810340384 A CN 201810340384A CN 108492347 A CN108492347 A CN 108492347A
- Authority
- CN
- China
- Prior art keywords
- oscillogram
- segment
- design
- draw
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/20—Drawing from basic elements, e.g. lines or circles
- G06T11/203—Drawing of straight lines or curves
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
- G10L25/18—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters the extracted parameters being spectral information of each sub-band
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Signal Processing (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Computational Linguistics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Processing Or Creating Images (AREA)
Abstract
The present invention provides a kind of image generating method, device and computer readable storage medium, this method includes：The amplitude characteristic and frequecy characteristic of target audio are extracted, oscillogram is generated；Clustering processing is carried out to oscillogram using similarity algorithm, generates multiple segment oscillograms；According to the peak value and wavelength of segment oscillogram, generate that segment oscillogram is corresponding to draw a design；Coordinate system is established by origin of the corresponding center to draw a design of segment oscillogram, calculates the corresponding vector to draw a design of segment oscillogram；According to the corresponding vector to draw a design of segment oscillogram, corresponding draw a design of segment oscillogram is overlapped according to the time sequencing of segment oscillogram on painting canvas, generates the corresponding image of target audio.Audio can be combined with image by the above method, generation and the one-to-one image of audio greatly reduce the duration of Image Rendering, meet customization demand of the people to image artifacts.
Description
Technical field
The present invention relates to image generating technologies fields, and in particular to a kind of image generating method, device with it is computer-readable
Storage medium.
Background technology
Graphics art creation is artist using social life as source, life Experience accumulation, creating conception, artistic expression three
Creation inspiration after the big stage bursts out result.But time-consuming for this painting creation mode, paint can only art of expression man
Thought.And with the development of computer and artificial intelligence technology, sound how is combined, machine is allowed to carry out that there is " human thinking "
Painting creation, reduce the duration of painting creation, art technology become to the customization demand of paint to meet people
Personnel's urgent problem to be solved.
Invention content
The object of the present invention is to provide a kind of image generating method, device and computer readable storage mediums, can be by sound
Frequency is combined with image, and generation and the one-to-one image of audio greatly reduce the duration of Image Rendering, meet people to image
The customization demand of works.
An embodiment of the present invention provides a kind of image generating methods, including：
The amplitude characteristic and frequecy characteristic of target audio are extracted, oscillogram is generated；
Clustering processing is carried out to the oscillogram using similarity algorithm, generates multiple segment oscillograms；
According to the peak value and wavelength of the segment oscillogram, generate that the segment oscillogram is corresponding to draw a design；
Coordinate system is established as origin using the corresponding center to draw a design of the segment oscillogram, calculates the segment waveform
Scheme the corresponding vector to draw a design；
It, will according to the time sequencing of the segment oscillogram according to the corresponding vector to draw a design of the segment oscillogram
Corresponding draw a design of the segment oscillogram is overlapped on painting canvas, generates the corresponding image of the target audio.
Preferably, the peak value and wavelength according to the segment oscillogram, generates that the segment oscillogram is corresponding to paint
After pattern, further include：
Amplitude characteristic in the segment oscillogram is added up, it is cumulative to obtain the corresponding volume of the segment oscillogram
Value；
According to the corresponding volume accumulated value of the segment oscillogram and setting ratio, adjusts the segment oscillogram and correspond to
The area to draw a design.
Preferably, the peak value and wavelength according to the segment oscillogram, generates that the segment oscillogram is corresponding to paint
Pattern specifically includes：
The summation of all wavelengths in the segment oscillogram is calculated, and is generated using the summation of the wavelength as the circle of perimeter；
According to the segment oscillogram medium wavelength, the minimum point of the wave in the segment oscillogram is reflected according to direction initialization
It is mapped on the circumference of the circle, and the segment oscillogram is surrounded into the circle；
The peak point for connecting the segment oscillogram, generates that the segment oscillogram is corresponding to draw a design.
Preferably, described that coordinate system is established as origin using the corresponding center to draw a design of the segment oscillogram, it calculates
The corresponding vector to draw a design of the segment oscillogram, specifically includes：
Coordinate system is established using the corresponding center to draw a design of the segment oscillogram as origin, is drawn a design described in calculating
The corresponding coordinate of apex angle；
The corresponding coordinate of all apex angles in described draw a design is added up, the vector to draw a design described in calculating.
Preferably, described according to the corresponding vector to draw a design of the segment oscillogram, according to the segment oscillogram
Time sequencing corresponding draw a design of the segment oscillogram is overlapped on painting canvas, generate the target audio and correspond to
Image, specifically include：
Using the center of painting canvas coordinate system is established as origin；
It will be mapped on painting canvas positioned at corresponding draw a design of segment oscillogram of first time sequence, wherein position
It is Chong Die with the center of the painting canvas in the corresponding center to draw a design of the segment oscillogram of first time sequence；
According to positioned at the corresponding vector difference to draw a design of two of the adjacent time sequence segment oscillograms, position is calculated
In the corresponding position on the corresponding coordinate system of the painting canvas to draw a design of the segment oscillogram of next time sequencing
It sets, and corresponding draw a design of the segment oscillogram of next time sequencing is mapped on the position；
When corresponding draw a design of segment oscillogram positioned at next time sequencing draws a design for the last one
When, generate the corresponding image of the target audio.
Preferably, described image generation method further includes：
Extract the tamber characteristic of the segment oscillogram；
According to the preset filling table of comparisons, filling pattern corresponding with the tamber characteristic of segment oscillogram is obtained, and
In the filling pattern is filled into that the segment oscillogram is corresponding drawing a design.
Preferably, the filling pattern includes a variety of different color lumps of transparency or the texture of a variety of different materials.
The embodiment of the present invention additionally provides a kind of video generation device, including：
First waveform figure generation module, amplitude characteristic and frequecy characteristic for extracting target audio generate oscillogram；
Second oscillogram generation module, it is raw for carrying out Clustering processing to the oscillogram using similarity algorithm
At multiple segment oscillograms；
Pattern drafting module generates the segment oscillogram pair for the peak value and wavelength according to the segment oscillogram
That answers draws a design；
Vector calculation module, for establishing coordinate as origin using the corresponding center to draw a design of the segment oscillogram
System, calculates the corresponding vector to draw a design of the segment oscillogram；
Image Rendering module, for according to the corresponding vector to draw a design of the segment oscillogram, according to the segment
Corresponding draw a design of the segment oscillogram is overlapped by the time sequencing of oscillogram on painting canvas, generates the target sound
Frequently corresponding image.
The embodiment of the present invention additionally provides a kind of video generation device, including processor, memory and is stored in described
In memory and it is configured as the computer program executed by the processor, when the processor executes the computer program
Realize such as above-mentioned image generating method.
The embodiment of the present invention additionally provides a kind of computer readable storage medium, and the computer readable storage medium includes
The computer program of storage, wherein control when the computer program is run and set where the computer readable storage medium
It is standby to execute such as above-mentioned image generating method.
Compared with the existing technology, a kind of advantageous effect of image generating method provided in an embodiment of the present invention is：It is described
Image generating method, including：The amplitude characteristic and frequecy characteristic of target audio are extracted, oscillogram is generated；Using similarity algorithm
Clustering processing is carried out to the oscillogram, generates multiple segment oscillograms；According to the peak value and wave of the segment oscillogram
It is long, generate that the segment oscillogram is corresponding to draw a design；It is original with the corresponding center to draw a design of the segment oscillogram
Point establishes coordinate system, calculates the corresponding vector to draw a design of the segment oscillogram；It is corresponding according to the segment oscillogram
The vector to draw a design is drawing corresponding draw a design of the segment oscillogram according to the time sequencing of the segment oscillogram
It is overlapped on cloth, generates the corresponding image of the target audio.Audio can be combined with image, is generated by the above method
With the one-to-one image of audio, the duration of Image Rendering is greatly reduced, meets customization demand of the people to image artifacts.
The embodiment of the present invention additionally provides a kind of video generation device and computer readable storage medium.
Description of the drawings
Fig. 1 is a kind of flow chart of image generating method provided in an embodiment of the present invention；
Fig. 2 is a kind of schematic diagram of video generation device provided in an embodiment of the present invention.
Specific implementation mode
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation describes, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, those of ordinary skill in the art are obtained every other without creative efforts
Embodiment shall fall within the protection scope of the present invention.
Referring to Fig. 1, it is a kind of flow chart of image generating method provided in an embodiment of the present invention, described image generates
Method includes：
S100：The amplitude characteristic and frequecy characteristic of target audio are extracted, oscillogram is generated；
S200：Clustering processing is carried out to the oscillogram using similarity algorithm, generates multiple segment oscillograms；
S300：According to the peak value and wavelength of the segment oscillogram, generate that the segment oscillogram is corresponding to draw a design；
S400：Coordinate system is established by origin of the corresponding center to draw a design of the segment oscillogram, calculates described
The corresponding vector to draw a design of section oscillogram；
S500：According to the corresponding vector to draw a design of the segment oscillogram, according to the time of the segment oscillogram
Corresponding draw a design of the segment oscillogram is overlapped by sequence on painting canvas, generates the corresponding figure of the target audio
Picture.
In the present embodiment, the target audio can be the audio file extracted from local data base or from
Internet uploads to the audio file of the audio input port of setting.Use similarity algorithm by the target in the present embodiment
The corresponding oscillogram of audio is clustered, and similar one section of waveform in the oscillogram is divided into one group, is generated multiple described
Segment oscillogram；According to the wavelength and peak value of different fragments oscillogram, generate it is different draw a design, and be superimposed and paint on painting canvas
Pattern generates the corresponding image of the target audio；The above method can be combined target audio with image, generation and target
The corresponding image of audio, greatly reduces the duration of Image Rendering, meets customization demand of the people to image artifacts.
In a kind of optional embodiment, in S300 according to the peak value and wavelength of the segment oscillogram, described is generated
Section oscillogram is corresponding draw a design after, further include：
Amplitude characteristic in the segment oscillogram is added up, it is cumulative to obtain the corresponding volume of the segment oscillogram
Value；
According to the corresponding volume accumulated value of the segment oscillogram and setting ratio, adjusts the segment oscillogram and correspond to
The area to draw a design.
In the present embodiment, by being superimposed the amplitude of all waves in any one segment oscillogram, any one piece is calculated
The corresponding volume accumulated value of section oscillogram, such as 10db；And the setting ratio is 1db=5px；Then any one described segment
The size of oscillogram is 50px.Through this embodiment, the characteristic parameter of target audio, raising can greatly be utilized to be painted
The image of system it is rich.
In a kind of optional embodiment, S300：According to the peak value and wavelength of the segment oscillogram, the segment is generated
Oscillogram is corresponding to draw a design, and specifically includes：
The summation of all wavelengths in the segment oscillogram is calculated, and is generated using the summation of the wavelength as the circle of perimeter；
According to the segment oscillogram medium wavelength, the minimum point of the wave in the segment oscillogram is reflected according to direction initialization
It is mapped on the circumference of the circle, and the segment oscillogram is surrounded into the circle；
The peak point for connecting the segment oscillogram, generates that the segment oscillogram is corresponding to draw a design.
In the present embodiment, it is drawn and is justified as perimeter using the summation of all wavelengths of the segment oscillogram, and according to setting side
To the minimum point of the wave in the segment oscillogram is mapped on the circumference of the circle and surrounds the segment oscillogram
The circle, at this point, the segment oscillogram is far from the circle；Then the peak point of the segment oscillogram is connected with straight line, it is raw
It draws a design at the segment oscillogram is corresponding；The drawing image is polygon, and the direction initialization is clockwise
Or counterclockwise.Unique polygon that the corresponding segment oscillogram can be generated by the above method, realizes the segment
Conversion of the oscillogram to polygon pattern.
In a kind of optional embodiment, S400：Using the corresponding center to draw a design of the segment oscillogram as origin
Coordinate system is established, the corresponding vector to draw a design of the segment oscillogram is calculated, specifically includes：
Coordinate system is established using the corresponding center to draw a design of the segment oscillogram as origin, is drawn a design described in calculating
The corresponding coordinate of apex angle；
The corresponding coordinate of all apex angles in described draw a design is added up, the vector to draw a design described in calculating.
In the present embodiment, coordinate system is established as origin using the corresponding center to draw a design of the segment oscillogram, it can
With the corresponding coordinate of the apex angle to draw a design described in calculating, such as the apex angle coordinate for the A that draws a design is respectively a1(0,3.3), a2
(3.1,1), a3(2, -2.7), a4(- 2, -2.7), a5(- 3.1,1)；The apex angle coordinate of B of drawing a design is respectively b1(0,1.6), b2
(1.4,1), b3(1.5,-0.3)、b4(0.7,-1.4)、b5(-0.7,-1.4)、b6(- 1.5, -0.3), b7(-1.4,1)；Draw figure
The apex angle coordinate of case C is respectively c1(1.7,2.6), c2(3, -0.5), c3(0.6,-2.9)、c4(-2.7,-1.5)、c5(-2,-
2.4)；At this point, show that the vector for the A that draws a design is (0, -0.1) respectively by add operation, the vector of the B that draws a design be (0,
0.2), draw a design C vector be (0.6,4.7).
In a kind of optional embodiment, S500：According to the corresponding vector to draw a design of the segment oscillogram, according to
Corresponding draw a design of the segment oscillogram is overlapped by the time sequencing of the segment oscillogram on painting canvas, generates institute
The corresponding image of target audio is stated, is specifically included：
Using the center of painting canvas coordinate system is established as origin；
It will be mapped on painting canvas positioned at corresponding draw a design of segment oscillogram of first time sequence, wherein position
It is Chong Die with the center of the painting canvas in the corresponding center to draw a design of the segment oscillogram of first time sequence；
According to positioned at the corresponding vector difference to draw a design of two of the adjacent time sequence segment oscillograms, position is calculated
In the corresponding position on the corresponding coordinate system of the painting canvas to draw a design of the segment oscillogram of next time sequencing
It sets, and corresponding draw a design of the segment oscillogram of next time sequencing is mapped on the position；
When corresponding draw a design of segment oscillogram positioned at next time sequencing draws a design for the last one
When, generate the corresponding image of the target audio.
By taking the above-mentioned A that draws a design, the B that draws a design, the C that draws a design as an example, wherein the A that draws a design is located at the oscillogram
First time sequence, the B that draws a design be located at the second time sequencing of the oscillogram, the oscillogram described in C that draws a design the
Three time sequencings；Drawing image A is mapped to the center of painting canvas first so that in the center and painting canvas of drawing image A
The heart is overlapped；Vector difference (0-0,0.2- (- 0.2))=(0,0.3) of calculating and plotting image A and drawing image B, according to vector difference
(0,0.3) by drawing image B along vector difference (0,0.3) vector direction move 0.3 unit distance to the painting canvas correspondence
Position；The vector difference of calculating and plotting image C and drawing image B, drawing image C are moved using method identical with drawing image B
To the corresponding position of the painting canvas.The different location of the painting canvas can be moved to multiple drawing images by the above method, most
Throughout one's life at the corresponding image of the target audio.
In a kind of optional embodiment, described image generation method further includes：
Extract the tamber characteristic of the segment oscillogram；
According to the preset filling table of comparisons, filling pattern corresponding with the tamber characteristic of segment oscillogram is obtained, and
In the filling pattern is filled into that the segment oscillogram is corresponding drawing a design.
In a kind of optional embodiment, the filling pattern includes the color lump or a variety of different of a variety of different transparencies
The texture of material.
In the present embodiment, it can also will be obtained using the color lump of different transparencies or the texture of unlike material by corresponding
Multiple described draw a design be filled.The preset filling table of comparisons has recorded various different transparencies or unlike material
The correspondence of texture and different tone colors such as corresponds to 10%-90% from low to high respectively from thin sharp high pitch to simple and honest bass
Transparency, and material is then corresponded to from light and soft feather, velvet, is transitioned into using thick and heavy stone material, metal texture etc..By carrying
The tamber characteristic of target audio is taken, the rich of drawn image can be further increased.
Referring to Fig. 2, it is a kind of schematic diagram of video generation device provided in an embodiment of the present invention, described image generates
Device includes：
First waveform figure generation module 1, amplitude characteristic and frequecy characteristic for extracting target audio generate oscillogram；
Second oscillogram generation module 2, it is raw for carrying out Clustering processing to the oscillogram using similarity algorithm
At multiple segment oscillograms；
Pattern drafting module 3 generates the segment oscillogram pair for the peak value and wavelength according to the segment oscillogram
That answers draws a design；
Vector calculation module 4, for establishing coordinate as origin using the corresponding center to draw a design of the segment oscillogram
System, calculates the corresponding vector to draw a design of the segment oscillogram；
Image Rendering module 5, for according to the corresponding vector to draw a design of the segment oscillogram, according to the segment
Corresponding draw a design of the segment oscillogram is overlapped by the time sequencing of oscillogram on painting canvas, generates the target sound
Frequently corresponding image.
In the present embodiment, the target audio can be the audio file extracted from local data base or from
Internet uploads to the audio file of the audio input port of setting.Use similarity algorithm by the target in the present embodiment
The corresponding oscillogram of audio is clustered, and similar one section of waveform in the oscillogram is divided into one group, is generated multiple described
Segment oscillogram；According to the wavelength and peak value of different fragments oscillogram, generate it is different draw a design, and be superimposed and paint on painting canvas
Pattern generates the corresponding image of the target audio；Above-mentioned apparatus can be combined target audio with image, generation and target
The corresponding image of audio, greatly reduces the duration of Image Rendering, meets customization demand of the people to image artifacts.
In a kind of optional embodiment, described image generating means further include：
Volume computing module obtains the segment wave for the amplitude characteristic in the segment oscillogram to add up
The corresponding volume accumulated value of shape figure；
Pattern area adjusts module, is used for according to the corresponding volume accumulated value of the segment oscillogram and setting ratio,
Adjust the corresponding area to draw a design of the segment oscillogram.
In the present embodiment, by being superimposed the amplitude of all waves in any one segment oscillogram, any one piece is calculated
The corresponding volume accumulated value of section oscillogram, such as 10db；And the setting ratio is 1db=5px；Then any one described segment
The size of oscillogram is 50px.Through this embodiment, the characteristic parameter of target audio, raising can greatly be utilized to be painted
The image of system it is rich.
In a kind of optional embodiment, pattern drafting module 3 includes：
Circle generation unit, the summation for calculating all wavelengths in the segment oscillogram, and generate with the wavelength
Summation is the circle of perimeter；
Oscillogram deformation unit, for according to the segment oscillogram medium wavelength, according to direction initialization by the segment wave
The minimum point of wave in shape figure is mapped on the circumference of the circle, and the segment oscillogram is surrounded the circle；
Point line unit, the peak point for connecting the segment oscillogram generate that the segment oscillogram is corresponding to paint
Pattern.
In the present embodiment, it is drawn and is justified as perimeter using the summation of all wavelengths of the segment oscillogram, and according to setting side
To the minimum point of the wave in the segment oscillogram is mapped on the circumference of the circle and surrounds the segment oscillogram
The circle, at this point, the segment oscillogram is far from the circle；Then the peak point of the segment oscillogram is connected with straight line, it is raw
It draws a design at the segment oscillogram is corresponding；The drawing image is polygon, and the direction initialization is clockwise
Or counterclockwise.Unique polygon that the corresponding segment oscillogram can be generated by above-mentioned apparatus, realizes the segment
Conversion of the oscillogram to polygon pattern.
In a kind of optional embodiment, vector calculation module 4 includes：
First coordinate calculating unit is sat for being established as origin using the corresponding center to draw a design of the segment oscillogram
Mark is the corresponding coordinate of apex angle to draw a design described in calculating；
Vector calculation unit, for the corresponding coordinate of all apex angles in described draw a design to add up, described in calculating
The vector to draw a design.
In the present embodiment, coordinate system is established as origin using the corresponding center to draw a design of the segment oscillogram, it can
With the corresponding coordinate of the apex angle to draw a design described in calculating, such as the apex angle coordinate for the A that draws a design is respectively a1(0,3.3), a2
(3.1,1), a3(2, -2.7), a4(- 2, -2.7), a5(- 3.1,1)；The apex angle coordinate of B of drawing a design is respectively b1(0,1.6), b2
(1.4,1), b3(1.5,-0.3)、b4(0.7,-1.4)、b5(-0.7,-1.4)、b6(- 1.5, -0.3), b7(-1.4,1)；Draw figure
The apex angle coordinate of case C is respectively c1(1.7,2.6), c2(3, -0.5), c3(0.6,-2.9)、c4(-2.7,-1.5)、c5(-2,-
2.4)；At this point, show that the vector for the A that draws a design is (0, -0.1) respectively by add operation, the vector of the B that draws a design be (0,
0.2), draw a design C vector be (0.6,4.7).
In a kind of optional embodiment, Image Rendering module 5 includes：
Canvas coordinate system establishes unit, for establishing coordinate system as origin using the center of painting canvas；
First map unit, the corresponding mapping that draws a design of the segment oscillogram for first time sequence will to be located at
Onto painting canvas, wherein positioned at the corresponding center to draw a design of the segment oscillogram of first time sequence and the painting canvas
Center overlapping；
Second map unit, for scheming according to being located at corresponding draftings of two of the adjacent time sequence segment oscillograms
The vector difference of case, calculate positioned at next time sequencing the segment oscillogram it is corresponding draw a design in the painting canvas pair
The position on coordinate system answered, and by the segment oscillogram of next time sequencing it is corresponding draw a design be mapped to it is described
On position；
Image generation unit, for when positioned at next time sequencing the segment oscillogram it is corresponding draw a design for
When the last one draws a design, the corresponding image of the target audio is generated.
By taking the above-mentioned A that draws a design, the B that draws a design, the C that draws a design as an example, wherein the A that draws a design is located at the oscillogram
First time sequence, the B that draws a design be located at the second time sequencing of the oscillogram, the oscillogram described in C that draws a design the
Three time sequencings；Drawing image A is mapped to the center of painting canvas first so that in the center and painting canvas of drawing image A
The heart is overlapped；Vector difference (0-0,0.2- (- 0.2))=(0,0.3) of calculating and plotting image A and drawing image B, according to vector difference
(0,0.3) by drawing image B along vector difference (0,0.3) vector direction move 0.3 unit distance to the painting canvas correspondence
Position；The vector difference of calculating and plotting image C and drawing image B, drawing image C are moved using device identical with drawing image B
To the corresponding position of the painting canvas.The different location of the painting canvas can be moved to multiple drawing images by above-mentioned apparatus, most
Throughout one's life at the corresponding image of the target audio.
In a kind of optional embodiment, described image generating means further include：
Tone color extraction module, the tamber characteristic for extracting the segment oscillogram；
Module is filled, for according to the preset filling table of comparisons, obtaining corresponding with the tamber characteristic of segment oscillogram
Filling pattern, and the filling pattern be filled into that the segment oscillogram is corresponding draw a design.
In a kind of optional embodiment, the filling pattern includes the color lump or a variety of different of a variety of different transparencies
The texture of material.
In the present embodiment, it can also will be obtained using the color lump of different transparencies or the texture of unlike material by corresponding
Multiple described draw a design be filled.The preset filling table of comparisons has recorded various different transparencies or unlike material
The correspondence of texture and different tone colors such as corresponds to 10%-90% from low to high respectively from thin sharp high pitch to simple and honest bass
Transparency, and material is then corresponded to from light and soft feather, velvet, is transitioned into using thick and heavy stone material, metal texture etc..By carrying
The tamber characteristic of target audio is taken, the rich of drawn image can be further increased.
The embodiment of the present invention additionally provides a kind of video generation device, including processor, memory and is stored in described
In memory and it is configured as the computer program executed by the processor, when the processor executes the computer program
Realize such as above-mentioned image generating method.
Illustratively, the computer program can be divided into one or more module/units, one or more
A module/unit is stored in the memory, and is executed by the processor, to complete the present invention.It is one or more
A module/unit can be the series of computation machine program instruction section that can complete specific function, and the instruction segment is for describing institute
State implementation procedure of the computer program in described image generating means.For example, the computer program can be divided into as
First waveform figure generation module 1 shown in Fig. 2, the second oscillogram generation module 2, pattern drafting module 3, vector calculation module 4,
Image Rendering module 5, each module concrete function are as follows：First waveform figure generation module 1, the amplitude for extracting target audio are special
It seeks peace frequecy characteristic, generates oscillogram；Second oscillogram generation module 2, for using similarity algorithm to the oscillogram into
The processing of row Clustering, generates multiple segment oscillograms；Pattern drafting module 3, for the peak value according to the segment oscillogram
And wavelength, generate that the segment oscillogram is corresponding to draw a design；Vector calculation module 4, for the segment oscillogram pair
The center to draw a design answered is that origin establishes coordinate system, calculates the corresponding vector to draw a design of the segment oscillogram；Figure
As drafting module 5, for according to the corresponding vector to draw a design of the segment oscillogram, according to the segment oscillogram when
Between sequence corresponding draw a design of the segment oscillogram is overlapped on painting canvas, generate the corresponding figure of the target audio
Picture..
Described image generating means can be the calculating such as desktop PC, notebook, palm PC and cloud server
Equipment.Described image generating means may include, but be not limited only to, processor, memory.It will be understood by those skilled in the art that
The schematic diagram is only the example of video generation device, does not constitute the restriction to video generation device, may include than figure
Show more or fewer components, either combine certain components or different components, such as described image generating means can be with
Including input-output equipment, network access equipment, bus etc..
Alleged processor can be central processing unit (Central Processing Unit, CPU), can also be it
His general processor, digital signal processor (Digital Signal Processor, DSP), application-specific integrated circuit
(Application Specific Integrated Circuit, ASIC), ready-made programmable gate array (Field-
Programmable Gate Array, FPGA) either other programmable logic device, discrete gate or transistor logic,
Discrete hardware components etc..General processor can be microprocessor or the processor can also be any conventional processor
Deng the processor is the control centre of described image generating means, and various interfaces and connection whole image is utilized to generate
The various pieces of device.
The memory can be used for storing the computer program and/or module, and the processor is by running or executing
Computer program in the memory and/or module are stored, and calls the data being stored in memory, described in realization
The various functions of video generation device.The memory can include mainly storing program area and storage data field, wherein storage journey
It sequence area can storage program area, application program (such as sound-playing function, image player function needed at least one function
Deng) etc.；Storage data field can be stored uses created data (such as audio data, phone directory etc.) etc. according to mobile phone.This
Outside, memory may include high-speed random access memory, can also include nonvolatile memory, such as hard disk, memory, insert
Connect formula hard disk, intelligent memory card (Smart Media Card, SMC), secure digital (Secure Digital, SD) card, flash memory
Block (Flash Card), at least one disk memory, flush memory device or other volatile solid-state parts.
Wherein, if module/unit that described image generating means integrate is realized in the form of SFU software functional unit and makees
It is independent product sale or in use, can be stored in a computer read/write memory medium.Based on this understanding,
The present invention realizes all or part of flow in above-described embodiment method, can also be instructed by computer program relevant hard
Part is completed, and the computer program can be stored in a computer readable storage medium, which is being handled
When device executes, it can be achieved that the step of above-mentioned each embodiment of the method.Wherein, the computer program includes computer program generation
Code, the computer program code can be source code form, object identification code form, executable file or certain intermediate forms
Deng.The computer-readable medium may include：Any entity or device, record of the computer program code can be carried
Medium, USB flash disk, mobile hard disk, magnetic disc, CD, computer storage, read-only memory (ROM, Read-Only Memory), with
Machine accesses memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium etc..
It should be noted that the content that the computer-readable medium includes can be according to legislation and patent practice in jurisdiction
It is required that carrying out increase and decrease appropriate, such as in certain jurisdictions, do not wrapped according to legislation and patent practice, computer-readable medium
Include electric carrier signal and telecommunication signal.
It should be noted that the apparatus embodiments described above are merely exemplary, wherein described be used as separating component
The unit of explanation may or may not be physically separated, and the component shown as unit can be or can also
It is not physical unit, you can be located at a place, or may be distributed over multiple network units.It can be according to actual
It needs that some or all of module therein is selected to achieve the purpose of the solution of this embodiment.In addition, device provided by the invention
In embodiment attached drawing, the connection relation between module indicates there is communication connection between them, specifically can be implemented as one or
A plurality of communication bus or signal wire.Those of ordinary skill in the art are without creative efforts, you can to understand
And implement.
The embodiment of the present invention additionally provides a kind of computer readable storage medium, and the computer readable storage medium includes
The computer program of storage, wherein control when the computer program is run and set where the computer readable storage medium
It is standby to execute such as above-mentioned image generating method.
Compared with the existing technology, a kind of advantageous effect of image generating method provided in an embodiment of the present invention is：It is described
Image generating method, including：The amplitude characteristic and frequecy characteristic of target audio are extracted, oscillogram is generated；Using similarity algorithm
Clustering processing is carried out to the oscillogram, generates multiple segment oscillograms；According to the peak value and wave of the segment oscillogram
It is long, generate that the segment oscillogram is corresponding to draw a design；It is original with the corresponding center to draw a design of the segment oscillogram
Point establishes coordinate system, calculates the corresponding vector to draw a design of the segment oscillogram；It is corresponding according to the segment oscillogram
The vector to draw a design is drawing corresponding draw a design of the segment oscillogram according to the time sequencing of the segment oscillogram
It is overlapped on cloth, generates the corresponding image of the target audio.Audio can be combined with image, is generated by the above method
With the one-to-one image of audio, the duration of Image Rendering is greatly reduced, meets customization demand of the people to image artifacts.
The embodiment of the present invention additionally provides a kind of video generation device and computer readable storage medium.
The above is the preferred embodiment of the present invention, it is noted that for those skilled in the art
For, various improvements and modifications may be made without departing from the principle of the present invention, these improvements and modifications are also considered as
Protection scope of the present invention.
Claims (10)
1. a kind of image generating method, which is characterized in that including：
The amplitude characteristic and frequecy characteristic of target audio are extracted, oscillogram is generated；
Clustering processing is carried out to the oscillogram using similarity algorithm, generates multiple segment oscillograms；
According to the peak value and wavelength of the segment oscillogram, generate that the segment oscillogram is corresponding to draw a design；
Coordinate system is established as origin using the corresponding center to draw a design of the segment oscillogram, calculates the segment oscillogram pair
The vector to draw a design answered；
It, will be described according to the time sequencing of the segment oscillogram according to the corresponding vector to draw a design of the segment oscillogram
Corresponding draw a design of segment oscillogram is overlapped on painting canvas, generates the corresponding image of the target audio.
2. image generating method as described in claim 1, which is characterized in that the peak value according to the segment oscillogram and
Wavelength further includes after generating that the segment oscillogram is corresponding and drawing a design：
Amplitude characteristic in the segment oscillogram is added up, the corresponding volume accumulated value of the segment oscillogram is obtained；
According to the corresponding volume accumulated value of the segment oscillogram and setting ratio, adjust that the segment oscillogram is corresponding to paint
The area of pattern.
3. image generating method as described in claim 1, which is characterized in that the peak value according to the segment oscillogram and
Wavelength, generates that the segment oscillogram is corresponding to draw a design, and specifically includes：
The summation of all wavelengths in the segment oscillogram is calculated, and is generated using the summation of the wavelength as the circle of perimeter；
According to the segment oscillogram medium wavelength, the minimum point of the wave in the segment oscillogram is mapped to according to direction initialization
On the circumference of the circle, and the segment oscillogram is surrounded into the circle；
The peak point for connecting the segment oscillogram, generates that the segment oscillogram is corresponding to draw a design.
4. image generating method as claimed in claim 3, which is characterized in that described with the corresponding drafting of the segment oscillogram
The center of pattern is that origin establishes coordinate system, calculates the corresponding vector to draw a design of the segment oscillogram, specifically includes：
Coordinate system, the top to draw a design described in calculating are established using the corresponding center to draw a design of the segment oscillogram as origin
The corresponding coordinate in angle；
The corresponding coordinate of all apex angles in described draw a design is added up, the vector to draw a design described in calculating.
5. image generating method as claimed in claim 4, which is characterized in that described to be painted according to the segment oscillogram is corresponding
The vector of pattern draws a design in painting canvas according to the time sequencing of the segment oscillogram by the segment oscillogram is corresponding
On be overlapped, generate the corresponding image of the target audio, specifically include：
Using the center of painting canvas coordinate system is established as origin；
It will be mapped on painting canvas positioned at corresponding draw a design of segment oscillogram of first time sequence, wherein be located at the
The corresponding center to draw a design of the segment oscillogram of one time sequencing is Chong Die with the center of the painting canvas；
According to positioned at the corresponding vector difference to draw a design of two of the adjacent time sequence segment oscillograms, under calculating is located at
The corresponding position on the corresponding coordinate system of the painting canvas to draw a design of the segment oscillogram of one time sequencing, and
Corresponding draw a design of the segment oscillogram of next time sequencing is mapped on the position；
When the segment oscillogram positioned at next time sequencing is corresponding draw a design draw a design for the last one when, it is raw
At the corresponding image of the target audio.
6. image generating method as described in claim 1, which is characterized in that described image generation method further includes：
Extract the tamber characteristic of the segment oscillogram；
According to the preset filling table of comparisons, filling pattern corresponding with the tamber characteristic of segment oscillogram is obtained, and by institute
It states during filling pattern is filled into that the segment oscillogram is corresponding and draws a design.
7. image generating method as claimed in claim 6, which is characterized in that the filling pattern includes a variety of different transparencies
Color lump or a variety of different materials texture.
8. a kind of video generation device, which is characterized in that including：
First waveform figure generation module, amplitude characteristic and frequecy characteristic for extracting target audio generate oscillogram；
Second oscillogram generation module generates more for carrying out Clustering processing to the oscillogram using similarity algorithm
A segment oscillogram；
It is corresponding to generate the segment oscillogram for the peak value and wavelength according to the segment oscillogram for pattern drafting module
It draws a design；
Vector calculation module is counted for establishing coordinate system as origin using the corresponding center to draw a design of the segment oscillogram
Calculate the corresponding vector to draw a design of the segment oscillogram；
Image Rendering module, for according to the corresponding vector to draw a design of the segment oscillogram, according to the segment waveform
Corresponding draw a design of the segment oscillogram is overlapped by the time sequencing of figure on painting canvas, generates the target audio pair
The image answered.
9. a kind of video generation device, which is characterized in that including processor, memory and be stored in the memory and by
It is configured to the computer program executed by the processor, is realized when the processor executes the computer program as right is wanted
Seek the image generating method described in any one of 1 to 7.
10. a kind of computer readable storage medium, which is characterized in that the computer readable storage medium includes the calculating of storage
Machine program, wherein equipment where controlling the computer readable storage medium when the computer program is run is executed as weighed
Profit requires the image generating method described in any one of 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810340384.XA CN108492347B (en) | 2018-04-11 | 2018-04-11 | Image generation method, device and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810340384.XA CN108492347B (en) | 2018-04-11 | 2018-04-11 | Image generation method, device and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108492347A true CN108492347A (en) | 2018-09-04 |
| CN108492347B CN108492347B (en) | 2022-02-15 |
Family
ID=63316057
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810340384.XA Active CN108492347B (en) | 2018-04-11 | 2018-04-11 | Image generation method, device and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108492347B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110853675A (en) * | 2019-10-24 | 2020-02-28 | 广州大学 | Device for music synaesthesia painting and implementation method thereof |
| CN110978881A (en) * | 2019-12-12 | 2020-04-10 | 广东智媒云图科技股份有限公司 | Sand painting drawing method and device based on audio file and terminal equipment |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20010037181A1 (en) * | 1999-10-14 | 2001-11-01 | Masaya Matsuura | Audio processing and image generating apparatus, audio processing and image generating method, recording medium and program |
| CN101577114A (en) * | 2009-06-18 | 2009-11-11 | 北京中星微电子有限公司 | Method and device for implementing audio visualization |
| CN103489454A (en) * | 2013-09-22 | 2014-01-01 | 浙江大学 | Voice endpoint detection method based on waveform morphological characteristic clustering |
| CN104167211A (en) * | 2014-08-08 | 2014-11-26 | 南京大学 | Multi-source scene sound abstracting method based on hierarchical event detection and context model |
| CN104424955A (en) * | 2013-08-29 | 2015-03-18 | 国际商业机器公司 | Audio graphical expression generation method and equipment, and audio searching method and equipment |
| US20160247520A1 (en) * | 2015-02-25 | 2016-08-25 | Kabushiki Kaisha Toshiba | Electronic apparatus, method, and program |
| CN106504767A (en) * | 2016-10-12 | 2017-03-15 | 耿梁宸 | Musical instrument tuning and song pitch determination methods by sound waveform |
-
2018
- 2018-04-11 CN CN201810340384.XA patent/CN108492347B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20010037181A1 (en) * | 1999-10-14 | 2001-11-01 | Masaya Matsuura | Audio processing and image generating apparatus, audio processing and image generating method, recording medium and program |
| CN101577114A (en) * | 2009-06-18 | 2009-11-11 | 北京中星微电子有限公司 | Method and device for implementing audio visualization |
| CN104424955A (en) * | 2013-08-29 | 2015-03-18 | 国际商业机器公司 | Audio graphical expression generation method and equipment, and audio searching method and equipment |
| CN103489454A (en) * | 2013-09-22 | 2014-01-01 | 浙江大学 | Voice endpoint detection method based on waveform morphological characteristic clustering |
| CN104167211A (en) * | 2014-08-08 | 2014-11-26 | 南京大学 | Multi-source scene sound abstracting method based on hierarchical event detection and context model |
| US20160247520A1 (en) * | 2015-02-25 | 2016-08-25 | Kabushiki Kaisha Toshiba | Electronic apparatus, method, and program |
| CN106504767A (en) * | 2016-10-12 | 2017-03-15 | 耿梁宸 | Musical instrument tuning and song pitch determination methods by sound waveform |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110853675A (en) * | 2019-10-24 | 2020-02-28 | 广州大学 | Device for music synaesthesia painting and implementation method thereof |
| CN110978881A (en) * | 2019-12-12 | 2020-04-10 | 广东智媒云图科技股份有限公司 | Sand painting drawing method and device based on audio file and terminal equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108492347B (en) | 2022-02-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108898643B (en) | Image generation method, device and computer readable storage medium | |
| CN108615253B (en) | Image generation method, device and computer readable storage medium | |
| CN108970113B (en) | Collision detection method, device, equipment and medium | |
| CN108961303A (en) | A kind of image processing method, device, electronic equipment and computer-readable medium | |
| CN108805171B (en) | Method, device and computer readable storage medium for converting image to music melody | |
| CN109559366A (en) | Display methods, device, storage medium and the equipment of potted plant growth course | |
| CN107437272B (en) | Interactive entertainment method and device based on augmented reality and terminal equipment | |
| US11288499B2 (en) | Interactive method for generating strokes with Chinese ink painting style and device thereof | |
| CN108960250B (en) | Method and device for converting image into melody and computer readable storage medium | |
| CN109151540A (en) | The interaction processing method and device of video image | |
| CN110119733B (en) | Page identification method and device, terminal equipment and computer readable storage medium | |
| CN110070499A (en) | Image processing method, device and computer readable storage medium | |
| CN111047509A (en) | Image special effect processing method and device and terminal | |
| CN108492347A (en) | Image generating method, device and computer readable storage medium | |
| CN108986181B (en) | Dot-based image processing method, device and computer readable storage medium | |
| CN109145688A (en) | The processing method and processing device of video image | |
| CN110378336A (en) | Semantic class mask method, device and the storage medium of target object in training sample | |
| CN110570497B (en) | Drawing method and device based on layer superposition, terminal equipment and storage medium | |
| CN112950753B (en) | Virtual plant display method, device, equipment and storage medium | |
| CN109934896B (en) | Drawing method and device, terminal equipment and computer readable storage medium | |
| CN109191539B (en) | Oil painting generation method and device based on image and computer readable storage medium | |
| CN112070227A (en) | Neural network training method, device and equipment | |
| CN109408943A (en) | A kind of water spray control method, apparatus, system and storage medium | |
| CN115588065A (en) | Method and device for generating hair model, computer equipment and storage medium | |
| CN108171794B (en) | Plane view projection method, device, equipment and medium based on three-dimensional model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |