WO2023008702A1

WO2023008702A1 - Digital content generation device based on recording operation for multi-tracks, and method therefor

Info

Publication number: WO2023008702A1
Application number: PCT/KR2022/006704
Authority: WO
Inventors: 이영욱; 이제원; 윤데이빗
Original assignee: 아이디아이디 주식회사
Priority date: 2021-07-30
Filing date: 2022-05-11
Publication date: 2023-02-02
Also published as: KR102377081B1

Abstract

The present invention relates to a digital content generation device and a method therefor, and a digital content generation device according to one embodiment comprises: a data reception unit for receiving real-time recorded data corresponding to at least one track from among multi-tracks displayed on a screen; a data management unit for applying a preset image effect and/or a voice effect to the real-time recorded data; and a multi-track control unit for encoding the real-time recorded data to which at least one effect is applied.

Description

Digital content generating device and method based on multi-track recording operation

The present invention relates to a digital content generating device, and more particularly, to a technical idea of controlling a recording operation for multi-tracks.

Recently, demand for video sharing applications such as YouTube, TikTok, and Triller is increasing.

These video sharing applications provide overall functions necessary for video sharing, such as video recording, editing, and uploading, but do not provide efficient visualization, recording, and editing functions for multi-tracks including multiple audio tracks. can't

An object of the present invention is to provide a digital content generating device capable of more efficiently recording a multi-track consisting of a bit track, a video track, and an audio track.

In addition, the present invention is intended to provide a digital content generating device capable of applying video effects and audio effects to data corresponding to each multi-track in real time.

A digital content generating device according to an embodiment of the present invention includes a data receiving unit receiving real-time recorded data corresponding to at least one track among multi-tracks displayed on a screen, and at least one of preset video effects and audio effects. It may include a data management unit that applies to real-time recorded data and a multi-track controller that encodes the real-time recorded data to which at least one effect is applied.

According to one side, the data management unit may apply at least one effect corresponding to preset initial effect information to real-time recording data.

According to one side, the data management unit may receive a user input for applying an effect, and apply at least one effect corresponding to the user input for applying the effect to the real-time recorded data.

According to one side, the preset image effect may include at least one of a color filter effect and an augmented reality sticker effect.

According to one side, the preset voice effect includes at least one of a reverb effect, a distortion effect, a delay effect, a pitch correction effect, and a pitch shift effect. can do.

According to one side, the multi-track may include at least two or more tracks displayed on a screen among a bit track, a video track, and an audio track.

According to one side, the real-time recorded data may include any one of video data recorded in real time through a video input unit and audio data recorded in real time through an audio input unit.

A digital content creation method according to an embodiment of the present invention includes the steps of receiving real-time recorded data corresponding to at least one track among multi-tracks displayed on a screen in a data receiving unit, and among preset video effects and audio effects in a data management unit. The method may include applying at least one effect to real-time recorded data and encoding the real-time recorded data to which the at least one effect is applied in a multi-track controller.

According to one embodiment, the present invention can more efficiently perform a recording operation for multi-tracks composed of bit tracks, video tracks, and audio tracks.

According to one embodiment, the present invention can apply video effects and audio effects to data corresponding to each multi-track in real time.

1 is a diagram for explaining a digital content generating device according to an embodiment.

2 is a diagram for explaining a digital content generating device according to a first embodiment.

3 is a diagram for explaining a digital content generating device according to a second embodiment.

4 is a diagram for explaining a digital content generating device according to a third embodiment.

5 is a diagram for explaining an operation example of a digital content generating device according to an embodiment.

6 is a diagram for explaining an example of activating a multi-track using a digital content generating device according to an embodiment.

7 is a diagram for explaining an example of generating a bit track using a digital content generating device according to an embodiment.

8 is a diagram for explaining an example of generating a video track using a digital content generating device according to an exemplary embodiment.

9 is a diagram for explaining an example of generating a voice track using a digital content generating device according to an embodiment.

10 is a diagram for explaining an operating method of a digital content generating device according to a second embodiment.

11 is a diagram for explaining an operating method of a digital content generating device according to a third embodiment.

Hereinafter, various embodiments of this document will be described with reference to the accompanying drawings.

Examples and terms used therein are not intended to limit the technology described in this document to specific embodiments, and should be understood to include various modifications, equivalents, and/or substitutes of the embodiments.

In the following description of various embodiments, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the subject matter of the invention, the detailed description will be omitted.

In addition, terms to be described below are terms defined in consideration of functions in various embodiments, and may vary according to intentions or customs of users or operators. Therefore, the definition should be made based on the contents throughout this specification.

In connection with the description of the drawings, like reference numerals may be used for like elements.

Singular expressions may include plural expressions unless the context clearly dictates otherwise.

In this document, expressions such as "A or B" or "at least one of A and/or B" may include all possible combinations of the items listed together.

Expressions such as "first," "second," "first," or "second," may modify the corresponding components regardless of order or importance, and are used to distinguish one component from another. It is used only and does not limit the corresponding components.

When a (e.g., first) component is referred to as being "(functionally or communicatively) connected" or "connected" to another (e.g., second) component, a component refers to said other component. It may be directly connected to the element or connected through another component (eg, a third component).

In this specification, "configured to (or configured to)" means "suitable for," "having the ability to," "changed to" depending on the situation, for example, hardware or software ," can be used interchangeably with "made to," "capable of," or "designed to."

In some contexts, the expression "device configured to" can mean that the device is "capable of" in conjunction with other devices or components.

For example, the phrase "a processor configured (or configured) to perform A, B, and C" may include a dedicated processor (eg, embedded processor) to perform the operation, or by executing one or more software programs stored in a memory device. , may mean a general-purpose processor (eg, CPU or application processor) capable of performing corresponding operations.

Also, the term 'or' means 'inclusive or' rather than 'exclusive or'.

That is, unless otherwise stated or clear from the context, the expression 'x employs a or b' means any one of the natural inclusive permutations.

In the above-described specific embodiments, components included in the invention are expressed in singular or plural numbers according to the specific embodiments presented.

However, singular or plural expressions are selected appropriately for the presented situation for convenience of explanation, and the above-described embodiments are not limited to singular or plural components, and even components expressed in plural are composed of a singular number or , Even components expressed in the singular can be composed of plural.

Meanwhile, in the description of the invention, specific embodiments have been described, but various modifications are possible without departing from the scope of the technical idea contained in the various embodiments.

Therefore, the scope of the present invention should not be limited to the described embodiments and should not be defined, but should be defined by not only the claims to be described later, but also those equivalent to these claims.

Referring to FIG. 1 , a digital content generating apparatus 100 according to an embodiment may include an input unit 110, a multi-track controller 120, a recording controller 130, and a display controller 140.

For example, the digital content generating device 100 may be provided in at least one user terminal of a smart phone and a tablet PC, and may operate in conjunction with a video sharing application executed in the user terminal.

The input unit 110 according to an embodiment includes a video input unit capable of recording an image such as a camera, an audio input unit capable of recording audio such as a microphone, and a user's touch input on a screen provided in a user terminal. It may include a touch input means that can be.

User input described below means a user's touch input through a touch input means, but user input according to an embodiment is not limited thereto, and a user's gesture input using a video input means and a user's voice using a voice input means May contain command input.

The multi-track controller 120 according to an embodiment may generate a UI control signal for displaying at least two or more multi-tracks among a bit track, a video track, and an audio track in response to a user input.

For example, the bit track may include at least one of bit sound source data and background sound source data. In addition, the bit track may be downloaded from a bit data server in advance and stored in the user terminal, or may be prepared in advance by the user and stored in the user terminal.

In addition, the video track is a track corresponding to at least one of video data being recorded (ie, recorded) in real time through the video input unit and video data that has been recorded (ie, recorded), and the audio track is a track corresponding to the audio input unit. It may be a track corresponding to at least one of voice data that is being recorded (ie, recorded) in real time and voice data that has been recorded (ie, recorded).

The display control unit 140 according to an embodiment receives a UI control signal from the multi-track control unit 120 and, based on the received UI control signal, efficiently displays the multi-track on the screen without covering the image displayed on the screen as much as possible. can be visualized as

In addition, the display controller 140 may visualize at least one virtual button among a record button, a record completion button, a play button, and a stop button, along with an image and multi-tracks displayed on the screen, on the screen.

Specifically, when a user input to the record button is detected in a state in which one of the multi-tracks is activated by a user input, the multi-track controller 120 records real-time recording data (ie, real-time recording) corresponding to the activated track. video/audio data) can be created.

In addition, after a user input to the record button is detected, the multi-track controller 120 detects a user input to the stop button (or stop button + recording completion button) or sets a preset threshold when a user input to the stop button is detected. If it is not detected until the time is exceeded, the recorded data from the first recording point (0:00) to the time when the input to the stop button is detected or the critical time / or voice data).

According to one side, the preset threshold time may be the total playback time (ie, bit length) of the bit track.

Specifically, when the bit length of the preset bit track is 3 minutes and 40 seconds, the multi-track control unit 120 may perform recording (recording/recording) for a single track up to 3 minutes and 40 seconds, and one track If the recording progresses for more than 3 minutes 40 seconds, only one file (data) can be created for one track, and if the recording progresses in three divisions within 3 minutes 40 seconds for one track, one Three files (data) can be created for a track.

According to one side, the input unit 110, the multi-track controller 120, and the display controller 140 may be controlled through the recording controller 130. That is, the multi-track controller 120 may be controlled through the recording controller 130 to generate real-time recorded data and/or recorded data.

Meanwhile, when a user input to the record button is detected, the display controller 140 may display a stop button instead of the record button at a location corresponding to the record button, and then, when a user input to the stop button is detected, the stop button The record button may be displayed again instead of the stop button at the corresponding location.

Also, the display controller 140 may visualize a recording completion button together with a stop button.

The multi-track controller 120 and the display controller 140 according to an exemplary embodiment will be described in more detail with reference to FIG. 2 hereinafter.

The recording controller 130 according to an embodiment receives real-time recorded data or recorded data corresponding to at least one track among multi-tracks displayed on the screen, and applies at least one of video effects and audio effects to the received data. can be applied

Specifically, the recording control unit 130 may provide functions necessary for the overall multi-track recording process and editing process.

To this end, the recording controller 130 may include an audio manager (AudioManager), an audio recording manager (AudioRecordManager), a camera view manager (CameraViewManager), a media manager (MediaManager), and a video view manager (VideoViewManager).

Specifically, an audio manager is a manager that supports simultaneous playback of multiple tracks with voice effects (effects such as Reverb, Delay, Distortion, Pitch Shift, and Pitch correction), where the tracks are N simultaneously (where N is a positive integer). ) can be played.

The audio manager can directly load PCM data to reproduce voice tracks with voice effects, and process and reproduce data with voice effects.

More specifically, the audio manager includes a startSession function that initializes the audio track, a stopSession function that stops the audio track and returns the corresponding resource, to apply voice effects to the audio track, and a function that is actually output from the speaker of the user terminal when the audio is played. You can use the getAudioDelay function that returns the delay time, the setEffect function that applies audio effects to the selected audio track, and the setEQ function that sets EQ for all tracks of the selected audio track.

In addition, the audio manager includes a setVolume function that adjusts the volume of the selected voice track, a setPlaylist function that inputs information about the voice track to be played, a setTotalDurationTime function that determines the total playback length of the selected voice track, and a fade-in function during voice playback. in), setFadeTiming function that specifies the fade-out time, play function that plays audio according to the start time of the audio track, and stop function that stops the currently playing audio track.

The audio recording manager is a manager for recording (recording) only the voice without recording (recording) the video. To this end, the audio recording manager uses setEchoMode to echo (monitor) the voice currently coming into the microphone with the currently applied voice effect. You can use functions, startAudioRecord function to start recording audio in the audio track, and stopAudioRecord function to stop recording audio in the audio track.

Here, the startAudioRecord function can control to create and add a new audio file separately from the existing audio file when the audio track is re-recorded after the recording is stopped.

The camera view manager is a manager for outputting the screen received from the camera together with shaders and recording the video output in shader state. startSession function that comes in, endSession function that ends the camera and all related operations, setCameraDirection function that sets the front/rear camera, setFlashLight function that sets the camera flash, determines the shader of the camera view, and applies the set video effect. You can use the setShader function to set the zoom of the camera, the setZoom function to set the zoom of the camera, the startVideoRecord function to record real-time video and audio information visualized in the current camera view, and the stopVideoRecord function to end recording.

Here, only one video track is visualized among the multi-tracks, but the stopVideoRecord function can create and add a new video file without recording following the existing video file when re-recording after stopping recording.

The media manager is a manager for getting information from video and audio files recorded in correspondence with video and audio tracks. getVideoPreviewBase64 function that returns one image as a string, getVideoPreviewListBase64 function that returns several thumbnail images from multiple video files as a string list, getAudioFileDescription function that gets audio length information from an audio file, and getAudioVisualPCMData that gets PCM data from an audio file. function can be used.

The video view manager is a manager for playing multiple recorded video files with designated shaders. To do this, the video view manager has a setPlaylist function that stores a list of video files in video tracks, fade-in and fade-out times when playing video. setFadeTiming function to save, play function to play the video according to the start time, stop function to end the video being played, seek function and video view to describe/play a list of multiple video files with the corresponding file and time according to the video search You can use the setShader function to set the shader of

Here, the video track is a single track, but the setPlaylist function can create and add a new video file without continuing to record the existing video file when re-recording after stopping recording. In addition, the setShader function can be applied to various video effect filters including LUT (Look-Up Table) filters of video.

Meanwhile, the multi-track control unit 120 may encode data recorded/edited through the recording control unit 130, and may include an encoding manager (EncodingManager) for this purpose.

The encoding manager is a manager for generating the final video based on the project information stored in the project configuration information (Project.json). You can use the stop function, where the Project.json file can be created when entering the recording screen.

According to one side, in the multi-track recording process using the digital content generating device 100, when the recording screen is loaded after the recording screen is executed on the screen of the user terminal, the camera view manager activates the camera, and the audio manager activates the camera. You can reset the volume, EQ and voice effects of all tracks.

Next, when a video track is added/changed on the screen, the camera view manager can initialize camera settings (direction, zoom, etc.), and the audio manager can initialize the volume, EQ, and voice effect of the corresponding track. When a voice track is added/changed, the audio manager can initialize the volume, EQ, and voice effects of the corresponding track.

Next, when a video track is recorded on the screen, the camera view manager records video/audio data, and the audio manager can initialize the volume, EQ, and audio effects of the corresponding track.

Next, when a voice track is recorded on the screen, the audio recording manager records voice data, and the audio manager can initialize the volume, EQ, and voice effects of the corresponding track.

Next, when a video effect is added/changed on the screen, the camera view manager applies the video effect to the video data, and when a voice effect is added/changed on the screen, the audio recording manager monitors the audio data, and You can initialize and change the voice effects of voice tracks in the manager.

According to one side, in a multi-track editing process using the digital content generating device 100, when an editing screen is executed on the screen of a user terminal, the media manager can call video/audio file information for each track.

Next, when the edit screen is loaded on the screen, the video view manager can activate the video, and when stickers/texts are added to the video on the screen, the video view manager can add a video texture.

Next, when the length of the video is adjusted on the screen, the video view manager provides a preview of the video, and the media manager can load corresponding video file information.

Next, when a video fade-in/fade-out effect is applied on the screen, the video view manager provides a preview of the video fade effect, and the media manager can call corresponding video file information.

Next, when a voice fade-in/fade-out effect is applied on the screen, the audio manager provides a preview of the voice fade effect, and the media manager can load corresponding voice file information.

Next, when EQ is applied on the screen, the audio manager provides a preview of the voice EQ, and the media manager can call the corresponding voice file information.

Next, when a video cover image is selected on the screen, the video view manager provides a corresponding video preview, and the media manager can call information about one image selected from the video preview through a user's input.

The recording control unit 130 according to an embodiment will be described in more detail with reference to FIGS. 3 and 4 hereinafter.

Referring to FIG. 2 , the digital content generating apparatus 200 according to the first embodiment can more efficiently perform a recording operation for multi-tracks composed of a bit track, a video track, and an audio track.

In addition, the digital content generating device 200 can efficiently visualize multi-tracks on the screen without covering the images displayed on the screen as much as possible.

In addition, the digital content generating apparatus 200 can improve user convenience by dividing multi-tracks into a guide track and a recording track, and simultaneously visualizing the divided guide track and recording track on a screen.

To this end, the digital content generating device 200 may include a multi-track control unit 210 and a display control unit 220. That is, the multi-track controller 210 and the display controller 220 described with reference to FIG. 2 may be the multi-track controller and display controller of the digital contents generating device according to the embodiment described with reference to FIG. 1 .

The multi-track controller 210 according to the first embodiment may generate a UI control signal based on a user input.

In addition, the display controller 220 may display at least two multi-tracks among a bit track, a video track, and an audio track on a first area of the screen based on the UI control signal, and display a record button on a second area of the screen. there is.

For example, the display controller 220 may set a first area and a second area disposed below the first area on the screen of the user terminal.

Also, in the multi-track, at least one of a video track and an audio track may be synchronized with a bit track. In other words, in multi-tracks, other tracks may be synchronized based on time information of bit tracks.

According to one side, the multi-track controller 210 divides the multi-track into a guide track on which recording is completed and a shooting track on which recording is possible, displays the guide track in a first sub area provided in the first area, and captures the track. A UI control signal for displaying in the second sub-area provided in the first area may be generated.

For example, the progress of the selected track can be displayed in the guide track, recording progress in the recording track, preview in the case of a video track, and waveforms of recorded audio in the case of an audio track.

According to one side, the multi-track controller 210 may classify a track in which the length of recorded data is equal to or greater than a preset critical time and a bit track among video tracks and audio tracks as guide tracks.

Also, the multi-track controller 210 may classify a track on which a recording completion signal is received according to a user input and a bit track among video tracks and audio tracks as guide tracks.

That is, the beat track may be fixedly displayed in the first area at the same time as the guide track is selected according to the user input.

Specifically, the multi-track controller 210 sets a preset threshold for a user input for a stop button after any one of a video track and an audio track is activated and a user input for a record button displayed on the screen is detected. If it is not detected until the time has elapsed or if a user input to the recording completion button displayed on the screen is detected, the activated track may be classified as a guide track.

In addition, the multi-track controller 210 divides the first area into a first sub area and a second sub area disposed below the first sub area, displays at least one guide track in the first sub area, and displays the first sub area. A UI control signal for displaying at least one shooting track in the 2 subregions may be generated.

According to one side, the first sub-region includes a first track region displaying a first guide track among N tracks (where N is a positive integer) divided into guide tracks, and an icon corresponding to the first guide track. A displayed first icon area may be included.

In addition, the second sub area includes a second track area displaying a first recording track among M tracks divided into recording tracks (where M is a positive integer), and an icon corresponding to the first recording track is displayed. A second icon area may be included.

According to one side, the multi-track controller 210 transmits a UI control signal for controlling display of a second guide track among N tracks divided into guide tracks in the first track area when a user input to the first icon area is detected. can create

In addition, the multi-track controller 210 may generate a UI control signal for controlling display of a second recording track among M tracks divided into recording tracks in the second track area when a user input to the second icon area is detected. can

Specifically, in each of the first sub-region and the second sub-region, only a first guide track among the first to N-th guide tracks and a first shooting track among the first to M-th shooting tracks may be displayed on the screen. Tracks displayed on the screen may be sequentially changed whenever a user input is detected for the first icon area and the second icon area provided in the first sub area and the second sub area, respectively.

In other words, the second to Nth guide tracks are sequentially displayed in the first track area whenever a user input to the first icon area is detected, and the second track is displayed whenever a user input to the second icon area is sensed. In the area, the second through Mth recording tracks may be sequentially displayed.

In addition, when a user input to the first icon area is detected while the Nth guide track is displayed in the first track area, the first guide track is displayed again in the first track area, and the Mth shooting track is displayed in the second track area. In the displayed state, when a user input to the second icon area is detected, the first shooting track may be displayed again in the second track area.

For example, each of the first to Nth guide tracks and the first to Mth photographic tracks may be sequentially numbered based on the time they were created.

Meanwhile, the display controller 220 may display a play button on the second area of the screen when a user input for the guide track is detected.

Specifically, when the guide track is activated, the display control unit 220 may display a play button instead of a record button at a position corresponding to the record button in the second area, and when the recording track is activated, the record button is displayed in the second area. can be displayed as is.

According to one side, the multi-track controller 210 sets at least one of the video track and the first audio track and the bit track to the first region when a user input to the record button or a user input for executing a preset application is detected. A UI control signal for displaying can be generated.

For example, the preset application is a video sharing application that provides a recording function for video/audio, displays multi-tracks in the first area when the application is executed based on the operation of the digital content generating device 200, and displays the second application in the second area. A record button can be displayed in the area.

Specifically, the multi-track controller 210 may generate one video track and one audio track as a recording track when a predetermined application is executed through a user input.

In addition, the multi-track controller 210 may create one video track and one audio track as a recording track when a user input to the record button is initially detected after a preset application is executed through a user input. .

In addition, the multi-track control unit 210 may select any one of a plurality of beats based on a user's input to generate a guide track, and guide the preset beat track when a preset application is executed through the user's input. It can also be automatically created as a track.

According to one side, the multi-track controller 210 records a second audio track when at least one of a video track and a first audio track and a bit track are displayed in the first area and a user input to the record button is detected. A UI control signal for displaying in area 1 may be generated.

Thereafter, the multi-track control unit 210 generates a UI control signal for generating and displaying third to ith audio tracks (where i is a positive integer) on the first area whenever a user input to the record button is input. can create

That is, one bit track, one video track, and i audio tracks may be displayed in the first area.

According to one side, the multi-track controller 210 may generate a UI control signal for controlling activation of any one track when a user input according to a touch operation of any one of the multi-tracks is sensed once.

In addition, the multi-track controller 210 may generate a UI control signal for displaying an edit menu for any one track when a user input corresponding to a touch operation of any one track is detected twice.

In addition, the multi-track controller 210 generates a UI control signal for displaying a track option menu for one track when a user input according to a touch operation of one track lasting for a predetermined input time is detected. can

Specifically, the multi-track control unit 210 activates any one of the multi-tracks when it is briefly touched once, and provides an editing function for any one track when any one track is briefly touched twice. And, if any one track is touched for a long time, a track option function can be provided.

For example, the edit menu for any one track may include at least one of an audio FX function, a track break function, a volume control function, and a track initialization function.

Also, the track option function may include at least one of a track delete function and a track duplication function.

Meanwhile, when a recording or reproducing operation corresponding to the video track is performed based on a user input for the video track, the display controller 220 displays video data according to the recording or reproducing operation on the screen. It can be controlled so that they are overlapped and displayed.

In other words, the display controller 220 can efficiently visualize multi-tracks on the screen without covering the images displayed on the screen as much as possible.

The digital content generating device 200 according to the first embodiment will be described in more detail with reference to FIGS. 5 to 9 hereinafter.

Referring to FIG. 3 , the digital content generating apparatus 300 according to the second embodiment can more efficiently perform a recording operation for multi-tracks composed of a bit track, a video track, and an audio track.

In addition, the digital content generating device 300 may apply video effects and audio effects to data corresponding to each of the multi-tracks in real time.

To this end, the digital content generating device 300 may include a data receiving unit 310, a data management unit 320, and a multi-track control unit 330. That is, the data receiving unit 310 and the data management unit 320 described with reference to FIG. 3 may be included in the recording control unit of the digital content generating device according to the embodiment described with reference to FIG. 1, and the multi-track control unit 330 may be It may be a multi-track control unit of a digital content generating device according to an embodiment.

The data receiving unit 310 according to the second embodiment may receive real-time recorded data corresponding to at least one track among multi-tracks displayed on the screen.

For example, the multi-track may include at least two or more tracks displayed on a screen among a bit track, a video track, and an audio track.

Also, the real-time recording data may include any one of video data being recorded in real time through the video input unit and audio data being recorded in real time through the audio input unit.

The data management unit 320 according to the second embodiment may apply at least one of preset video effects and audio effects to real-time recorded data.

For example, the preset video effect includes at least one of a color filter effect and an augmented reality (AR) sticker effect, and the preset audio effect includes a reverb effect, a distortion effect, and a delay effect. ) effect, a pitch correction effect, and a pitch shift effect.

According to one side, the screen includes a first area where multi-tracks are displayed, a second area disposed below the first area and displaying at least one virtual button selected from among a record button, a record completion button, a play button, and a stop button, and a second area. It may include a third area disposed at the bottom of area 2 and displaying information on any one activated track among the multi-tracks displayed in the first area.

Specifically, when the beat track displayed in the first area is briefly touched twice, at least one of a beat break icon, a volume icon, and an initialization icon for initializing edits made in the corresponding session may be displayed in the third area, Then, when any one of the beat break icon and the volume icon displayed on the third area is touched, a detailed editing function for applying the effect of the touched icon may be displayed on the second area.

For example, when the beat break icon is touched, a record button is displayed in the second area, and then the beat of a desired section in the beat track is recorded through a swipe motion for the track displayed in the first area and a user input to the record button. can be deleted

In addition, when the volume control icon is touched, a gauge bar for controlling the volume of the beat track is displayed in the second region, and then the volume of beat data corresponding to the beat track can be controlled through a user input to the gauge bar.

When the voice track displayed in the first area is briefly touched twice, an audio FX icon, an audio track break icon, a volume icon, and an initialization icon may be displayed in the third area, and then the audio FX icon and audio track break displayed in the third area When any one of the icon and the volume icon is touched, a detailed editing function for applying an effect of the touched icon may be displayed in the second area.

For example, when an audio FX icon is touched, a revolve control icon, a distortion control icon, a delay control icon, and a pitch control icon, which are detailed functions of the audio FX, are displayed in the second area, and then the user for the icons displayed in the second area Through the input, the relevant voice data can be adjusted.

In addition, when the audio track break icon is touched, a record button is displayed in the second area, and then audio data of a desired section in the voice track is recorded through a swipe motion for the track displayed in the first area and a user input to the record button. can be deleted

Similarly, if the video track displayed in the first area is briefly touched twice, a color filter icon and an augmented reality sticker icon may be displayed in a third area, and then at least one of the color filter icon and the augmented reality sticker icon displayed in the third area. When an icon of is touched, a detailed editing function for applying an effect of the touched icon may be displayed in the second area.

According to one side, the augmented reality sticker is linked to the corresponding content, and the data management unit 320 may reflect at least one augmented reality sticker among a plurality of augmented reality stickers to the image data according to a user input, and to the image data. When the reflected augmented reality sticker is touched, content corresponding to the touched augmented reality sticker may be provided to the user.

For example, content corresponding to an augmented reality sticker may include image content.

According to one side, the data management unit 320 may apply at least one effect corresponding to preset initial effect information to real-time recording data.

According to one side, the data management unit 320 may receive a user input for applying an effect, and apply at least one effect corresponding to the user input for applying an effect to real-time recorded data.

Also, the multi-track controller 330 according to the second embodiment may encode real-time recorded data to which at least one effect is applied.

Referring to FIG. 4 , the digital content generating apparatus 400 according to the third embodiment can more efficiently perform a recording operation for multi-tracks composed of a bit track, a video track, and an audio track.

In addition, the digital content creation device 400 can more efficiently edit recorded data corresponding to each of the multi-tracks.

To this end, the digital content generating device 400 may include a data receiving unit 410, a data editing unit 420, and a multi-track control unit 430. That is, the data receiving unit 410 and the data editing unit 420 described with reference to FIG. 4 may be included in the recording control unit of the digital content generating device according to the embodiment described with reference to FIG. 1, and the multi-track control unit 430 may be It may be a multi-track control unit of a digital content generating device according to an embodiment. Also, the data receiving unit 410 may be a data receiving unit according to the second embodiment described with reference to FIG. 3 .

Meanwhile, the digital content generating device 400 according to the third embodiment may be performed when the screen of the user terminal is converted to an editing mode (ie, a mastering mode).

The data receiving unit 410 according to the third embodiment may receive recorded data corresponding to at least one track among multi-tracks displayed on the screen.

According to one side, the digital content generating device 400 according to the third embodiment may further include a data rendering unit that renders image data among recorded data.

The data editing unit 420 according to the third embodiment may apply at least one of an image sticker effect, a text effect, a clip adjustment effect, and a fade effect to recorded data.

For example, the image sticker effect is an effect of inserting at least one image sticker selected through a user input from among a plurality of preset image stickers when the recorded data is video data, and the clip adjustment effect is a video clip corresponding to the video data. and an effect of adjusting the size of an audio clip corresponding to audio data, and the fade effect may be an effect of adjusting fade-in and fade-out effects for video data and audio data.

Also, the text effect may include at least one of a text input effect for video data among recorded data, a color editing effect for text input to the video effect, a font editing effect, and a size editing effect.

According to one side, the data editing unit 420 may receive a user input for applying an effect, and apply at least one effect corresponding to the user input for applying an effect to the recorded data.

According to one side, the screen displays information about a first region in which multi-track images recorded through a data generating device are combined and reproduced as one image, and an integrated track disposed at the bottom of the first region and integrating the multi-tracks. A second area and a third area disposed below the second area and displaying at least one virtual button selected from among a record button, a recording completion button, a play button, and a stop button.

According to one side, when the screen of the user terminal is switched to the editing mode through a user input, at least one of a sticker effect icon, a text effect icon, a clip adjustment icon, a fade effect icon, an audio EQ icon, and a cover setting icon is displayed in the third area. An icon may be displayed, and a control bar corresponding to the integrated video file may be displayed in the first area.

For example, when an audio EQ icon displayed in the third region is touched, detailed functions for controlling the audio EQ may be displayed in the first region.

In addition, when the cover setting icon displayed in the third region is touched, a function of applying any one of a pre-stored cover image or an image included in video data as a cover image may be provided.

According to one side, the data editing unit 420 may apply at least one effect to recorded data on which rendering has been performed.

According to one aspect, the data editing unit 420 may calculate a delay time between at least one data track of a video track and an audio track and a bit track, and synchronize the data track and the bit track based on the calculated delay time.

Specifically, the digital content creation device 400 visualizes each track as a single video file by loading the saved project setting file in the multi-track recording process. In this process, the bit track, each recorded video track, and audio track A lag time will occur between them.

Accordingly, the data editing unit 420 may calculate a delay time between tracks and compensate for the calculated delay time so as not to cause a problem when reproduced as a single video.

Meanwhile, the multi-track controller 430 according to the third embodiment may encode recorded data to which at least one effect is applied.

Referring to FIG. 5, the screen of the user terminal according to the operation of the digital content generating device according to an embodiment is a first area where at least two

multi-tracks

511 and 513 of a bit track, a video track, and an audio track are displayed ( 510), and a second area 520 disposed below the first area 510 and displaying a record button 521.

For example, the screen of the user terminal according to the operation of the digital content generating device may be a studio mode screen according to a video sharing application.

In addition, the second region 520 may display at least one virtual button among a record completion button, a play button, and a stop button in addition to the record button 521 according to a user input.

According to one side, the screen of the user terminal is disposed at the bottom of the second area and includes a third area 530 displaying information about one of the activated tracks among the multi-tracks displayed on the first area and an image recorded through the screen. A fourth area 540 providing a control function for at least one of data and voice data may be included.

For example, the information on any one activated track displayed in the third area 530 includes an initialization function for initializing the current work for the activated track, type information (video, audio) of the activated track, and activated track information. You can include an undo function to undo previously performed actions on the track.

In addition, the control functions displayed in the fourth area 540 include a camera switching function for switching the camera direction (front/rear) of the user terminal, a function to load pre-stored lyrics, a function to monitor the corresponding video/voice, and a volume control function. , A camera flash on/off function and a beauty mode on/off function may be included, and a control function item may be selected according to a touch operation of an arrow icon displayed at the bottom of the fourth area 540. In the fourth area 540, the whole display or summary display may be displayed.

Meanwhile, when a recording operation or a playback operation corresponding to a video track is performed, the screen of the user terminal displays an image according to the recording or playback operation on the screen, but overlaps with the first to fourth areas to display the image data. may further include an image display area to zoom in/out on an image displayed in the image display area when a user input of swiping up/down on the screen is detected in the image display area.

According to one side, the first region 510 includes

first sub-regions

511 and 512 displaying guide tracks 511 on which recording work has been completed among multi-tracks, and second sub-regions displaying recording tracks available for recording work. (513, 514).

Specifically, the

first sub areas

511 and 512 include a first track area 511 displaying a first guide track among N tracks divided into guide tracks, and a first track area 511 displaying an icon corresponding to the first guide track. 1 icon area 512 may be included.

In addition, the

second sub areas

513 and 514 include a second track area 513 displaying a first recording track among M tracks divided into recording tracks, and a second track area 513 displaying an icon corresponding to the first recording track. An icon area 514 may be included.

In other words, each of the

first sub-regions

511 and 512 and the

second sub-regions

513 and 514 includes a first guide track among the first to N-th guide tracks and a first shooting track among the first to M-th shooting tracks. Only the first icon area 512 and the second icon area 514 provided in the

first subarea

511 and 512 and the

second subarea

513 and 514 can be displayed on the screen, respectively. Each time an input is detected, the tracks displayed on the screen can be sequentially changed.

That is, whenever a user input to the first icon area 512 is detected, the second through Nth guide tracks are sequentially displayed in the first track area 511, and the user input to the second icon area 514 is displayed. Whenever this is detected, the second through Mth recording tracks may be sequentially displayed in the second track area 513 .

In addition, when a user input to the first icon area 512 is detected while the Nth guide track is displayed on the first track area 511, the first guide track is displayed again on the first track area 511. When a user input to the second icon area 514 is sensed while the M recording track is displayed in the second track area 513, the first recording track may be displayed again in the second track area 513.

According to one side, the screen of the user terminal displays the guide track number displayed in the first track area 511 and information on the total number of guide tracks as 'guide track number/total number of guide tracks' in a fifth area. 515, the number of recording tracks displayed in the second track area 513, and a sixth area 516 displaying information on the total number of recording tracks as 'shooting track number/total number of recording tracks' can include more.

According to one side, the first area 510 activates any one track among the multi-tracks when it is briefly touched once, and provides an editing function for any one track when any one track is briefly touched twice. and, if any one track is touched for a long time, a track option function can be provided.

For example, an activated track may be displayed in a separate color on the screen to be distinguished from an inactive track.

In addition, the edit menu for any one track may include at least one of an audio FX function, a track break function, a volume control function, and a track initialization function.

Specifically, when the voice track displayed in the first area is briefly touched twice, an audio FX icon, an audio track break icon, a volume icon, and an initialization icon may be displayed in the third area, and then the audio FX icon displayed in the third area, When any one of the audio track break icon and the volume icon is touched, a detailed editing function for applying an effect of the touched icon may be displayed in the second area.

Referring to FIG. 6, (a) of FIG. 6 illustrates an example of visualizing a beat track among guide tracks disposed in a first subregion, and (b) of FIG. 6 illustrates an example of visualizing tracks disposed in a second subregion. 6(c) shows an example of activating a video track, and FIG.

According to (a) of FIG. 6, when the beat track disposed in the first subregion is activated, track name information 610 is displayed on the activated track for a predetermined activation time (eg, 1 second), A play button 640 may be displayed in the second area.

According to (b) and (c) of FIG. 6, when the recording tracks (video track and audio track) disposed in the second sub-region are activated, track name information 620, 630) may be displayed,

record buttons

650 and 660 may be displayed in the second area, and

name information

670 and 680 of the activated track may be displayed in the third area.

On the other hand, when other tracks (video track and audio track) other than the beat track are activated among the guide tracks arranged in the first sub area, track name information is displayed on the activated track for a predetermined activation time, and in the second area A play button is displayed, and name information of an activated track may be displayed in the third region.

7(a) shows an example of displaying a guide message for selecting a bit track, and FIG. 7(b) shows an example of selecting one of a plurality of bits according to a user input, (c) of FIG. 7 shows an example of applying a selected bit to a bit track.

According to (a) to (c) of FIG. 7, the screen of the user terminal according to the operation of the digital content generating device is displayed as a bit track as shown by reference numeral 710, if the beat track is not set in advance when the studio mode screen is initially entered. You can display a message for selection.

For example, in the studio mode screen, the indicator is fixed in the center of the first area, and then the indicator moves when the user swipes the track (or indicator) left/right or the track is played or recorded and the track moves. , As the track moves, the playing time can be displayed in minutes/seconds (0:00) at the top.

Next, when the user selects the message indicated by reference numeral 710, a plurality of bit lists can be displayed on the screen as shown in FIG. 7(b).

Specifically, the bit list may include an image display area 720 corresponding to each bit, along with bit maker information, bit title, playback time, and BPM information for each bit, and the image display area 720 ), the corresponding bit data can be reproduced or stopped.

In addition, when a user swipe left or selects any one bit 730 among a plurality of bits included in the bit list, a bit use button 740 may be displayed.

According to one side, after the record button is displayed in the second area by the user activating the recording track while the message for bit selection is displayed, as shown by reference numeral 710, when the user selects the record button, a bit selection error is displayed. A bit list can also be displayed on the screen.

Then, when the beat use button 740 is selected, the sound waveform of the beat selected as the guide track is displayed on the first sub-region, and at the same time, information on the selected beat track can be displayed as shown by reference numeral 750.

Meanwhile, when the user selects reference numeral 750, a menu for changing the beat may be displayed on the screen. At this time, when the user selects the bit change, all work performed by the user through the selected beat track may be deleted.

Referring to (a) to (c) of FIG. 8 , the screen of the user terminal according to the operation of the digital content generating device according to an embodiment is displayed in a state in which the user activates a video track among shooting tracks displayed in the second subregion. When the record button 810 displayed in the second area is touched, a recording (recording) operation for the activated image track can be performed.

For example, when a user long-presses the record button 810 while recording an activated video track and swipes up/down on the screen, a zoom-in/zoom-out effect may be applied to the video during recording.

In addition, when a user swipes up/down on a screen area where an image is being displayed during a recording operation for an activated image track, a zoom-in/zoom-out effect may be applied to the image during recording.

Next, when a recording (recording) operation for the activated video track is performed, a stop button 820 may be displayed in the second area, and when the user touches the recording stop button, a recording operation for the activated video track is performed. may be stopped

Next, when the recording operation for the activated video track is stopped, a record completion button 830 may be displayed along with a record button in the second area, and when the user touches the record completion button 830, all tracks are recorded. When this is finally finished, the user's terminal screen may be moved to an editing mode (ie, a mastering mode).

According to one side, after the recording operation for the activated video track is stopped, if a recording button is touched, recording can be performed continuously from the time when recording is stopped by touching the recording stop button.

Referring to (a) to (c) of FIG. 9 , the screen of the user terminal according to the operation of the digital content generating device according to an embodiment is displayed in a state in which the user activates a voice track among photographic tracks displayed in the second subregion. When the record button 910 displayed in the second area is touched, a recording (recording) operation for the activated voice track can be performed.

According to one side, the recording operation for the activated voice track may proceed with recording from the time selected by swiping left/right on the indicator or the activated voice track.

Next, when a recording operation for the activated voice track is performed, a stop button 920 may be displayed in the second area, and the recording operation for the activated voice track may be stopped when the user touches the recording stop button. there is.

Next, when the recording operation for the activated voice track is stopped, the recording completion button 930 may be displayed along with the recording button in the second area. When the user touches the recording completion button 930, all tracks are recorded. When this is finally finished, the user's terminal screen may be moved to an editing mode (ie, a mastering mode).

In addition, after the recording operation for the activated voice track is stopped, if the recording button is touched, recording may be continued from the time point when the recording stop button is touched and recording is stopped.

In other words, FIG. 10 is a diagram for explaining a digital content generating method using the digital content generating device according to the second embodiment described with reference to FIGS. 1 to 9, and FIG. 1 to FIG. Descriptions overlapping those described with reference to FIG. 9 will be omitted.

Referring to FIG. 10 , in step 1010, the digital content creation method may receive real-time recording data corresponding to at least one track among multi-tracks displayed on the screen from the data receiving unit.

Next, in step 1020, at least one of a video effect and an audio effect preset in the data management unit may be applied to the recorded data in real time.

For example, the preset video effect includes at least one of a color filter effect and an augmented reality sticker effect, and the preset audio effect includes a reverb effect, a distortion effect, a delay effect, At least one of a pitch correction effect and a pitch shift effect may be included.

Next, in step 1030, the multi-track control unit may encode the real-time recorded data to which at least one effect is applied into a video file in the case of a video track or into an audio file in the case of an audio track according to the selected track.

In other words, FIG. 11 is a diagram for explaining a digital content generating method using the digital content generating device according to the third embodiment described with reference to FIGS. 1 to 9, and FIG. 1 to FIG. Descriptions overlapping those described with reference to FIG. 9 will be omitted.

Referring to FIG. 11 , in step 1110 of the digital content creation method, the data receiving unit may receive recorded data corresponding to at least one track among multi-tracks displayed on the screen.

Next, in step 1120, at least one of a sticker effect, a text effect, a clip adjustment effect, and a fade effect may be applied to the recorded data in the digital content generating method in the data editing unit.

According to one side, in step 1120, the digital content generation method may further include a step of rendering image data among recorded data by a data rendering unit, and then the data editing unit at least renders the rendered image data. Only one effect can be applied.

According to one side, in step 1120, the digital content creation method calculates a delay time between at least one data track and a bit track of a video track and an audio track in a data editing unit, and synchronizes the data track and the bit track based on the calculated delay time. can do.

Next, in step 1130, in the digital content generation method, the multi-track control unit may encode all data recorded in the multi-track into a single video file or an audio file if no video is recorded.

Consequently, using the present invention, it is possible to more efficiently perform a recording operation for multi-tracks composed of bit tracks, video tracks, and audio tracks.

In addition, if the present invention is used, multi-tracks can be efficiently visualized on the screen without covering the images displayed on the screen as much as possible.

In addition, according to the present invention, a user's convenience can be improved by dividing a multi-track into a guide track and a shooting track, and simultaneously visualizing the divided guide track and shooting track on a screen.

In addition, using the present invention, video effects and audio effects can be applied in real time to data corresponding to each of the multi-tracks.

In addition, by using the present invention, data recorded corresponding to each of the multi-tracks can be edited more efficiently.

As described above, although the embodiments have been described with limited drawings, those skilled in the art can make various modifications and variations from the above description. For example, the described techniques may be performed in an order different from the method described, and/or components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. Or even if it is replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

Claims

a data receiver for receiving real-time recording data corresponding to at least one track among the multi-tracks displayed on the screen;

A data management unit for applying at least one of preset video effects and audio effects to the real-time recorded data; and

Multi-track controller for encoding real-time recorded data to which the at least one effect is applied

A digital content generating device comprising a.
According to claim 1,

The data management unit,

Applying the at least one effect corresponding to preset initial effect information to the real-time recorded data

Digital content creation device.
According to claim 1,

The data management unit,

Receiving a user input for applying an effect, and applying the at least one effect corresponding to the user input for applying the effect to the real-time recorded data

Digital content creation device.
According to claim 1,

The preset video effect,

Including at least one effect of a color filter effect and an augmented reality sticker effect

Digital content creation device.
According to claim 1,

The preset sound effect,

At least one effect of a reverb effect, a distortion effect, a delay effect, a pitch correction effect, and a pitch shift effect

Digital content creation device.
According to claim 1,

The multi-track,

Including at least two or more tracks displayed on the screen among a bit track, a video track, and an audio track

Digital content creation device.
According to claim 1,

The real-time recorded data,

Including any one of the video data being recorded in real time through the video input means and the audio data being recorded in real time through the audio input means

Digital content creation device.
receiving real-time recording data corresponding to at least one track among the multi-tracks displayed on the screen, by a data receiving unit;

Applying, in a data management unit, at least one of preset video effects and audio effects to the real-time recorded data; and

Encoding, in a multi-track controller, real-time recording data to which the at least one effect is applied

Digital content creation method comprising a.
According to claim 8,

The preset video effect,

Including at least one effect of a color filter effect and an augmented reality sticker effect

How to create digital content.
According to claim 8,

The preset sound effect,

At least one effect of a reverb effect, a distortion effect, a delay effect, a pitch correction effect, and a pitch shift effect

How to create digital content.