KR101951874B1 - Real-time incording device and real-time incording method - Google Patents

Abstract

The real-time encoding apparatus includes a camera module for generating first image data; A sensing module for receiving sensing data from an external sensor; An image processing module for generating second image data by visualizing the sensing data, and merging the first image data and the second image data to generate output image data; A display module for displaying an image based on the output image data; A codec module for encoding the output image data to generate encoded data; And a transmission module for transmitting the encoded data to an external device.

Description

REAL-TIME INCORDING DEVICE AND REAL-TIME INCORDING METHOD [0002]

The present invention relates to a real-time encoding apparatus, and more particularly, to a real-time encoding apparatus and a real-time encoding method for encoding captured moving image and sensor data.

A mobile terminal (e.g., a smart phone) includes a camera and various sensors, photographs a moving picture using a camera, and uses sensors (e.g., a gyro sensor, an acceleration sensor, a GPS sensor, Lt; / RTI >

The mobile terminal can generate the sensor data (for example, the moving speed of the mobile terminal) in the form of a layer and display it together with the moving image. That is, the mobile terminal can overlay the visualized sensor data on the moving image.

Meanwhile, the mobile terminal can transmit the moving image data and the sensor data to the other mobile terminal, and the other mobile terminal can display the image substantially the same as the image displayed on the mobile terminal. However, the mobile terminal and other mobile terminals require a separate application for visualizing the sensor data.

It is an object of the present invention to provide a real-time encoding device that enables other mobile terminals to confirm sensor data without additional applications.

It is an object of the present invention to provide a real-time encoding method that allows sensor data to be verified without additional application in another mobile terminal, but minimizes the size of transmission data.

In order to accomplish one object of the present invention, a real-time encoding apparatus according to embodiments of the present invention includes: a camera module for generating first image data; A sensing module for receiving sensing data from an external sensor; An image processing module for generating second image data by visualizing the sensing data, and merging the first image data and the second image data to generate output image data; A display module for displaying an image based on the output image data; A codec module for encoding the output image data to generate encoded data; And a transmission module for transmitting the encoded data to an external device.

According to an embodiment, the display module displays an image corresponding to the second image data through a second display area of the display area, and the image processing module displays the image corresponding to the second image data, At least one of the position, size, and shape of the display area can be determined.

According to an embodiment, the sensing data includes sub-sensing data for each of temperature, pressure, level, voltage, and current, and the image processing module may generate the second image data based on a selected one of the sub- Lt; / RTI >

According to an embodiment of the present invention, the real-time encoding apparatus further includes a memory device for storing the accumulated data in which the sensing data is accumulated, wherein the image processing module includes: And the second image data can be generated using the accumulated data.

According to an embodiment, the image processing module periodically extracts the sensing values included in the sensing data, and generates the caption data based on the time at which the sensing values are extracted and the sensing values.

According to an embodiment, the display module may display the sensing value in a third display area of the display area based on the caption data.

In order to accomplish one object of the present invention, a real-time encoding method according to embodiments of the present invention can be performed in a real-time encoding device interlocked with a camera and a sensor. Wherein the real-time encoding method includes: generating second image data by visualizing sensed data provided from the sensor; Generating output image data by merging first image data and second image data provided from the camera; Encoding the output image data to generate encoded data; And transmitting the encoded data to an external device.

A real-time encoding apparatus according to embodiments of the present invention generates sensed image data by visualizing sensed data, generates encoded data by merging / coding background image data and sensed image data, It is possible to check the sensing data even if a separate application is not provided.

In the real-time encoding method according to the embodiments of the present invention, real-time streaming is performed by merging sensed data (or sensing video data, caption data) visualized in background image data in real time and encoding the merged data using a general- .

However, the effects of the present invention are not limited to the above effects, and may be variously extended without departing from the spirit and scope of the present invention.

1 is a block diagram illustrating a mobile on-site monitoring system in accordance with embodiments of the present invention.
2 is a block diagram illustrating an example of a real-time encoding device included in the mobile field monitoring system of FIG.
3A is a diagram showing an example of sensor data displayed in the real-time encoding apparatus of FIG.
FIG. 3B is a view for explaining an example of selecting sensor data in the real-time encoding apparatus of FIG.
4 is a block diagram illustrating an example of an image processing module included in the real-time encoding apparatus of FIG.
5 is a diagram showing an example of visualizing sensor data in the image processing module of FIG.
FIG. 6 is a block diagram illustrating an example of real-time encoding apparatus of FIG. 2 implemented as an algorithm.
7 is a flowchart illustrating a real-time encoding method according to embodiments of the present invention.

The description of the disclosed technique is merely an example for structural or functional explanation and the scope of the disclosed technology should not be construed as being limited by the embodiments described in the text. That is, the embodiments are to be construed as being variously embodied and having various forms, so that the scope of the disclosed technology should be understood to include equivalents capable of realizing technical ideas.

Meanwhile, the meaning of the terms described in the present application should be understood as follows.

The terms " first ", " second ", and the like are intended to distinguish one element from another, and the scope of the right should not be limited by these terms. For example, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component.

It is to be understood that when an element is referred to as being "connected" to another element, it may be directly connected to the other element, but there may be other elements in between. On the other hand, when an element is referred to as being "directly connected" to another element, it should be understood that there are no other elements in between. On the other hand, other expressions that describe the relationship between components, such as "between" and "between" or "neighboring to" and "directly adjacent to" should be interpreted as well.

It should be understood that the singular " include " or " have " are to be construed as including a stated feature, number, step, operation, component, It is to be understood that the combination is intended to specify that it does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

In each step, the identification code (e.g., a, b, c, etc.) is used for convenience of explanation, the identification code does not describe the order of each step, Unless otherwise stated, it may occur differently from the stated order. That is, each step may occur in the same order as described, may be performed substantially concurrently, or may be performed in reverse order.

All terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the disclosed technology belongs, unless otherwise defined. Commonly used predefined terms should be interpreted to be consistent with the meanings in the context of the related art and can not be interpreted as having ideal or overly formal meaning unless explicitly defined in the present application.

1 is a block diagram illustrating a mobile on-site monitoring system in accordance with embodiments of the present invention.

Referring to FIG. 1, the mobile field monitoring system 100 may include a real-time encoding device 110, a sensor 120, and a remote terminal 130. The real-time encoding device 110 and the remote terminal 130 may be connected via a network. The sensor 120 may be directly coupled to the real-time encoding device 110, or indirectly via a network. For example, the sensor 120 may be connected to the real-time encoding device 100 using a short-range wireless technology such as a wired cable, Bluetooth.

The real-time encoding device 110 can visualize the sensing data (or sensor data) provided from the sensor 120 by displaying the background image by photographing surrounding objects. For example, the real-time encoding device 110 displays the background image for the welding device, and the sensing data provided from the sensor 120 that senses the characteristics (e.g., current, voltage, pressure) Can be displayed on the image. Here, the sensing data may be displayed in a text form, a graph form, or the like.

Also, the real-time encoding device 110 merges the background image data for the background image and the sensing image data obtained by visualizing the sensing data into one image data, generates image data by encoding the image data, To the remote terminal 130. For example, the real-time encoding device 110 may combine the background image data and the sensing data to generate image data for the display image shown in FIG. 1 and use a general-purpose codec (for example, MPEG-4 / AAC) So that encoded data can be generated by encoding the video data and the audio data (and the caption data). Here, the audio data may include a user's voice using the real-time encoding device 110, a sound associated with the background image, and the caption data may include sensing values expressed in a text form. In this case, the remote terminal 130 can visually confirm the sensor data by decoding the encoded data and obtaining the image data, even if the remote terminal 130 does not have a separate application (i.e., a separate dedicated application for visualizing the sensing data) .

That is, the real-time encoding apparatus 110 according to embodiments of the present invention generates sensed image data by visualizing sensing data, generates encoded data by merging / coding the background image data and sensed image data, The receiving remote terminal 130 can confirm the sensing data without having a separate application.

In particular, the real-time encoding device 110 merges visualized sensing data (or sensing video data, caption data) in the background image data in real time and encodes the merged data using the universal codec to enable real-time streaming . In this case, the arbitrary remote terminal 130 can check the background image and the sensing values in real time using only general applications (for example, a general media player, a video call application, etc.).

Meanwhile, the function of visualizing the sensing data, the function of merging and encoding the background image and the visualized sensing data may be implemented by a real-time encoding program and included in the real-time encoding device 110.

The sensor 120 includes at least one of a temperature sensor, a humidity sensor, a voltage sensor (or voltmeter), a current sensor (or ammeter), a level sensor (or a height meter), a density measurement sensor can do. That is, the sensor 120 may include general measurement devices used to monitor a particular system.

The sensor 120 may acquire an analog type sensing signal (e.g., mA, mV), convert the sensing signal into digital sensing data, and provide the sensing data to the real-time encoding device 110. For example, the sensor 120 includes a signal converter (or an analog-to-digital converter (ADC) such as 4-20 mA to BLE, RS232-to-BLE, etc.) and may use Bluetooth technology.

The remote terminal 130 may be substantially the same as the real-time encoding device 110 and may be, for example, a smartphone, a tablet computer, a personal computer, or the like. However, the remote terminal 130 may not include a real-time encoding function or a real-time encoding program.

The remote terminal 130 can decode the encoded data provided from the real-time encoding apparatus 110 using the universal codec to acquire image data and audio data (and caption data). Also, the remote terminal 130 can output video data and audio data (and caption data) through a general application (for example, a general media player, a video call application). Accordingly, the user using the remote terminal 130 can confirm the sensing values even if the user does not have a separate dedicated application.

2 is a block diagram illustrating an example of a real-time encoding device included in the mobile field monitoring system of FIG.

2, it includes a sensing module 210, a camera module 220, an image processing module 230, a display module 240, a codec module 250, a communication module 260, and a microphone module 270 can do.

The sensing module 210 may receive sensing data from the sensor 120. The sensing module 210 may assign identification codes (e.g., IDs) to the sensing data provided from the plurality of sensors to distinguish the sensing data. Although not shown in FIG. 1, the sensing data may be stored in a storage module (e.g., a memory device).

The camera module 220 can acquire an image around the real-time encoding device 110 (i.e., a background image). The background image may include an image for a particular device (e.g., a welding device, a motor, etc.), or may include an image of a user using the real time encoding device 110. The camera module 220 is implemented by a general digital camera and can generate first image data (or background image data) for a background image.

The image processing module 230 visualizes the sensing data to generate second image data (or sensing image data), merges the first image data and the second sensing image data, and outputs the image data (or output image data) Can be generated.

The display module 240 can display an image based on the image data generated by the image processing module 230. Display module 240 also includes an input element (e.g., a touch screen) and may receive a touch input from a user.

FIG. 3 is a diagram illustrating an example of sensor data displayed in the real-time encoding apparatus of FIG. 2, and FIG. 4 is a diagram illustrating an example of selecting sensor data in the real-time encoding apparatus of FIG.

Referring to FIG. 3, the image processing module 230 generates a plurality of layers, arranges first image data and second image data on a plurality of layers, and the display module 240 corresponds to a plurality of layers (I.e., the image data in which the first image data and the second image data are merged) in the display areas R1 to R4.

Referring to FIG. 4, the display module 240 receives a user's sensing image type selection request (e.g., a touch input signal) through a menu icon, and correspondingly generates a plurality of sensing image types (i.e., (For example, a tachometer, a digital meter, and the like) of the sensing values to be displayed. Thereafter, the display module 240 may receive a user selection signal (e.g., a drag and drop touch input signal), and the image processing module 230 may display a display The position, size, and shape of the region can be determined. That is, the image processing module 230 can determine the number, position, size, shape, and the like of the layers.

Referring again to FIG. 3, the image processing module 230 (or the real-time encoding device 110) arranges the first image data (or the background image IMAGE corresponding to the first image data) in the first layer (I.e., a graph representing a change over time with a power value calculated on the basis of the voltage value and the current value) to the second to fourth layers (i.e., the layers overlaid on the first layer) (IMAGE2), voltage / current data (or current / voltage image IMAGE3), and pressure data (or pressure image IMAGE4). In this case, the background image IMAGE1 is displayed in the first area R1 of the display module 240 (or all display areas of the display module 240). A graphical power image IMAGE2 is displayed in the second area R2 of the display module 240 and a current / voltage image IMAGE3 is displayed in the third area R3 of the display module 240, A pressure image can be displayed in the fourth region R4.

3, three sensing images IMAGE2 to IMAGE4 are shown, but this is merely exemplary and the present invention is not limited thereto. For example, the real-time encoding device 110 may display one, two, or four or more sensing images.

Referring again to FIG. 2, the codec module 250 may encode the image data generated by the image processing module 230 to generate encoded data. As described above, the codec module 250 can encode video data using a general-purpose codec (MPEG-4 / AAC, H.264, etc.). In addition, the codec module 250 may encode the voice data provided by the microphone module 270. That is, the codec module 250 may encode video data and audio data to generate video data.

The communication module 260 can transmit the encoded data (or video data) to the remote terminal 130. The communication module 260 can also receive data (e.g., video data, etc.) from the remote terminal 130 (or an external device).

In one embodiment, the communication module 260 (or the codec module 250) may convert the encoded data into a digital voice stream and a video stream using a program such as FFMPEG (fast forward MPEG). In this case, the communication module 260 can provide the encoded data to the remote terminal 130 using the real-time streaming technology.

4 is a block diagram illustrating an example of an image processing module included in the real-time encoding apparatus of FIG. 5 is a diagram showing an example of visualizing sensor data in the image processing module of FIG.

4, the image processing module 230 may include a sensing data selector 410, a visualizer 420, an image synthesizer 430, and a textizer 440.

The sensing data selection unit 410 can select the sub sensing data to be visualized and textualized from the sensed data. For example, when the sensing data includes sub-sensing data for pressure, voltage / current, power, etc., the sensing data selector 410 may select a sensing value, a selection signal (for example, The sub-sensing data to be visualized and the sub-sensing data to be text can be selected, respectively.

3A, for example, the sensing data selection unit 410 determines power data and pressure data to be visualized, provides the power data and pressure data to the visualization unit 420, and the sensing data selection unit 410 ) Can determine the voltage / current data as a text to be written, and provide the voltage / current data to the textizer 440.

The visualization unit 420 can generate the second image data based on the sub-sensing data (i.e., the sensing data selected as the visualization target in the sensing data selection unit 410).

Referring to FIG. 5, the visualization unit 420 may display the sub-sensing data as a graph indicating a change in size of a sensing value over time. For example, the visualization unit 420 may store a sensing value for a specific time using a buffer or a memory device, and generate a graph based on the size of each of the sensing points and the providing time of each of the sensing values . For example, the visualization unit 420 includes a buffer capable of storing sensing values for 5 seconds, stores sensed values using a FIFO (first input first output) method, and stores the sensed values into a graph transformation algorithm The graph can be derived by applying it.

Alternatively, the visualization unit 420 may display the sub-sensing data as image data such as a tachometer. 3A, for example, the visualization unit 420 generates a guidance (or needle, indicator) image based on the sensed value, and outputs the guidance image as an external shape image having a shape of a tachometer such as a pressure image IMAGE4 Lt; / RTI >

That is, the visualization unit 420 may visualize the sensing data in various forms using a general technique (or algorithm) for visualizing the sensing data.

Meanwhile, the visualization unit 420 may generate the second image data with a period equal to the period of the first image data (i.e., the background image data). For example, when generating the first image data of 60 frames (or 60 Hz) in the camera module 120, the visualization unit 420 may generate the second image data of 60 frames.

The image combining unit 430 may generate image data by merging the first image data (i.e., data for the background image) and the second image data (i.e., the visualized sensing data). For example, the image synthesizer 430 may include a visualization layer (i.e., a plurality of layers including the visualized sensed data) using the OpneGL ES framework (i.e., a framework for processing 3D graphics in an embedded environment) And the first image data.

Meanwhile, the image data generated by the image combining unit 430 may be provided to the codec module 250.

The text creator 440 can generate text data (or character data) based on the sub-sensing data (i.e., the sensing data selected by the sensing data selector 410 as text data).

Referring to FIG. 3A, for example, the text converting unit 440 converts a basic character portion (for example, Voltage, Current representing the type of sensing value, and their units A), and determine a change character portion (e.g., 30.99, 3.000, which is the sensing value) corresponding to the sensing value (i.e., the sensing value in digital form). The text creator 440 can repeatedly generate subtitles with a specific period (for example, a period equal to or shorter than 60 frames, which is the period of video data).

In one embodiment, the textizer 440 may generate a subtitle if the change in the sensed value exceeds a threshold (or a reference value). For example, the text generating unit 440 may determine a unit of voltage to be a decimal point or more and generate a subtitle for a voltage only when the voltage changes to a decimal point or more (that is, 1V or more). For example, the textizer 440 can generate a subtitle for a current when the current changes by 0.1 A or more.

For reference, the real-time encoding device 110 may provide the remote terminal 130 with the encoded data (i.e., the data including the background image and the sensing data) as one media file. In this case, the real-time encoding device 110 may generate the caption data as a caption file having the same file name as the media file, and provide the generated caption file to the remote terminal 130.

Alternatively, the real-time encoding device 110 may provide the encoded data to the remote terminal 130 using a real-time streaming technique. In this case, the real-time encoding device 110 may periodically update the caption data and periodically provide the updated caption data to the remote terminal 130. [ The remote terminal 130 displays an image in real time, and can call the caption (i.e., sensing value) included in the updated caption data and display it on the image.

Alternatively, the real-time encoding device 110 may provide the encoded data to the remote terminal 130 using the video call technology. In this case, the real-time encoding device 110 can generate a text message using the caption data. In the video communication with the remote terminal 130, the remote terminal 130 can transmit a text message That is, a sensing value). Accordingly, the user using the remote terminal 130 can recognize not only the background image but also the sensing data in real time.

In the case where all the sensing data is visualized in the real-time encoding device 110, since the relatively small-sized sensing data is converted into relatively large (i.e., large-capacity) image data, The size of data provided to the terminal 130 increases and the load of both the real-time encoding device 110, the network 120, and the remote terminal 130 can be increased.

Meanwhile, when at least a part of the sensing data is visualized in a character form using the caption data (or a text message) in the real-time encoding device 110, the sensing data is relatively small in size (Or character data) having the same size), it is possible to minimize the increase in the size of data provided to the remote terminal 130 in the real-time encoding apparatus 110. [

FIG. 6 is a block diagram illustrating an example of real-time encoding apparatus of FIG. 2 implemented as an algorithm.

2 and 7, the real-time encoding apparatus 110 includes a sensor data integration unit (sensor DAQ) 610, a gauge droper 620, a gauge viewer 630, a surfacetexture 640, an openGL ES 650, a surfaceview 660, a media codec 670, and an FFMPEG 680 .

The sensor data integration unit 610 may acquire sensor values (or sensor data) from the sensors 120 and may assign identification numbers (or IDs) to the sensors to distinguish the plurality of sensors have. May be substantially the same as the sensor module 210 described above with reference to FIG.

A gauge droper 620 can visualize sensor values in various forms. And may be substantially the same as the visualization unit 420 described with reference to FIG.

A gauge viewer 630 may control the sensor visualization layer. Here, the sensor visualization layer may correspond to a layer that displays visualized sensor values. As described above with reference to FIG. 3B, the gauge viewer 630 can create, move, and delete the sensor visualization layer based on a selection signal provided from the user.

The surfacetexture 640 may obtain first image data (or image frame data) from the camera module 220.

OpenGL ES 650 is a framework for processing three-dimensional graphics in an embedded environment and is a framework for rendering first graphics data and second graphics data to FFMPEG 680 for screen rendering and codec conversion, Surface (EGLSurface) can be included.

The surface viewer 660 may mix the visualization layer (i.e., the visualized sensing data) and the camera image frame (i.e., the first image data) via the OpenGL ES framework and output it on the display module 240 .

The media codec 670 may encode video data (i.e., data including the first and second video data) using a codec that can be reproduced in a mobile environment. The media codec 670 may be substantially the same as the codec module 250 described with reference to FIG.

 FFMPEG (680) is a cross-platform supporting media framework that can provide various codecs, image playback, and editing functions. The FFMPEG 680 stores video data encoded in the media codec 670 (i.e., data including video data and audio data) as a video stream, and outputs a file output (e.g., mp4-latm) or a streaming output For example, the rtsp method can be used.

7 is a flowchart illustrating a real-time encoding method according to embodiments of the present invention.

Referring to Figures 2 and 7, the method of Figure 7 may be performed in the real-time encoding device 110 of Figure 2.

7 may generate the second image data by visualizing the sensing data provided from the sensor 120 (S710).

As described above with reference to FIG. 5, the method of FIG. 7 may convert the sensed data into a graph image, a tachometer image, or the like that represents a sensing value change over time using the visualization unit 420.

In one embodiment, the method of FIG. 7 may generate caption data by textizing at least a portion of the sensing data provided from the sensor 120.

As described above with reference to FIG. 5, the method of FIG. 7 selects sub-sensing data to be converted into text data from sensing data and converts a sensing value (digital sensing value) into a character form to generate a subtitle at a specific point in time have.

As described above, when the method of FIG. 7 provides the encoded data to the remote terminal 130 using the real-time streaming technique, the method of FIG. 7 periodically updates and transmits the caption data, and the remote terminal 130 The caption (i.e., the sensing value) included in the updated caption data can be called up and displayed on the video. 7, when the method of FIG. 7 provides the encoded data to the remote terminal 130 using the video communication technology, the method of FIG. 7 generates a text message using the caption data, During the communication, the remote terminal 130 can display a text message (i.e., a sensing value) by using the push notification function.

 7, the first image data provided by the camera module 220 may be merged with the second image data to generate output image data (S720). As described above, the method of FIG. 7 uses a visualization layer (i.e., a plurality of layers including visualized sensed data) and an OpneGL ES framework (i.e., a framework for processing 3D graphics in an embedded environment) The first image data can be synthesized.

The method of FIG. 7 generates encoded data by encoding the output image data (S730), and may transmit the encoded data to the remote terminal 130 (S740).

As described above, the method of FIG. 7 can encode video data (i.e., video data, audio data, caption data, etc.) using a general-purpose codec (MPEG-4 / AAC, H.264, etc.). 7, the encoded data is converted into a digital voice stream and a video stream by using a program such as FFMPEG (fast forward MPEG), and the encoded data is provided to the remote terminal 130 using the real-time streaming technology .

As described above, in the real-time encoding method according to the embodiments of the present invention, the second image data is generated by visualizing the sensing data, and the first image data and the second image data are merged / The remote terminal 130 receiving the encoded data can display the sensing data (for example, the sensing value) even without the separate application.

In particular, the method of FIG. 7 realizes streaming by merging sensed data visualized in the first image data (or second image data, caption data) in real time, and encoding the merged data using the universal codec . Accordingly, the arbitrary remote terminal 130 can acquire / display background images and sensing values in real time using only general applications (for example, a general media player, a video call application, etc.).

The real-time encoding apparatus and the real-time encoding method according to embodiments of the present invention can be applied to a video data encoding apparatus and a real-time monitoring system.

100: Mobile field monitoring system
110: Real-time encoding device 120: Sensor
130: Remote terminal 210: Sensor module
220: camera module 230: image processing module
240: Display module 250: Codec module
260: communication module 270: microphone module
410: Sensing data selection unit 420: Visualization unit
430: image synthesizing unit 440:

Claims (7)

A camera module for generating first image data;
A sensing module for receiving sensing data from an external sensor;
An image processing module for generating second image data by visualizing the sensing data, and merging the first image data and the second image data to generate output image data;
A display module for displaying an image based on the output image data;
A codec module for encoding the output image data to generate encoded data; And
And a transmission module for transmitting the encoded data to an external device,
The sensing data includes sub-sensing data for each of temperature, pressure, level, voltage and current,
The image processing module includes:
A sensing data selector for selecting the first sub sensing data and the second sub sensing data among the sub sensing data, respectively;
A visualization unit for generating the second image data based on the first sub-sensing data;
An image synthesizer for merging the first image data and the second image data to generate the output image data; And
The method of claim 1, further comprising: periodically extracting a sensing value included in the second sub-sensing data, generating caption data based on a time at which the sensing value is extracted and the sensed value, And a dextreater for updating the caption data,
Wherein the transmission module transmits the caption data to the external device independently of the encoded data,
A background image corresponding to the first image data is displayed in a first area of a display area of the display module, a sensing image corresponding to the second image data is displayed in a second area of the display area, Is displayed in the third area of the display area,
Wherein the first region, the second region, and the third region do not overlap with each other. delete The real-time encoding apparatus of claim 1, wherein the transmission module generates a text message based on the caption data and transmits the text message to the external device. The method according to claim 1,
Further comprising a memory device for storing the accumulation data in which the sensing data is accumulated,
Wherein the image processing module generates the second image data using the accumulation data corresponding to the object section including the time at which the sensing data is generated. delete delete In a real-time encoding method performed in a real-time encoding device interlocked with a camera and a sensor,
Generating second image data by visualizing sensed data provided from the sensor;
Generating output image data by merging first image data and second image data provided from the camera;
Encoding the output image data to generate encoded data; And
And transmitting the encoded data to an external device,
The sensing data includes sub-sensing data for each of temperature, pressure, level, voltage and current,
Wherein the step of generating the output image data comprises:
Selecting first sub sensing data and second sub sensing data among the sub sensing data, respectively;
Generating the second image data based on the first sub-sensing data;
Generating the output image data by merging the first image data and the second image data; And
The method of claim 1, further comprising: periodically extracting a sensing value included in the second sub-sensing data, generating caption data based on a time at which the sensing value is extracted and the sensed value, And updating the caption data,
Wherein the step of transmitting the encoded data to the external apparatus includes transmitting the caption data to the external apparatus independently of the encoded data,
The background image corresponding to the first image data is displayed in the first area of the display area of the display module and the sensing image corresponding to the second image data is displayed in the second area of the display area, The corresponding caption is displayed in the third area of the display area,
Wherein the first region, the second region, and the third region do not overlap with each other.

Images