KR101986681B1 - Apparatus for processing images and method thereof - Google Patents

Abstract

The present invention relates to an image processing apparatus and a method thereof, and more particularly, A communication unit for communicating with an external device; An image processing unit for processing an image; And a control unit for determining whether the camera unit photographs the image on the basis of the image processing setting information provided by the user and the user's state information provided from the apparatus for monitoring the user's state and editing and storing the photographed image .

Description

[0001] APPARATUS FOR PROCESSING IMAGES AND METHOD THEREOF [0002]

The present invention relates to an image processing apparatus and a method thereof, and more particularly, to an image processing apparatus and method that can provide an image of a user's interest in consideration of the state of a user.

With the development of portable electronic devices, it is possible to capture various pictures and images, and the speed of technology development of image processing devices such as digital cameras, smart phones, and TVs is progressing very rapidly. Particularly, the development of the drones further develops the user's interest in the image shooting.

Generally, a user shoots a desired image by pressing a photographing button provided on the image processing apparatus or by touching the touch screen. In addition, a desired image is photographed using a tool such as a cellar, or the operation of the image processing apparatus is controlled using a remote controller or the like.

Since the user needs to directly control the user in order to shoot the desired image, the user often needs a quick power to shoot the desired image. If the user does not touch the shooting button at that moment or touches the desired image, There is a problem that it is difficult to shoot an image. In addition, when recording is performed for a long period of time, it is troublesome to edit only the image of interest among the recorded images separately, and there is a problem that the storage space becomes short due to a large capacity of the recorded image.

The present invention relates to an image processing apparatus and method for automatically capturing an image desired by a user.

The present invention also relates to a method and apparatus for image processing that can capture an image of a user's interest in consideration of the state of a user.

The present invention also relates to an image processing apparatus and a method thereof, which can edit a photographed image and provide it to a user.

The present invention relates to an image sensing apparatus, A communication unit for communicating with an external device; An image processing unit for processing an image; And a controller for determining whether or not the camera unit photographs the image based on the image processing setting information provided by the user and the user's state information provided from the apparatus for monitoring the state of the user and editing and storing the photographed image Device.

Here, the apparatus may further include a display unit for displaying an image, wherein the control unit associates the photographed image with the image displayed on the display unit for a time period during which the image is photographed by the camera unit, It is possible to control the processing section.

In addition, the control unit may control the camera to photograph the additional image according to whether the condition of the user satisfies a predetermined condition.

In addition, the image processing setting information may include at least one of a start point, an end point, and a photographing time for photographing an image in the camera unit.

In addition, the user's state information may include at least one of the biometric information of the user and the motion information of the user.

According to another aspect of the present invention, there is provided a monitoring device comprising: a monitoring device for monitoring biometric information of a user using at least one sensor; And an image processor for capturing an image based on information for capturing an image provided by a user and biometric information of the user received from the monitoring device and displaying the image to the user,

The image processing apparatus edits the photographed image and the displayed image so that the image displayed on the display unit and the image photographed by the camera unit correspond to each other in a predetermined time unit for a time period during which the camera unit captures an image, Thereby providing an apparatus for generating an album.

Since the present invention automatically captures an image in consideration of the state of a user, it is possible to capture a scene in which a user has caused an interest without fail.

In addition, the edited image album is provided by linking the photographed image with the displayed image, and the image can be conveniently photographed because no separate image editing is required.

In addition, since the image is captured, edited, and stored for a time period during which the user is interested, storage space of the image data can be reduced.

BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a diagram showing a configuration of an image processing system according to an embodiment of the present invention; Fig.
2 is a diagram showing a configuration of a monitoring apparatus according to an embodiment of the present invention;
3 is a diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention.
4 is a flowchart illustrating an operation of a monitoring apparatus according to an embodiment of the present invention.
5 is a flowchart illustrating an operation of an image processing apparatus according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

1 is a diagram showing a configuration of an image processing system 10 according to an embodiment of the present invention.

Referring to FIG. 1, the image processing system 10 of the present invention includes a monitoring apparatus 100, an image processing apparatus 110, and a server 120.

The monitoring apparatus 100 transmits image processing setting information to the image processing apparatus 110 so that image capturing is performed in the image processing apparatus 110 according to the setting information. At this time, the image processing setting information includes information for photographing a video such as a starting point, a photographing time, and a shooting end point of photographing the image in the image processing apparatus 110, And various information necessary for image processing such as information about image albums such as whether to configure them or color them.

In addition, the monitoring apparatus 100 monitors the status of the user and transmits the status information of the user to the image processing apparatus 110 so that the image processing apparatus 110 can grasp the status of the user. At this time, the user's state information includes user's biometric information such as heart rate, body temperature, and the like, motion information of the user such as the degree of movement of the user, the moving speed, and the moving direction.

The monitoring device 100 may be configured as a portable device and may be configured to allow the user to contact the body. For example, the monitoring device 100 can be configured as a variety of devices such as a smart watch, a headset, etc., which are portable and easy to attach and detach and can be used in contact with the body.

The image processing apparatus 110 captures an image according to predetermined image processing setting information, and the image processing setting information can be received from the monitoring apparatus 100 or directly input from the user. The image processing apparatus 110 may be a smart device capable of photographing or displaying images such as a smart TV, a smart phone, a smart pad, a digital camera, and a head mounted display (HMD). The image processing apparatus 110 may store the photographed image information or transmit the photographed image information to the server 120.

For example, the present invention divides a user's state into a normal state and an excited state, and allows the user to take an image when the user is in an excited state. The state of the user can use biometric information such as a user's heart rate. Accordingly, if the user's heart rate is monitored and the heart rate is equal to or greater than the predetermined value, the user's state is determined to be in the excited state, and the image processing apparatus 110 photographs the user and ends the image photographing when the user's state is normal. At this time, the photographing time can be set so that the photographing is progressed for a predetermined time.

In determining the state of the user, motion information of the user other than the user's heart rate may be used. For example, if the user moves more than a normal motion by more than a predetermined amount, the user can be determined to be in an excited state.

The server 120 may store the image information received from the image processing apparatus 110 and may transmit the image information to an electronic device such as the monitoring apparatus 100 that the user can confirm the image to provide the stored image information to the user.

FIG. 2 is a diagram showing a configuration of a monitoring apparatus 100 according to an embodiment of the present invention.

2, the monitoring apparatus 100 includes a sensor unit 220, a communication unit 240, and a control unit 260. The monitoring unit 100 includes an input unit 200, an output unit 210, a detachable unit 230, And a display unit 250, as shown in FIG.

The input unit 200 has a function of converting a physical input from the outside into a specific electric signal, and can receive a user input such as touch, push operation, voice, etc. of the user. The input unit 200 may be implemented using various types of buttons, a touch sensor for receiving a touch input, a proximity sensor for receiving an approaching motion, and the like. Also, the input unit 200 can receive image processing setting information from a user.

The output unit 210 is for providing a notification to a user and allows the user to hear, feel, or visually confirm such as sound, vibration, light, and the like.

The sensor unit 220 is for sensing the state of the user and includes various sensors. For example, the sensor unit 220 may include a bio-signal detection sensor for detecting a bio-signal such as a heartbeat or a body temperature of the user, a degree of movement of the user such as an acceleration sensor, a gyro sensor, a gravity sensor, , A movement direction, and the like. In addition, a position sensor such as a GPS sensor for confirming the position of the user may be included.

The detachable unit 230 allows the user to attach and detach the monitoring device 100 to the body so that the user can conveniently carry the monitoring device 100.

The communication unit 240 is for communicating with other devices such as the image processing apparatus 110 and specifically includes a wireless Internet unit (not shown), a broadcast transmission / reception unit (not shown), a mobile communication unit (not shown) ), And a wired communication unit (not shown). The wireless Internet accesses the wireless Internet to acquire or transmit data. The wireless Internet that can be accessed through the wireless Internet unit includes a wireless LAN (WLAN), a wireless broadband (WIBRO), a wireless interoperability for microwave access (WIMAX) Downlink Packet Acess). The broadcast transmitting and receiving unit transmits and receives broadcast signals through various broadcasting systems. The broadcasting system that can transmit and receive through the broadcasting transmitting and receiving unit includes Digital Multimedia Broadcasting Terrestrial (DMBT), Digital Multimedia Broadcasting Satellite (DMBS), MediaFLO (Media Fowell Link Only) Digital Video Broadcast Handheld (DVBH), Integrated Services Digital Broadcast Tereestrial (ISDBT), and the like. The broadcast signal transmitted / received through the broadcast transmission / reception unit may include traffic data, living data, and the like.

The mobile communication unit can access and communicate with a mobile communication network according to various mobile communication standards such as 3rd Generation (3G), 3rd Generation Partnership Project (3GPP), and Long Term Evolution (LTE). The short-range communication unit is for short-range communication, and includes Bluetooth, Radio Frequency Identification (RFID), Infra Data Association (IrDA), Ultra WidBand (UWB), ZigBee, Near Field Communication (NFC) Lt; / RTI > The wired communication unit is an interface device that can be wired to another device, and may be a USB module capable of communicating through a USB port.

The display unit 250 displays various data, such as displaying status information of a user, displaying image information, and the like. The display unit 250 may include a touch screen.

The control unit 260 may transmit the image processing setting information input from the user to the image processing apparatus 110 or may transmit the status information of the user monitored by the sensor unit 220 to the image processing apparatus 110, And controls the inventive monitoring device 100 as a whole.

3 is a diagram showing a configuration of an image processing apparatus 110 according to an embodiment of the present invention.

3, the image processing apparatus 110 includes a camera unit 310, a communication unit 330, a display unit 320, an image processing unit 340, and a control unit 360. The image processing apparatus 110 includes an input unit 300, (350). ≪ / RTI >

The input unit 300 has a function of converting a physical input from the outside into a specific electric signal, and can receive a user input such as touch, push operation, voice, etc. of the user. The input unit 200 may be implemented using various types of buttons, a touch sensor for receiving a touch input, a proximity sensor for receiving an approaching motion, and the like. Also, the input unit 200 can receive image processing setting information from a user.

The camera unit 310 may capture an image of a user viewing the displayed image under the control of the controller 360 or an environment around the user.

The display unit 320 displays various images. For example, the display unit can display a broadcast-related image such as a movie, a drama, or a documentary, a photograph or a moving image captured by the user, or a variety of images downloaded from the outside via the Internet . The display unit 250 may include a touch screen.

The communication unit 330 is for communicating with other devices such as the monitoring device 100 and the specific configuration may include the same or similar configuration as the communication unit 330 of the monitoring device 100 described above.

The image processing unit 340 processes the image photographed by the camera unit 310 or the image displayed on the display unit 320 and stores the processed image in the storage unit 350 or the server 120. For example, the image processing unit 340 processes a video image captured by the camera unit 310 and an image displayed on the display unit 320 to generate a video album by overlapping or overlapping the displayed video image with the captured user image can do. At this time, the image processing unit 340 may associate the displayed image with the photographed image on a time unit basis. For example, the image processed by the photographed image from the time the image was photographed until the photographed time, . In addition, the image processing unit 340 may form a still image or a moving image.

The storage unit 350 stores various data and applications. The storage unit 350 may include a random access memory (RAM), a flash memory, a ROM (Read Only Memory), an EPROM (Erasable Programmable ROM), an Electrically Erasable and Programmable ROM (EEPROM) A removable disk, a removable disk, a memory card, a Universal Subscriber Identity Module (USIM), and the like, as well as a detachable form of storage such as a USB memory.

The control unit 360 controls the image processing apparatus 110 of the present invention such as generating image albums in the image processing unit 340 using the image processing setting information and the user status information. For example, when the user's state is determined to be in the excited state, the control unit 360 causes the camera unit 310 to start photographing the user, and ends the photographing when the user's state is normal. (340).

In addition, the image displayed on the display unit 320 is transmitted to the image processing unit 340 during a period from the time when the camera unit 310 starts shooting to the end point, and the image processing unit 340 generates the image album. At this time, the image album may include information to know the situation at the time of shooting the image. For example, the image album may include various information such as location information, time information, user status information, and weather information.

4 is a flowchart illustrating an operation of the monitoring apparatus 100 according to an embodiment of the present invention.

Referring to FIG. 4, when the user inputs image processing setting information, the controller 260 transmits the inputted image processing setting information to the image processing apparatus 110 (400). Thereafter, the user's status such as the user's heart rate and motion is monitored 410, the status information of the user is transmitted to the image processing apparatus 110, and the status information of the user is continuously transmitted or transmitted in units of a predetermined time 420 (420) .

Meanwhile, the controller 260 can determine whether the user is in the normal state or excited state. For example, if the user's heart rate is equal to or greater than the predetermined value, the controller 260 determines that the user is in the excited state. If it is determined that there is a user's state change (e.g., YES in 430), the user changes state from the normal state to the excited state or from the excitement state to the normal state, the control unit 260 transmits the user's state change information to the image processing apparatus 110 (440). At this time, the controller 260 may selectively determine whether the user's state has changed.

5 is a flowchart illustrating an operation of the image processing apparatus 110 according to an embodiment of the present invention.

Referring to FIG. 5, when receiving image processing setting information from the monitoring apparatus 100 or receiving image processing setting information from a user, the controller 360 confirms image processing setting information (500). Thereafter, upon receiving the user status information from the monitoring apparatus 100, the status change of the user is determined (510). If it is determined that the user's state changes from the normal state to the excited state, or from the excitement state to the normal state (520 in the example), the image processing is performed according to the image processing setting information, (530), and stores image processing information such as a video album (540). At this time, the image processing information may be transmitted to the server 120.

Alternatively, the image processing apparatus 110 may determine whether the user's state is changed. For example, when the user's state change information is received from the monitoring apparatus 100, the image processing apparatus 110 does not judge it separately It is possible to determine whether or not the image is photographed based on the state change information of the received user.

Hereinafter, various embodiments to which the above-described contents are applied will be described.

Hereinafter, the user carries the monitoring apparatus 100, determines the change of the user's state by using the user's heart rate, and determines that the user is in an excited state when the user's heart rate is 90 or more. It is also assumed that the image capturing is started when the user is in the excited state and the image capturing is terminated when the user is in the normal state.

First Embodiment

When the user watches a movie through the smart TV 110, the monitoring device 100 monitors the user's heart rate and transmits it to the smart TV 110. When the user's heart rate is equal to or higher than 90, the smart TV 110 starts to shoot the user and ends the shooting when the user's heart rate is less than 90. Then, a video album is generated by processing the displayed image and the photographed user's image so that they correspond to each other on a time basis during a time period during which the video is photographed. The video album is composed of a mixture of a moving image and a still image.

The created image album is saved or transmitted to the server. At this time, the image album may include location information of the user, time information, movie information to be displayed, heart rate information, and the like. This allows you to see the scene in which the user reacted in the movie, and you can check the status of the user. For example, in the case of a horror movie, it is possible to know in which scene the user feared the fear, and in the case of a comedy movie, it is possible to grasp which scene the user laughs has been caused.

Second Embodiment

When the user is fishing and capturing a user through the smartphone 110, the monitoring device 100 monitors the heart rate of the user and transmits the monitored heart rate to the smartphone 110. When the user requests the bait and the user moves rapidly or the user's heart rate is 90 or more, the smartphone 110 starts to shoot the user and ends the shooting when the user's heart rate is less than 90 or the user's motion becomes normal. After that, the user's image can be processed on a time-by-time basis to generate a video album, and the image album can include the user's location information, time information, heart rate information, and the like.

At this time, a vibration sensor is provided on the fishing fountain, a vibration sensor transmits a signal to the monitoring device 100, and the monitoring device 100 notifies the user so that the user can react, To create a video album.

Third Embodiment

When the user is in operation and the surroundings of the vehicle are photographed through the black box 110, the monitoring device 100 monitors the user's heart rate and transmits it to the black box 110. When the user's heart rate is equal to or higher than 90, the black box 110 starts to shoot the surrounding environment, and when the user's heart rate becomes less than 90, the shooting ends and a video album is generated. When the black bar 110 is performing real-time recording, a video album is created using the photographed image when the user's state is in the excited state.

Fourth Embodiment

When the user wears the Google glass 110 and walks away, the monitoring device 100 monitors the user's heart rate and transmits it to the Google glass 110. When the user's heart rate is 90 or more, the Google glass 110 starts to photograph the surrounding environment, and when the user's heart rate is less than 90, the shooting ends and a video album is created. At this time, the monitoring apparatus 100 may be formed integrally with the Google glass 110, and thus all operations are performed on the Google glass 110.

While the invention has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention.

Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined by the scope of the appended claims, and equivalents thereof.

100: monitoring device 110: image processing device
120: Server

Claims (12)

An image processing apparatus comprising:
A camera unit for capturing an image;
A communication unit for communicating with an external device;
An image processing unit for processing an image;
A display unit for displaying an image;
And a controller for determining whether or not the camera unit photographs an image based on image processing setting information provided from a user and state information of the user provided from an apparatus for monitoring the state of the user,
Wherein the control unit controls the image processing unit to generate a video album by editing the video so that the video displayed on the display unit and the video shot by the camera unit correspond to each other at predetermined time intervals during the time of shooting the video in the camera unit, Control,
Wherein the image displayed on the display unit and the image captured on the camera unit are different from each other, and the image album includes both the image displayed on the display unit and the image captured on the camera unit.
The method according to claim 1,
Wherein the image photographed by the camera unit is a photographed image of the user.
The method according to claim 1,
Wherein the controller is configured to shoot the camera additional image according to whether the user's condition satisfies a predetermined condition.
The method according to claim 1,
Wherein the image processing setting information includes at least one of a start point, an end point, and a photographing time information for photographing an image in the camera unit.
The method according to claim 1,
Wherein the status information of the user includes at least one of biometric information of the user and motion information of the user.
In an image processing system,
A monitoring device for monitoring biometric information of a user using at least one sensor;
And an image processing apparatus for photographing an image using a camera unit based on information for capturing an image provided by a user and biometric information of the user received from the monitoring apparatus and displaying the image to the user using the display unit In addition,
Wherein the image processing apparatus generates a video album by editing the video so that the video displayed on the display unit and the video shot by the camera unit correspond to each other in a predetermined time interval during the time of shooting the video in the camera unit,
Wherein the image displayed on the display unit and the image captured on the camera unit are different from each other, and the image album includes both the image displayed on the display unit and the image captured on the camera unit.
In the image processing method,
Receiving image processing setting information from a user;
Receiving status information of the user from a device monitoring the status of the user;
Determining whether or not the image is captured based on the status information of the user;
And generating a video album by editing the video so that the displayed video and the photographed video are associated with each other in a predetermined time unit during the video shooting time,
Wherein the displayed image and the photographed image are different, and the image album includes both the displayed image and the photographed image.
8. The method of claim 7,
Wherein the photographed image is a photographed image of the user.
8. The method of claim 7,
Wherein the step of determining whether or not the image capturing is to be performed is determined according to whether a state of the user satisfies a predetermined condition.
8. The method of claim 7,
Wherein the image processing setting information includes at least one of a start point, an end point, and a photographing time for photographing the image.
8. The method of claim 7,
Wherein the status information of the user includes at least one of the biometric information of the user and the motion information of the user.
12. A storage medium on which a program for executing the method of any one of claims 7 to 11 is recorded.

Images