KR101082845B1 - Image providing system for smart phone using ip camera - Google Patents

Abstract

PURPOSE: A smart phone image supplying system through an IP camera is provided to smoothly express an image in a form of a multi layout on a smart. CONSTITUTION: An image processing unit(22) fills image data, which is received from an IP camera, in a buffer. The image processing unit converts the image data into image. The image processing unit displays the converted image on a display unit. The image processing unit converts the converted image based on a value that is stored in a buffer.

Description

Image Providing System For Smart Phone Using IP Camera}

The present invention relates to a smartphone video providing system using an IP camera, and more specifically, it is possible to provide the image information generated by the IP camera equipped with a wireless modem of various methods such as 3G, 4G, Wi-Fi to the smartphone. The present invention relates to a technology capable of smoothly displaying images in a multi-layout form on a smartphone having limited resources.

In home network systems, material warehouses, and security facilities, IP cameras are installed at various points, and images acquired through IP cameras are stored in storage devices, and the images are read in case of emergency and used for crime tracking.

In recent years, technology for transmitting images captured in real time using an IP camera having an Internet communication function is commercialized.

In particular, in the conventional home network service, a technology for monitoring an area to be monitored even at a remote location is applied by providing an image captured by an IP camera to a mobile communication terminal at a remote location through a wireless LAN.

However, in recent years, mobile communication terminals have been rapidly replaced by smart phones, and the single mode layout displaying a single view screen in applying the remote video monitoring method to smart phones according to the functional characteristics of the smart phone operating as a small computer. In the video display, there is no obstacle, but in a multi-mode layout that displays multiple view screens at the same time, there is a problem in that it cannot provide a smooth multi-view due to limited resources of the smartphone.

In addition, since the remote video surveillance apparatus using the conventional IP camera only supports the wireless LAN, it is not applicable to the 3G or 4G modem which is a mobile communication network.

The present invention has been made to solve the above problems, an object of the present invention to the smart phone to the image information generated through the IP camera equipped with a wireless modem of various methods such as 3G, 4G, WiFi It is possible to provide a photographed image, and to provide a technology for smoothly displaying images in a multi-layout form on a smartphone with limited resources.

Another object of the present invention is to attach the tag to the thing, and to mount the RFID module in the IP camera so that the tag information is displayed on the portion where each object is located in the captured image to enable effective remote goods management.

According to an aspect of the present invention for achieving the above object, a modem device including any one of a mobile communication modem, a wireless LAN modem, a wireless portable Internet modem, an image pickup device for photographing the surrounding images, the image pickup device An image processing unit for processing an image, a touch screen for inputting control signals and operation control, a serial interface terminal for accessing the modem device, a mobile communication modem, a wireless LAN modem, and a wireless portable Internet modem are connected to the serial interface. IP camera including a multiple modem driver for driving a modem device and a smartphone connected to the IP camera via the modem device to display the image taken by the IP camera smart phone using an IP camera An image providing system is provided.

The apparatus may further include a fuselage recognition sensor for detecting a movement of a surrounding object, and the IP camera zooms in a position where the movement of the object is detected when an amount of movement of an object equal to or greater than a set value is detected based on the detection signal of the fuselage recognition sensor. It is preferable to transmit an image marked with a motion around the detected position to the smartphone.

The smartphone may include a display unit for displaying an image obtained from the IP camera, an input unit for receiving camera selection information from the user, and control information for zoom, pan, and tilt of the selected camera, and the IP cameras stored by the user. A storage unit for storing access information, and temporarily stores the image (H.264, MJPEG) data, a front buffer for storing the entire image (H.264, MJPEG) data, and previously input to the front buffer A buffer unit including a rear buffer for storing only changed parts by comparing stored image (H.264, MJPEG) data and currently input image (H.264, MJPEG) data, and dynamically selecting multi-view layout threads when selecting multiple images. Creates and fills the image data (H.264, MJPEG) data received from the IP camera into the buffer for each thread created, and converts the image into an image. And an image processor for repeatedly displaying the data in the ray unit, wherein the image processor changes only data changed from image (H.264, MJPEG) data stored in the front buffer based on a value stored in the rear buffer. It is desirable to display on the display unit.

The image processor converts the input image (H.264, MJPEG) data into bitmap data, converts the converted bitmap data according to the size of the view layout, and converts the converted image into a bitmap (H.264, MJPEG). 264, MJPEG) data is preferably displayed on the display unit by a set number of frames per second.

The image processing unit may more preferably display an image at a first frame rate in a single view layout state and an image at a second frame rate lower than the first frame rate in a multi view layout.

In addition, when a specific layout is touched while displaying an image at the second frame rate in the multi-view layout, the image processor displays the image of the layout at the first frame rate, and the layout displayed at the first rate is displayed. In case of long touch, it is also possible to change to single view layout.

In addition, a plurality of tags are installed at a specific location in an area photographed by the IP camera, and the IP camera further includes an RFID module for reading tag information recorded in the tag, wherein the IP camera is a photographed image. More preferably, the tag information is displayed in a text or graphic form on a portion where the tag is located.

According to the present invention, it is possible to provide the video information generated by the IP camera equipped with a wireless modem of various methods such as 3G, 4G, Wi-Fi to the smart phone has a very excellent effect.

In addition, it is possible to maximize the resource usage efficiency of limited smartphones through techniques such as variable frame rate and double buffering.

In addition, by mounting the RFID module in the IP camera, it is possible to know the management information of the goods located in the image to be displayed remotely, there is an effect that can be effective goods management.

1 is an overall configuration diagram of a smartphone video providing system using an IP camera according to the present invention.
FIG. 2 is a flowchart illustrating an internal configuration of the IP camera of FIG. 1.
3 is a flowchart illustrating an internal configuration of the smartphone of FIG. 1.
4 exemplarily illustrates a camera list providing screen of a smartphone.
5 shows an example of a single view layout.
6 shows an example of a multiview layout.

Hereinafter, with reference to the accompanying drawings will be described in detail a preferred embodiment of the present invention.

1 is an overall configuration diagram of a smartphone video providing system using an IP camera according to the present invention, Figure 2 is a flow chart showing the internal configuration of the IP camera of FIG.

As shown in FIG. 1, the smart phone image providing system using the IP camera according to the present invention has a fuselage recognition sensor 10 installed in connection with the IP camera 20 and the IP camera 20 installed in the monitoring target 1. ) And a modem device 30 connected to the IP camera 20 through a serial interface.

IP camera 20 is to shoot the image of the monitoring target area and transmit the captured image to the smartphone (2) of the administrator located in the remote location through the modem device 30, in addition to the wireless LAN modem 3G, 4G, WiBro, etc. It is possible to provide various wireless communication formats.

 To this end, as shown in FIG. 2, the IP camera 20 includes an image pickup device 21 for capturing a peripheral image, an image processor 22 for processing a captured image, and a touch screen for inputting control signals and controlling operations. 25, a multiple modem driver for driving a modem device connected to the serial interface unit 24 by supporting a serial interface unit 24 for connecting a modem device and a mobile communication modem, a wireless LAN modem, and a wireless portable Internet modem ( 23).

Here, the multi-modem driver 23 is implemented as firmware capable of supporting wireless communication standards such as 3G, 4G, mobile communication, WiBro, Wi-Fi, etc., and supports a tunnel tunneling and security by supporting a VPN and independently of a hardware interface. It is desirable to enable operation.

In addition, the serial interface unit 24 is for attaching and detaching a modem having a serial interface terminal. As shown in FIG. 2, a 3G modem, a 4G modem, a WiBro modem, a WLAN modem, and the like may be detachably connected. . The serial interface terminal is preferably a USB terminal.

In addition, the fuselage recognition sensor 10 is for detecting the movement of the surrounding object, and various sensors such as an infrared sensor and an ultrasonic sensor may be used, and the degree of motion detection may be adjusted according to the characteristics of a time zone or a monitoring area. For example, in areas or areas where there is a lot of human motion, the sensitivity of the sensor is lowered to lower the detection threshold value for judging an abnormal situation. It is desirable to be able to detect it immediately when it occurs.

The IP camera 20 is connected to the fuselage recognition sensor 10, and when an object moving amount of a set value or more is detected based on the detection signal of the fuselage recognition sensor 10, a push message indicating an intrusion occurrence or an emergency situation is detected by the administrator's smartphone. In addition to (2), the zoom-in and marking the image of the position where the movement of the object is detected and transmits to the smartphone. Preferably, a moving body tracking algorithm is applied to provide an image while tracking a moving object, and the marking process may highlight a moving object portion in a rectangle or a circle.

On the other hand, in the case where the surveillance target area is a goods management area such as a warehouse of goods, the RFID module 26 capable of attaching the tag T to the goods to be managed and reading the tag T in the IP camera 20 is provided. It is preferred to be provided.

In this case, the IP camera 20 reads the article information recorded on the tag T through the RFID module 26, that is, the tag information on the portion where the tag T is located in the captured image. Or it is possible to display in graphical form. Through this, the manager has a merit that it is possible to smoothly manage the inventory of the goods even at a remote location by grasping the status of the goods or the management information while looking at the monitoring screen beyond simply monitoring the shooting screen of the monitored area.

3 is a flowchart illustrating an internal configuration of the smartphone of FIG. 1.

As shown in FIG. 3, the smartphone 2 according to the present invention includes an input unit 210, a buffer unit 220, an image processor 230, a display unit 240, and a storage unit 250. Except for the function as a general communication means, the matters related to the remote video monitoring function, in particular, the technical part for smoothly providing a multi-view layout in a smart phone with limited resources will be described.

The input unit 210 may use a conventional touch screen as a part for receiving camera selection information and control information for zoom, pan, and tilt of the selected camera from the user.

The buffer unit 220 temporarily stores the image data transmitted through the IP camera 20, and includes a front buffer 221 for storing the entire input image (H.264, MJPEG) data, and a previous input. And a rear buffer 222 storing only the changed portion by comparing the image ((H.264, MJPEG)) data stored in the front buffer 221 with the image ((H.264, MJPEG)) data currently input. .

The image processor 230 dynamically generates multiple view layout threads when selecting multiple images and buffers the image ((H.264, MJPEG)) data received from the IP camera 20 for each generated thread. ), And then put them together to convert the image into an image to be displayed on the display unit 240 is repeated. In the image conversion process, input image ((H.264, MJPEG)) data is converted into bitmap data, and the converted bitmap data is converted to a bitmap after conversion into bitmap data ((H.264, MJPEG)). .264, MJPEG)) data is displayed on the display unit 240 by the set number of frames per second.

When the image processor 230 converts the data filled in the buffer 220 into an image, the image processor 230 compares the data stored in the front buffer 221 with the data stored in the rear buffer 222, that is, between the previous frame and the current frame. By comparing the changed data through the comparison and changing only the pixel data changed in the previous frame image, resources required for image conversion are minimized.

As another resource minimization scheme, the image processor 230 of the present invention displays an image at a first frame rate in a single view layout state in which only one image is displayed on the display unit 240, and is displayed on the display unit 240. In a multi-view layout that simultaneously displays a plurality of images, it is possible to display an image at a second frame rate (for example, about 0.5 times the first frame rate) lower than the first frame rate.

As another resource minimization scheme, the image processor 230 of the present invention displays an image at a second frame rate in a multi-view layout, and when a specific layout is touched in the middle, only an image of a selected layout among a plurality of layout images is first displayed. It is also possible to monitor the selected image more precisely by displaying at the frame rate, and to change the screen to a single view layout when the layout being displayed at the first rate is long touched.

4 exemplarily illustrates a camera list providing screen of a smartphone. The screen of FIG. 4 is an initial screen for driving a remote video surveillance application, and an additional menu (Add: 310) for additionally registering a specific IP camera 20 is located at the top of the screen, and an IP camera 20 registered at the bottom thereof. ), A camera list unit 320 displaying a still image, an installation region name, and connection information is located.

At the bottom of the screen, a list menu 330 showing an entire list of registered IP cameras 20 is located.

FIG. 5 illustrates an example of a single view layout. When the user selects any one of the IP cameras 20 listed in the camera list unit 320, the image of the IP camera 20 is displayed as shown in FIG. 5. Enlarged display.

In the single view layout, the single view screen 340 is enlarged and displayed, and a control button unit 350 for providing a monitoring function such as pan, tilt, and auto rotation of the camera is located at the bottom thereof.

6 shows an example of a multiview layout.

In the multi-view layout, multi-images are simultaneously displayed using a multi-stream real-time streaming method through a multi-threaded method. It can be expressed smoothly.

In the multi-view layout 360, a plurality of images are displayed, and a control button unit 350 for providing a monitoring function such as pan, tilt, and auto rotation of the camera is located on the right side.

Hereinafter, layout dynamic management for multiview layout processing will be described.

The static layout creates a central layout for limited smartphone resources, dynamically creates a layout to handle additional video views as the streaming video increases, and the dynamically generated layout is returned to memory after use.

Layout for Multi-view Management In multi-run control, if the additional views are connected, the dynamically generated layout is temporarily stored to control and manage the screen. Each view has a unique value and the set values are stored in the DB. By interworking with, the setting value can be maintained even when switching between single view and multi view.

If you touch a specific layout in multi view, it is converted to single view layout. At this time, all multi view layouts and threads defined in the multi view are terminated and deleted, and the converted single view is displayed inside the statically defined layout defined at the beginning. The view is dynamically created and defined and displayed on one screen.

Although the present invention has been described in connection with the above-mentioned preferred embodiments, it is possible to make various modifications or variations without departing from the spirit and scope of the invention. Accordingly, the appended claims will cover such modifications and variations as fall within the spirit of the invention.

1: area to be monitored 2: smartphone
10: fuselage recognition sensor 20: IP camera
21: image pickup device 22: image processing unit
23: multi modem driver 24: serial interface unit
25: touch screen unit 26: RFID module
30: modem device 210: input unit
220: buffer portion 221: front buffer
222: rear buffer 230: image processing unit
240: display unit 250: storage unit

Claims (7)

A plurality of tags installed at a specific position within the shooting target area;
A body recognition sensor for detecting a movement of a surrounding object;
A modem device including any one of a mobile communication modem, a wireless LAN modem, and a wireless portable Internet modem;
Image pickup device for taking pictures of the surrounding image, Image processing unit for processing the image taken by the image pickup device, Touch screen for control signal input and operation control, Serial interface terminal for connection of the modem device, Mobile communication modem, Wireless LAN modem, And a multiple modem driver for driving a modem device connected to the serial interface by supporting a wireless portable Internet modem, and an RFID module for reading tag information recorded in the tag, on a portion where the tag is located in a captured image. An IP camera that displays the tag information in text or graphic form; And
A display unit which is connected to the IP camera via the modem device to display an image photographed by the IP camera, a display unit which displays an image obtained by the IP camera, camera selection information from the user, zoom, pan for the selected camera, Input unit for receiving control information about the tilt, Storage unit for storing the connection information of the IP cameras stored by the user, Temporary storage of the image data, to store the entire image ((H.264, MJPEG)) data Compares the front buffer with the image ((H.264, MJPEG)) data previously input and stored in the front buffer and the image ((H.264, MJPEG)) data that is currently input. Dynamically create multi-view layout threads when selecting the buffer unit and multiple images, and receive the images received from the IP camera for each created thread. ((H.264, MJPEG)) Fills a buffer into a buffer, combines it, converts it into an image, and displays the display on the display. An image stored in the front buffer based on a value stored in the rear buffer. ((H.264, MJPEG)) includes a smart phone including an image processing unit for displaying only the changed data from the data to the display unit,
The IP camera zooms in the position where the movement of the object is detected and transmits an image marked around the position where the movement is detected to the smart phone when an object movement amount of a set value or more is detected based on the detection signal of the body recognition sensor. Smartphone video providing system using an IP camera, characterized in that.
delete delete delete The method of claim 1,
The image processor displays an image at a first frame rate in a single view layout state, and displays an image at a second frame rate lower than the first frame rate in a multi view layout. Provide system.
The method of claim 5, wherein
The image processing unit displays an image of the layout at the first frame rate when a specific layout is touched while displaying an image at the second frame rate in a multi-view layout, and the layout being displayed at the first speed is long touched. If it is, the smart phone video providing system using an IP camera, characterized in that for changing to a single view layout. delete

Images