KR101441755B1 - A smart camera with interchangeable image senosr module and main processing module - Google Patents

Abstract

[0001] The present invention relates to a smart camera, and more particularly, to an image sensor module and a main processing module which are manufactured so that they can be separated from each other and can be classified into a semiconductor type classification (CCD, CMOS) and a light source classification (X-ray, infrared ray, And an image sensor module according to the resolution of the sensor, the image acquisition speed, and other external information input (voice signal, acceleration information, USB support equipment, etc.); Enabling the selection of a suitable main processing module for the processing capacity, processing speed of the required image data, and required processing capability according to 3D image analysis, voice analysis, acceleration analysis, and the like; And more particularly, to a detachable smart camera which can be used in various applications.

Description

A smart camera with an interchangeable image senosr module and a main processing module, wherein the image sensor module and the main processing module are replaceable,

The present invention relates to a smart camera, and more particularly, to a smart camera which is manufactured to be replaceable with an image sensor module and a main processing module and is applicable to various fields using an image sensor module and a main processing module will be.

In general, the smart camera performs functions of motion detection, risk detection, night security surveillance, face detection and recognition, specific object detection, feature detection, medical analysis, and traffic volume analysis using images acquired by the image sensor, It can detect specific events from input images such as storing data or transferring data to a central management system, generating decision information that can be used in an intelligent or automated system, or extracting information that can be applied to an application. Can be defined as an "embedded vision system".

In order to be applied to various application fields, such a smart camera is required to have a suitable sensor according to the semiconductor type classification (CCD, CMOS) and the classification of the light source (X-ray, infrared ray, visible ray, ultraviolet ray) And additional external information such as a voice signal, an acceleration signal, GPS information, and the like is required depending on an application field. When three-dimensional information extraction is required, a stereo camera composed of two image sensors, A camera system using reflection techniques is also required. In addition, high-performance computation and data processing capabilities may be required for systems that require high-performance computation, such as three-dimensional image information extraction, or when the amount of information to be processed is large.

However, since the conventional smart camera provides only predetermined functions for each product, it should be used as a suitable product for different application fields or additional functions. This can be done by changing the system (changing the role in the equipment, changing the inspection product, etc.) and the additional functions required (night vision, GPS, acceleration sensor or voice signal recognition) In addition to the costly replacement of the system to cope with such problems, re-production of the program, retraining of the user, and verification of the stability of the newly introduced system are incurred.

According to an aspect of the present invention, there is provided an image sensor module and a main processing module, wherein the image sensor module and the main processing module are separately formed and replaceable, Making the camera available across a variety of applications.

In addition, it provides a consistent hardware interface and program development environment for various kinds of modules provided in the application of various required functions, thereby minimizing the additional cost in changing and reconfiguring the camera system.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not intended to limit the invention to the exact underlying teachings of the present invention. There will be.

In order to accomplish the object of the present invention, a smart camera according to an aspect of the present invention includes a main processing unit including an MCU module (Micro Controller Unit), a first FPGA (Field Programmable Gate Array), a first communication unit, And an image sensor module including a second FPGA, a second communication unit, a second memory unit, an image sensor, and a second input / output unit, wherein the main processing module and the image sensor module are connected to a cartridge, a flat cable and a connector Wherein the main processing module analyzes, stores, processes, and transmits / transmits information transmitted from the image sensor module, and the image sensor module acquires image, sound, light and vibration information, The information can be transmitted / transmitted to a module or a nearby camera, and the acquired information can be stored, thereby solving the above problems .

The smart camera, in which the image sensor module and the main processing module according to the present invention can be replaced, provides a system interface having the same structure that can be interchanged. Therefore, in the system design and program production for application to new application fields, It is possible to design and program the system and to perform the minimum system configuration change and the program change operation in accordance with the change and addition of the functions, so that it is possible to construct a stable system as well as a cost for constructing the system.

In addition, the various external information acquisition functions of the image sensor module and the high-performance computation function of the main processing module enable various types of system configurations, thereby expanding the application range of the application field.

In addition, it is possible to construct a multi-component camera system composed of a machine vision camera, a CAM, an infrared camera, a stereo camera, etc. using the camera link unit of the image sensor module, the second communication unit and the second extended communication unit, It can acquire various image data, thereby improving utilization efficiency.

On the other hand, by using information such as an acceleration sensor, a microphone input, and GPS, a device capable of transmitting information in the event of a disaster such as an earthquake, a fire or a natural disaster can be provided, .

1 is a system configuration diagram showing a smart camera in which an image sensor module and a main processing module according to the present invention can be replaced.
2 is a flowchart illustrating a smart camera in which an image sensor module and a main processing module according to the present invention can be replaced.
3 is a perspective view showing a smart camera in which the image sensor module and the main processing module can be replaced.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The sizes and shapes of the components shown in the drawings may be exaggerated for clarity and convenience. In addition, terms defined in consideration of the configuration and operation of the present invention may be changed according to the intention or custom of the user, the operator. Definitions of these terms should be based on the content of this specification. It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the inventions. Of course.

1 is a system configuration diagram showing a smart camera in which an image sensor module and a main processing module according to the present invention can be replaced. Here's a smart camera. In addition, the structure of the present invention will be described with reference to Fig. 1, and reference numerals of the constituent elements not shown in Fig.

A smart camera according to an embodiment of the present invention includes a main processing module 100 and an image sensor module 200.

In one aspect of the present invention, the main processing module 100 includes an MCU module 10, a first FPGA (Field Programmable Gate Array) 20, a first communication unit 30, a first extended communication unit 40, a first memory unit 70, a flash memory unit 60, and a VGA output unit 41. More specifically, the main processing module 100 is connected to a cartridge connection, a flat cable, or an external connector so that the user can easily replace the main processing module 100. The main processing module 100 processes the information acquired by the image sensor module 200 to be described later. For example, when the image sensor module 200 acquires information on the external shape or sound of a specific person, the information is transmitted to the main processing module 100, and the transmitted information is subjected to image processing, Analyze the situation, and perform actions such as communicating information to the central management system, alarming, monitoring and tracking objects continuously, and the like. The camera system thus formed is formed as a system in which the user does not need to operate or control the smart camera, thereby enabling the user to perform the operation more efficiently. The main processing module 100 can perform tasks such as monitoring, tracking, acquiring information, alerting and storing information when a preset event occurs. The main processing module 100 is provided on another substrate so as to be replaceable and can be replaced according to the needs of the user.

An MCU module (Micro Controller Unit) 10 is a processor for controlling a smart camera, and a smart camera processes specific information through algorithm operation such as image recognition or speech recognition. The MCU module 10 is connected to the first FPGA 20, the first communication unit 30, the first memory unit 70, and the interface unit 90, which will be described later. The information obtained by the image sensor module 200 is transmitted to the MCU module 10, and the transmitted data is analyzed and compared with the predetermined information, so as to take appropriate measures. For example, when information about a specific object is transmitted to the MCU module 10 through the image sensor module 200, the MCU module 10 tracks, monitors, Storage, and information about the target is transmitted to a nearby smart camera so that the target can be continuously tracked. Specifically, the MCU module 10 internally has a TMS320C6678 DSP processor having eight operation cores connected to one or more (N) of HyperLinks 11 to enable a high performance operation system composed of 8 * N operation cores. This high-performance computation system can perform a high-speed operation in an image processing system requiring a large amount of computation, so that it can expect higher processing performance than a conventional smart camera, and a very advanced image processing program such as 3D image processing To be used in real time.

The first FPGA 20 is connected to a flash memory unit 60, a first extended communication unit 40, a VGA output unit 41 and a first input / output unit 50, which will be described later. The first FPGA 20 transfers the data of the first extended communication unit 40 and the data of the MCU module 10 and outputs the monitor output through the VGA output unit 41, And provides a connection with the first input / output unit 50 so that the MCU module 10 can control the input / output port to a specific address.

The flash memory 60 is a nonvolatile flash memory, which is equipped with an OS (Operating System (Linux) kernel, various device drivers and application programs of a smart camera, and can store data of a smart camera. The flash memory unit 60 is formed of the JEDEC standard (eMMC4.41, eMMC4.5). If the flash memory unit 60 is used as an eMMC, the OS and programs can be driven at a higher speed.

The first extended communication unit 40 can exchange information with other devices by supporting communication such as USB, CAN (Controller Area Network, communication interface with an automation device) and RS232 by the MCU module 10. USB provides communication with devices such as data storage media, GPS receiver, external CAM, etc. that support USB communication method. CAN can drive various physical control devices through interlocking with robot or position speed control system , RS232 provides communication with devices such as a touch screen, a light controller (light controller) and the like. Since the first extended communication unit 40 is configured simply in terms of hardware, the manufacturing cost of the smart camera can be reduced and the configuration can be simplified.

The first communication unit 30 connects the MCU module 10 and the central management system. The first communication unit 30 can receive information from the central management system. The first communication unit 30 may be configured to transmit information to and receive information from other smart cameras when the main processing module 100 is used alone without the image sensor module 200. The first communication unit 30 is connected to the central management system via wired or wireless, and transmits status information of the smart camera, specific event information, and the like to the central management system. It is also possible for a user to operate a smart camera using a central management system. The first communication unit 30 supports Gigabit Ethernet and 10 Gigabit Ethernet. The first communication unit 30 is configured to process a large amount of information such as image information, sound, and specific object information by using Gigabit Ethernet or 10 Gigabit Ethernet suitable for transmission of a large amount of information, More stable information transfer is possible.

The first memory unit 70 includes a DDR3 SDRAM as a space in which an OS kernel and an application program are loaded from the flash memory unit 60 and a program is executed. A number of TMS320C6678 DSP-enabled MCU modules (10) support up to 8GBytes of DDR3 memory space.

The first input / output unit 50 is connected to the MCU module 10 through the first FPGA 20. The first input / output unit 50 can input and output a strobe signal that can accurately control one or more camera image acquisition timings that can be connected to an image sensor module 200 to be described later. And can input and output digital signals.

The first power supply unit 80 supplies power to the main processing module 100.

The image sensor module 200 includes a second FPGA 110, a second memory unit, an image sensor 140, a camera link 170, a second input / output unit, a second communication unit 120, a second extended communication unit 150, .

The image sensor module 200 is formed to be replaceable according to the needs of the user. More specifically, the image sensor module 200 is connected to a cartridge connection, a flat cable, or an external connector so that the user can easily replace the image sensor module 200. The connection between the image sensor module 200 and the main processing module 100 is connected through the interface unit 90. The interface unit 90 uses a standard PCI Express Gen2 x2 interface, which is a high-speed serial communication system of 10 Gbps. This is because it provides a consistent interface when transmitting information between the image sensor module 200 and the main processing module 100, so that it takes a minimum time and cost to change the system or change the program so that the user can use the device more efficiently. Specifically, the image sensor module 200 is formed of at least one selected from the group consisting of a real-time image analysis module, a sound recognition module, a heat and smoke recognition module, an external shape recognition module, an optical zoom lens module, and a three- By providing these various functions, the use of a smart camera can be made more effective. Also, by using the image sensor module 200, it is possible to analyze the image acquired from the nearby camera.

The second FPGA 110 acquires an image from an image sensor 140, a camera link unit 170 and a second communication unit 120 to be described later, stores the acquired image in the second memory unit 130, To the processing unit (100). The second FPGA 110 controls the image sensor module 200. This is implemented by using a System On Chip (SOC) FPGA with an embedded MCU. Specifically, the second FPGA 110 uses a SmartFusion2 FPGA. Since the second FPGA 110 includes an Arm-Cotex M3 Process, a program of a desired function can be flexibly installed and operated.

The second communication unit 120 is formed of Gigabit Ethernet or 10 Gigabit Ethernet widely used as a standard interface for machine vision. The second communication unit 120 may extend one or more cameras to the outside by connecting a network camera supporting Gigabit Ethernet or 10 Gigabit Ethernet. This is very effective in combining various kinds of cameras into the system. Also, since it forms a separate network separated from the first communication unit 30, it can transmit a large amount of image information without affecting the network of the central management system.

The second memory unit 130 is used as a temporary storage device for transferring information received from the second FPGA 110 to the main processing module 100. The second memory unit 130 is formed of a DDR3 SDRAM.

The image sensor 140 is a sensor mounted on the image sensor module 200 for acquiring image information. The image sensor 140 may control the second FPGA to acquire necessary image information. The second FPGA receives the image acquisition command through the main processing module 100 or the trigger image acquisition command of the second input / output unit 160, or in the case of the image acquisition condition at a predetermined time interval, the image acquisition waiting time expires The image acquisition procedure is performed. More specifically, the image sensor 140 exposes external light for a predetermined time and converts an electrical signal for an image formed on the image sensor 140 into an analog / digital (A / D) Is transferred to the second FPGA 110 in a predetermined sequence and is recorded in the second memory unit 130, which is a temporary storage space. When the recording is completed, the second FPGA 110 transmits the image to the main processing module 100 using the interface unit 90. The image sensor 140 is configured to measure the resolution of the sensor and the image acquisition speed based on the classification of a semiconductor type (CCD (charge coupled device), complementary metal-oxide semiconductor (CMOS)) and classification of a light source (X-ray, infrared ray, visible light, ultraviolet ray) . One or more image sensors 140 are provided so that the user can perform necessary functions.

The second input / output unit 160 may receive a signal including a voice signal, a trigger signal, and a GPS signal. The audio signal is input through a microphone formed in the second input / output unit 160 and is supplied with a predetermined specific audio signal (e.g., sound such as "help"), It is possible to detect a crisis situation and perform a role of tracking and monitoring objects. The analog signal input from the microcomputer is A / D (Analog / Digital) converted from the second FPGA 110 and is recorded in the second memory unit 130. When the recording is completed, the main processing module 100 transmits the image together with the image using the interface unit 90 of the PCI Express Gen2 x2. The main processing module 100 recognizes a danger or other situation by applying a speech recognition algorithm to the transmitted speech data. The trigger signal is a digital input signal that is input through the second input / output unit 160, and is formed to match an accurate time of image acquisition. Also, the main processing module 100 can receive input from the strobe signal output from the first input / output unit 50. The GPS signal transmits the current position information through the second input / output unit 160 as a digital input signal. This makes it possible to know the camera position of the current event destination.

The camera link unit 170 is an interface that supports the Camera Link 2.0 standard, and can be used when additional image acquisition is required to be connected to the outside of the system.

The second power supply unit 180 supplies power to the image sensor module 200.

In another embodiment of the present invention, the acceleration sensor 165 is formed in the second input / output unit 160. The acceleration information obtained by the acceleration sensor 165 of the second input / output unit 160 is transmitted to the main processing module 100. The transmitted acceleration information is transmitted to an alarm and alerts the user when a vibration of a predetermined level or more occurs or a dangerous situation is recognized. The alarm is connected to the acceleration sensor 165. Specifically, the alarm can be alarmed visually (e.g., a warning light) or audibly (e.g., siren). This protects the safety of the user by providing alarm in case of disaster such as earthquake or fire.

In another aspect of the present invention, the second extended communication unit 150 may be further included. The second extension communication unit 150 supports communication such as USB, CAN (Controller Area Network, communication interface with an automation device) and RS232, and can exchange information with other devices. USB provides communication with devices such as data storage media, GPS receiver, external CAM, etc. that support USB communication method. CAN can drive various physical control devices through interlocking with robot or position speed control system , RS232 provides communication with devices such as a touch screen, a light controller (light controller) and the like. This not only allows the main processing module 100 to store data, but also makes it possible to store data in the image sensor module 200, thereby facilitating information management. In addition, since the second extended communication unit 150 is simply configured in hardware, it is possible to reduce the production cost of the smart camera and simplify the configuration.

An embodiment of the smart camera of the present invention will be briefly described as follows.

The smart camera analyzes external information data such as image data, acceleration information, and sound information acquired at regular intervals. When the preset event situation is recognized, the smart camera tracks and monitors the image of the event object and acquires the data of the event object. The event target data may be transmitted to the central management system connected to the first communication unit 30 and another camera. All connected cameras are aware of the event occurrence, continuously track the event object, monitor and store the image, and deliver the event information to the central management system and the neighboring smart camera in real time. In addition, the central management system checks the path where the smart camera is installed, confirms the movement path of the event object, and continuously monitors whether the event object is present or not to the camera mounted on the adjacent path, thereby continuously tracking and monitoring the event object . Such an operation is performed by automatic recognition of a predetermined situation of a smart camera, and performs operations such as warning and image transmission to a user when necessary, and continuously tracks and monitors an event object without any user control, So that the user can take appropriate measures.

It will be apparent to those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof.

10: MCU module, 11: hyperlink,
20: first FPGA, 30: first communication unit,
40: first extension communication unit, 41: VGA output unit,
50: first input / output unit, 60: flash memory unit,
70: a first memory unit, 80: a first power supply unit,
90: interface part,
100: main processing module,
110: second FPGA, 120: second communication unit,
130: second memory unit, 140: image sensor,
150: second extension communication unit, 160: second input / output unit,
165: acceleration sensor, 170: camera link portion,
180: second power supply unit,
200: Image sensor module.

Claims (6)

A main processing module 100 including an MCU module (Micro Controller Unit) 10, a first FPGA (Field Programmable Gate Array) 20, a first communication unit 30 and a first memory unit 70,
And an image sensor module 200 including a second FPGA 110, a second communication unit 120, a second memory unit 130, an image sensor 140, and a second input / output unit 160,
The main processing module 100 and the image sensor module 200 are connected to each other by a cartridge, a flat cable, and a connector,
The main processing module 100 analyzes, stores, processes, transmits / transmits information transmitted from the image sensor module 200,
The image sensor module 200 may acquire image, sound, light, and vibration information to transmit / deliver information to the main processing module 100 or a neighboring camera, store the acquired information,
The image sensor module 200 includes a second extended communication unit 150. The second extended communication unit 150 includes a USB, a CAN (Controller Area Network), an RS232 device, And is configured to be capable of storing data and transmitting / transmitting data.
delete The method according to claim 1,
The second input / output unit 160 includes an acceleration sensor 165,
Wherein the data obtained from the acceleration sensor (165) is transmitted to the second FPGA (110) or the main processing module (100).
The method of claim 3,
The second input / output unit 160 may further include an alarm unit connected to the acceleration sensor 165,
Wherein the alarm is visually or audibly alarmed.
The method according to claim 1,
Wherein the MCU module (10) is connected to one or more MCUs, each of which is made up of multicore, by a HyperLink (11).
delete

Images