TW202027487A - Mobile image processing apparatus, method and computer readable storage medium - Google Patents

Abstract

The present invention provides a mobile image processing apparatus including at least one first camera and at least one second camera. Each of the first cameras includes a filter for filtering infrared rays, and one of the first cameras is provided with a switch. The switch is configured to switch between a first mode and a second mode on the first camera, the filter is turned off in the first mode, and the filter is turned on in the second mode. The image obtained by the first camera in the first mode and the image obtained by each of the second cameras are subjected to process to obtain a depth image. The present invention also provides a mobile image processing method and a computer readable storage medium. The mobile image processing device provided by the invention has a simple structure, can realize shooting in multiple modes with small volume and low cost.

Description

Mobile image processing device, method and computer readable storage medium

The invention relates to an image information collection technology, in particular to a mobile image processing device, method and computer readable storage medium.

In the prior art, a binocular stereo camera (Stereo Camera) is a common configuration of a 3D sensing device. In order to take into account both bright and dark scenes, the binocular stereo camera has changed from a single camera to a dual camera, including a color camera (RGB Camera) and a stereo camera (Stereo Camera).

Currently, binocular stereo cameras use two types of camera arrangements. The first type is composed of two color cameras. In this mode, it can work in a bright environment. When the illumination is low, it is necessary to use visible light to assist the illumination. Therefore, in this form, it is generally not usable in a low-light environment. The second type is a color camera plus two stereo cameras, and the stereo camera is equipped with a light supplement module. In the second form, the hardware cannot be further simplified, resulting in a larger camera volume and higher cost.

In view of the above situation, it is necessary to provide a mobile image processing device, method, and computer-readable storage medium that can solve the above-mentioned problem of large volume by changing the hardware arrangement of the camera.

The first aspect of the present invention provides a mobile image processing device for image shooting and image processing. The mobile image processing device includes an input unit, a display unit, a communication unit, a memory, a processor, at least one first camera, and at least A second camera; each of the first cameras includes a filter for filtering infrared, and one of the first cameras is provided with a switch, and the switch is used to implement the first camera in the first Switch between the mode and the second mode, the filter is turned off in the first mode, and the filter is turned on in the second mode; the memory stores a plurality of instructions, and the instructions are executed by the processor Load and execute the following steps: control the switch to switch the first camera to the first mode; control the first camera to take a first image in the first mode; control the second The second image is captured by the camera; the first image is processed and the second image is subjected to frame synchronization processing to obtain a depth image.

The second aspect of the present invention also provides a mobile image processing method, which is used in a mobile image processing device. The mobile image processing device includes at least one first camera and at least one second camera. The first camera includes a filter for filtering infrared rays, and one of the first cameras is provided with a switch, and the switch is used to switch the first camera between the first mode and the second mode, so The filter is turned off in the first mode, and the filter is turned on in the second mode; the moving image processing method includes the following steps: controlling the switch to switch the first camera to the second A mode; control the first camera to capture a first image in the first mode; control the second camera to capture a second image; the first image is processed with the second image The image undergoes frame synchronization processing to obtain a depth image.

The third party of the present invention also provides a computer-readable storage medium on which a computer program is stored, and the computer program is loaded by a processor and executes the above-mentioned moving image processing method.

The invention provides a mobile image processing device, which includes a first camera and a second camera. The first camera is provided with a switch, which is used to switch the first camera between the first mode and the second mode; the image obtained by the first camera in the first mode and the image obtained by each second camera The depth image is obtained after image processing. The invention also correspondingly provides a mobile image processing method and a computer-readable storage medium. The mobile image processing device provided by the invention has a simple structure, can realize multiple shooting modes, has a small volume and a low cost.

In order to be able to understand the above objectives, features and advantages of the present invention more clearly, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the embodiments of this application and the features in the embodiments can be combined with each other if there is no conflict.

In the following description, many specific details are explained in order to fully understand the present invention. The described embodiments are only a part of the embodiments of the present invention, rather than all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present invention.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments, and are not intended to limit the present invention.

The term "and/or" as used herein includes any and all combinations of one or plural related listed items.

Please refer to FIG. 1, the present invention provides a moving image processing device 10, and FIG. 1 is a schematic structural diagram of the moving image processing device 10. The mobile image processing device 10 includes an input unit 100, a display unit 200, a communication unit 300, a memory 400, a processor 500, at least one first camera 600 and at least one second camera 700. The input unit 100, the display unit 200, the memory 400, the first camera 600 and the second camera 700 are electrically connected to the processor 500, respectively. Specifically, the image collection surfaces of each first camera 600 and each second camera 700 are on the same plane, so that the resolution of the images obtained by shooting the same object by the first camera 600 and the second camera 700 is consistent.

The input unit 100 is used for the user to input various information and control commands. In this embodiment, the input unit 100 may include a touch screen, a remote control, a voice input device, etc., but is not limited thereto.

The display unit 200 is used to display the processing result of the processor 500. The display unit 200 includes at least one display.

The communication unit 300 is used for the mobile image processing device 10 to establish a communication connection with other mobile terminals. In this embodiment, the communication unit 300 establishes a communication connection with other mobile terminals through a wireless network or a limited network. The wireless network can be, but not limited to, WIFI, Bluetooth, cellular mobile network, satellite network, short-range Wireless communication module NFC, etc. The wired network can be but not limited to USB communication, IEEE1394 communication, etc. Specifically, in this embodiment, the communication unit 300 includes an independent WIFI port, which allows other mobile terminals (such as mobile phones) to access.

The memory 400 is used to store various types of data in the mobile image processing device 10, such as image data and program codes, and realize high-speed and automatic access to programs or data during the operation of the mobile image processing device 10.

The memory 400 also stores an image depth algorithm, and processes the image according to the image depth algorithm to obtain a depth image.

The memory 400 may be, but not limited to, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), programmable read-only memory (Programmable Read-Only Memory, PROM), and erasable Erasable Programmable Read-Only Memory (EPROM), One-time Programmable Read-Only Memory (OTPROM), Electronically-erasable rewritable read-only memory (Electrically-Erasable) Programmable Read-Only Memory (EEPROM), Compact Disc Read-Only Memory (CD-ROM) or other optical disk storage, magnetic disk storage, magnetic tape storage, or any computer readable that can be used to carry or store data Other media.

The processor 500 is a digital signal processor (DSP, Digital Signal Processor), a microcontroller unit (MCU, Microcontroller Unit), an embedded computer processor (ARM, Advanced RISC Machine), a field programmable gate array (FPGA, Field- Programmable Gate Array), central processing unit (CPU, Central Processing Unit), single-chip microcomputer, system-on-chip (SoC, System on Chip) or other equivalent dedicated chips.

The first camera 600 is mainly a color camera (RGB Camera), which is generally used under bright light conditions. The image effect obtained can also be called human vision, and the amount of calculation for data processing is small. The first camera 600 includes an image sensor (not shown), and an infrared filter must be used to adopt the first camera 600. Without the infrared filter, the image sensor itself responds to infrared light invisible to the human eye, so the image will be reddish, which is different from the actual image seen by the human eye.

The second camera 700 is mainly a stereo camera (Stereo Camera), and the stereo camera generally adds a light supplement module to make it take into account the two situations of bright light and dark light. The obtained image can be called machine vision. The second camera 700 also includes an image sensor, and mainly performs stereo (3D) sensing. Therefore, under machine vision, the amount of calculation for image processing is large. In general, a stereo camera uses two cameras.

In the present invention, in order to take into account the advantages of the first camera 600 and the second camera 700, the size of the mobile image processing device 10 is also reduced. The present invention proposes a new hardware architecture relationship between the first camera 600 and the second camera 700.

Specifically, each first camera 600 includes one or more of an infrared cut filter and a blue glass filter. Each second camera 700 includes a color filter and the aforementioned light supplement module. One of the first camera 600 is provided with a switch, and the switch controls the first camera 600 to switch between the first mode and the second mode. In the first mode, the switch controls one or more of the infrared cut filter of the first camera 600 and the blue glass filter to be turned off together. In the second mode, the switch controls one or more of the infrared cut filter of the first camera 600 and the blue glass filter to be turned on together.

In this embodiment, the number of the first camera 600 and the number of the second camera 700 is one. It is understandable that the number of the first camera 600 and the second camera 700 may be plural respectively, such as 2, 3, or 4.

Referring to FIG. 2, the mobile image processing device 10 also runs a control system 800. The control system 800 includes a computer command in the form of one or plural programs. The computer command in the form of one or plural programs is stored in the memory 400 and is executed by the processor. 500 execution. FIG. 2 is a schematic diagram of functional modules of the control system 800 in an embodiment of the present invention. Specifically, the control system 800 includes a mode switching module 810, an image processing module 820, and a transmission module 830.

The mode switching module 810 is used for controlling the switch to switch between the first mode and the second mode. In this embodiment, a user-controlled shooting program is pre-stored in the mode switching module 810, and the user completes shooting in different modes according to the controlled shooting program.

The image processing module 820 is used to receive the captured image data, and then perform corresponding image processing on the received image data according to different shooting modes.

The transmission module 830 is used for transmitting the images taken by the first camera 600 and the second camera 700 to the image processing module 820, and is also used for outputting images after image processing.

The mobile image processing device 10 is a camera, a mobile phone, a tablet computer, a notebook computer, a police station or a smart TV.

Please refer to FIG. 3, the present invention provides a moving image processing method, and FIG. 3 is a schematic flowchart of the moving image processing method. Specifically, the image movement processing method includes the following steps:

In step S301, the first camera 600 and the second camera 700 are turned on together.

The input unit 100 inputs an instruction to control the turning on of the first camera 600 and the second camera 700, and the control system 800 turns on the first camera 600 and the second camera 700 together to prepare to capture images.

In step S302, the switch switches the first camera 600 to the first mode.

The control system 800 controls the mode switching module 810 according to the instructions of the first camera 600 and the second camera 700 to turn on. The mode switching module 810 also closes the filter of the first camera 600 according to the pre-stored user shooting control program, including infrared cutoff The filter and the blue glass filter switch the first camera 600 to the first mode.

In step S303, the first camera 600 captures a first image in the first mode.

The control system 800 controls the image processing device to perform image capture, the first camera 600 captures the first image in the first mode, and the first image is stored in the memory 400.

Step S304, the second camera 700 captures a second image.

Similarly, the control system 800 controls the second camera 700 of the image processing device to perform image capture, the second camera 700 captures the second image, and the second image is stored in the memory 400.

Step S305: After the first image is processed, frame synchronization processing is performed with the second image to obtain a depth image.

The first image and the second image are transmitted to the image processing module 820 via the transmission module 830. The image processing module 820 first processes the first image captured by the first camera 600. This process includes cropping and scaling. The processed first image and the second image obtained by the second camera 700 undergo frame synchronization (picture synchronization) processing. An image depth algorithm is preset in the image processing module 820, and the frame synchronized image is processed according to the image depth algorithm to obtain a depth image.

Step S306, output the depth image.

The depth image is output to other mobile terminals communicatively connected with the mobile image processing device 10 via the communication unit 300.

In some alternative embodiments, referring to FIG. 4, the present invention also provides a moving image processing method. It includes the following steps:

Step S401: Turn on the first camera 600 and turn off the second camera 700.

The control system 800 controls the first camera 600 to be turned on and the second camera 700 to turn off.

Step S402, the switch switches the first camera 600 to the second mode.

The control system 800 controls the mode switching module 810. The mode switching module 810 turns on the filters of the first camera 600 together according to the pre-stored user shooting control program, including the infrared cut filter and the blue glass filter, so that the first The camera 600 switches to the second mode.

In step S403, the first camera 600 captures a third image in the second mode.

The control system 800 controls the first camera of the image processing device to capture images, the first camera 600 captures a third image in the second mode, and the third image is stored in the memory 400.

Step S404: After the third image is processed, the third image is output.

The third image is transmitted to the image processing module 820 via the transmission module 830. The image processing module 820 processes the third image to obtain an image of human vision. Finally, the processed third image is output via the communication unit 300 to other mobile terminals that are communicatively connected to the mobile image processing device 10.

It is understandable that in this embodiment, this shooting mode is suitable for shooting in a brightly lit environment.

It is understandable that if only the first camera 600 is turned on, the second camera 700 is turned off. The first camera 600 switches to the first mode for shooting. In this shooting mode, the filter is not turned on, and the image sensor responds to infrared light. Images taken in this mode are suitable for shooting in low-light environments.

The mobile image processing device 10 provided by the present invention includes a first camera 600 and a second camera 700. The first camera 600 is provided with a switch, and the switch is used to switch the first camera 600 between the first mode and the second mode; the image obtained by the first camera 600 in the first mode and each second camera 700 The obtained image is processed by the image to obtain the depth image. The invention also correspondingly provides a moving image processing method. The mobile image processing device 10 provided by the present invention has a simple structure, can realize multiple shooting modes, has a small volume and a low cost.

In addition, the functional devices in the various embodiments of the present invention may be integrated into the same data processing unit, or each device may exist alone physically, or two or more devices may be integrated into the same device. The above-mentioned integrated device can be realized in the form of hardware, or in the form of hardware plus software functional modules.

For those skilled in the art, it is obvious that the present invention is not limited to the details of the foregoing exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description, and therefore it is intended to fall on the rights. All changes within the meaning and scope of the required equivalent elements are included in the present invention. Any reference signs in the claims should not be regarded as limiting the claims involved. In addition, it is obvious that the word "include" does not exclude other means or steps, and the singular does not exclude the plural. The multiple devices or computer devices stated in the claims of the computer device can also be implemented by the same device or computer device through software or hardware. Words such as first and second are used to denote names, but do not denote any specific order.

Finally, it should be explained that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be Modifications or equivalent replacements are made without departing from the spirit and scope of the technical solution of the present invention.

10: Mobile image processing device 100: input unit 200: display unit 300: Communication unit 400: memory 500: processor 600: The first camera 700: second camera 800: control system 810: Mode switching module 820: image processing module 830: Transmission module

FIG. 1 is a schematic diagram of the hardware architecture of a mobile image processing device in an embodiment of the present invention.

Figure 2 is a schematic diagram of functional modules of the control system in an embodiment of the present invention.

FIG. 3 is a schematic flowchart of a moving image processing method in an embodiment of the present invention.

FIG. 4 is a schematic flowchart of a moving image processing method in another embodiment of the present invention.

no

Claims (10)

A mobile image processing device for image shooting and image processing. The improvement is that the mobile image processing device includes an input unit, a display unit, a communication unit, a memory, a processor, at least one first camera, and at least one second camera. Camera; each of the first camera includes a filter for filtering infrared, one of the first camera is provided with a switch, the switch is used to realize the first camera in the first mode and the first Switch between the two modes, the filter is turned off in the first mode, and the filter is turned on in the second mode; the memory stores a plurality of instructions, and the instructions are loaded by the processor and Perform the following steps: Controlling the switch to switch the first camera to the first mode; Controlling the first camera to take a first image in the first mode; Controlling the second camera to capture a second image; After the first image is processed, frame synchronization processing is performed with the second image to obtain a depth image. The mobile image processing device described in the first item of the scope of patent application, wherein: Each of the first cameras includes an infrared cut filter and/or a blue glass filter; Each of the second cameras includes a color filter and a light supplement module; And turn off the infrared cut filter and/or the blue glass filter in the first mode; In the second mode, the infrared cut filter and/or the blue glass filter are simultaneously turned on. According to the mobile image processing device described in claim 1, wherein the instruction is loaded by the processor and further executes the following steps: After obtaining the depth image, it further includes: Output the depth image. According to the mobile image processing device described in item 3 of the scope of patent application, the instructions are loaded by the processor and further execute the following steps: After obtaining the depth image, it further includes: The switch to switch the first camera to the second mode; The first camera captures a third image in the second mode; After the third image is processed, the third image is output. The mobile image processing device described in item 4 of the scope of patent application, wherein: An image depth algorithm is preset in the memory; The depth image is obtained by processing according to the image depth algorithm. A moving image processing method, the moving image processing method is used for a moving image processing device, the moving image processing device includes at least one first camera and at least one second camera, each of the first cameras includes a filter, Used to filter infrared rays, one of the first cameras is provided with a switch, and the switch is used to switch the first camera between the first mode and the second mode. In the first mode, the switch is turned off. The filter is turned on in the second mode; the improvement is that the moving image processing method includes the following steps: Controlling the switch to switch the first camera to the first mode; Controlling the first camera to take a first image in the first mode; Controlling the second camera to capture a second image; After the first image is processed, frame synchronization processing is performed with the second image to obtain a depth image; Output the depth image. The moving image processing method as described in item 6 of the scope of patent application, wherein the frame synchronization processing to obtain the depth image includes processing according to a preset image depth algorithm. The moving image processing method described in item 6 of the scope of patent application, wherein: Each of the first cameras includes an infrared cut filter and/or a blue glass filter; Each of the second cameras includes a color filter and a light supplement module; And turn off the infrared cut filter and/or the blue glass filter in the first mode; In the second mode, the infrared cut filter and/or the blue glass filter are simultaneously turned on. The moving image processing method described in item 6 of the scope of patent application, wherein the moving image processing method further includes: The switch to switch the first camera to the second mode; The first camera captures a third image in the second mode; After the third image is processed, the third image is output. A computer storage medium, the computer storage medium stores computer program code, the improvement is that when the computer program code runs on a computing device, the computing device causes the computing device to execute any one of claims 6-9 Mobile image processing method.

Images