CN113744116B - Image processing chip and electronic device - Google Patents

Image processing chip and electronic device Download PDF

Info

Publication number
CN113744116B
CN113744116B CN202010478357.6A CN202010478357A CN113744116B CN 113744116 B CN113744116 B CN 113744116B CN 202010478357 A CN202010478357 A CN 202010478357A CN 113744116 B CN113744116 B CN 113744116B
Authority
CN
China
Prior art keywords
operating system
processing chip
image
image processing
image data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010478357.6A
Other languages
Chinese (zh)
Other versions
CN113744116A (en
Inventor
吴集
黄亮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Oppo Mobile Telecommunications Corp Ltd
Original Assignee
Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Oppo Mobile Telecommunications Corp Ltd filed Critical Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority to CN202010478357.6A priority Critical patent/CN113744116B/en
Publication of CN113744116A publication Critical patent/CN113744116A/en
Application granted granted Critical
Publication of CN113744116B publication Critical patent/CN113744116B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00General purpose image data processing
    • G06T1/20Processor architectures; Processor configuration, e.g. pipelining

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Studio Devices (AREA)

Abstract

The embodiment of the application provides an image processing chip and electronic equipment. The image processing chip can run the first operating system and the second operating system, and the starting time of the first operating system is smaller than that of the second operating system. During the starting period of the second operating system, the image processing chip can process the image data through the image processing chip operated by the first operating system, so that the image processing chip during the starting period of the second operating system can normally process the image processing work, and the influence of long starting time of the second operating system on the image processing work of the image processing chip is reduced.

Description

Image processing chip and electronic device
Technical Field
The present application relates to the field of electronic technologies, and in particular, to an image processing chip and an electronic device.
Background
In the related art, a user needs to turn on a camera when photographing, and then the image processing chip and the application processing chip process a picture captured by the camera. When the image processing chip processes the picture captured by the camera, a corresponding application operating system can be operated, and the image processing chip can start normal image processing operation only after the application operating system starts to operate.
The application operating system operated by the existing image processing chip is generally a linux operating system, and the linux operating system is suitable for multitasking and complex computing work in image processing. However, since the linux operating system is huge, the starting takes a long time.
Disclosure of Invention
The embodiment of the application provides an image processing chip and electronic equipment, wherein the image processing chip can process image data through the image processing chip operated by a first operating system during the starting period of a second operating system, so that the image processing chip during the starting period of the second operating system can normally process image processing work, and the influence on the image processing work of the image processing chip during the starting period of the second operating system is reduced.
In a first aspect, an embodiment of the present application provides an image processing chip, where the image processing chip may run a first operating system and a second operating system;
the image processing chip receives a first starting instruction and starts a first operating system;
the image processing chip under the operation of the first operating system receives image data and processes the image data;
The image processing chip under the operation of the first operating system receives a second starting instruction in the process of processing image data, and starts the second operating system so as to switch the operating system from the first operating system to the second operating system;
And the image processing chip under the operation of the second operating system continuously receives the image data and processes the image data, wherein the starting time of the first operating system is smaller than that of the second operating system.
In a second aspect, an embodiment of the present application provides an electronic device, where the electronic device includes an image processing chip and an application processing chip;
The image processing chip receives a first starting instruction sent by the application processing chip and starts a first operating system;
the image processing chip under the operation of the first operating system receives image data and processes the image data;
The image processing chip under the operation of the first operating system sends the processed image data to the application processing chip, so that the application processing chip carries out reprocessing on the processed image data;
The image processing chip under the operation of the first operating system receives a second starting instruction sent by the application processing chip in the process of processing image data, and starts a second operating system so as to switch the operating system from the first operating system to the second operating system;
The image processing chip under the operation of the second operating system receives the image data and processes the image data;
The image processing chip under the operation of the second operating system sends the processed image data to the application processing chip, so that the application processing chip carries out reprocessing on the processed image data;
the starting time of the first operating system is smaller than that of the second operating system.
From the above, the image processing chip and the electronic device provided by the embodiments of the present application are described. The image processing chip can run the first operating system and the second operating system, the starting time of the first operating system is smaller than that of the second operating system, and the image processing chip can process image data through the image processing chip under the operation of the first operating system during the starting period of the second operating system, so that the image processing chip during the starting period of the second operating system can normally process image processing work, and the influence of the long starting time of the second operating system on the image processing work of the image processing chip is reduced.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic diagram of a first structure of an electronic device according to an embodiment of the present application.
Fig. 2 is a schematic flow chart of an image processing chip start-up operating system according to an embodiment of the present application.
Fig. 3 is a schematic diagram of a first structure of an image processing chip according to an embodiment of the present application.
Fig. 4 is a schematic diagram of a second structure of an image processing chip according to an embodiment of the present application.
Fig. 5 is a schematic diagram of a second structure of an electronic device according to an embodiment of the present application.
Fig. 6 is a schematic flow chart of processing image data by the electronic device shown in fig. 5.
Fig. 7 is a schematic diagram of a third structure of an electronic device according to an embodiment of the present application.
DETAILED DESCRIPTION OF EMBODIMENT (S) OF INVENTION
Referring to the drawings, wherein like reference numbers refer to like elements throughout, the principles of the present application are illustrated in an appropriate computing environment. The following description is based on illustrative embodiments of the application and should not be taken as limiting other embodiments of the application not described in detail herein.
The term "module" as used herein may be considered as a software object executing on the computing system. The various components, modules, engines, and services described herein may be viewed as implementing objects on the computing system. The apparatus and method described herein are preferably implemented in software, but may of course also be implemented in hardware, all within the scope of the application.
Referring to fig. 1, fig. 1 is a schematic diagram of a first structure of an electronic device according to an embodiment of the application. The electronic device may be a smart phone, a smart wearable device, a tablet computer, a PDA (Personal DIGITAL ASSISTANT), etc. The following will describe in detail.
The electronic device comprises an image sensor 10 and an image Processing chip (PRE IMAGE SIGNAL Processing chip, preISP) 20, when a user shoots or takes a video through the electronic device, the electronic device can acquire image data through the image sensor and send the acquired image data to the image Processing chip for Processing, and the image Processing chip can carry out enhancement Processing on the image data.
The image sensor may be a Complementary Metal Oxide Semiconductor (CMOS) image sensor, a charge coupled device (Charge Coupled Device, CCD) image sensor, or the like. The preview image collected by the image sensor is original image data, such as RAW data, that is, image data that has not been processed by the image processor.
The image sensor 10 may be electrically connected to the image processing chip 20, and the image sensor may collect RAW image data such as RAW image data and transmit to the image processing chip 20 for processing by an image processor inside the image processing chip 20.
The image processing chip 10 may run a first operating system and a second operating system, and the start-up time of the first operating system is smaller than the start-up time of the second operating system. The first operating system and the second operating system are different operating systems, and the first operating system requires less resources than the second operating system. The resource may be a storage space, an operation frequency, a required electric quantity, and the like. For example, the first operating system may be a real-time operating system (Real Time Operating System, RTOS). The second operating system may be a Time-sharing operating system (Time-sharing operating System). The time-sharing operating system may include an android operating system, a Linux operating system, a Windows operating system or an IOS operating system. It should be noted that the first operating system and the second operating system are two independent operating systems.
It can be understood that when the user turns on the camera, the image processing chip 20 needs to be started to process the image data acquired by the image sensor, and in the process of starting the image processing chip 20, the image processing chip 20 can be normally started only by starting the operating system corresponding to the image processing chip 20, and the received image data can be enhanced only by the image processing chip 20 under the operation of the operating system.
The existing image processing chip can only run the second operating system, but the second operating system is started too long, so that after a user turns on a camera, the image processing chip cannot work normally and cannot process image data within the starting time of the second operating system.
In this regard, the present application provides an image processing chip 20 capable of running a first operating system and a second operating system, where the image processing chip is capable of running the first operating system and the second operating system, and the startup time of the first operating system is less than the startup time of the second operating system. The image processing chip runs the first operating system with shorter starting time during the starting period of the second operating system, and processes the image data through the image processing chip under the operation of the first operating system, so that the image processing chip during the starting period of the second operating system can normally process the image processing work, and the influence of the long starting time of the second operating system on the image processing work of the image processing chip is reduced.
Specifically, referring to fig. 2, fig. 2 is a flowchart of an image processing chip for starting an operating system according to an embodiment of the present application, where the image processing chip is an image processing chip 10 in the electronic device shown in fig. 1, and the steps of starting a first operating system and a second operating system by the image processing chip 10 are as follows:
S101, the image processing chip receives a first starting instruction and starts a first operating system.
When the image processing chip receives a first starting instruction, the image processing chip reads a first configuration file of a first operating system from a first memory to start the first operating system.
S102, the image processing chip under the operation of the first operating system receives the image data and processes the image data.
The image processing chip under the operation of the first operating system receives the image data collected by the image sensor and processes the image data. The image processing chip 20 may include an image processor that may process image data, such as RAW data, acquired by the image processing chip 20 to enhance the quality of the image data.
S103, the image processing chip under the operation of the first operating system receives a second starting instruction in the process of processing the image data, and starts the second operating system so as to switch the operating system from the first operating system to the second operating system.
The image processing chip receives a second starting instruction in the process of processing the image data under the operation of the first operating system, and reads a second configuration file corresponding to the second operating system from the second memory to start the second operating system.
In some embodiments, the first memory and the second memory may be the same preset memory in the image processing chip, the first configuration file corresponding to the first operating system is stored in a first storage location in the preset memory, and the second configuration file corresponding to the second operating system is stored in a second storage location in the preset memory.
Of course, in some embodiments, the first memory and the second memory may be different memories in the image processing chip. Referring to fig. 3, fig. 3 is a schematic diagram of a first structure of an image processing chip according to an embodiment of the application.
The image processing chip includes a central processing unit 21, a first memory 22 and a second memory 23, wherein the first memory 22 is used for storing a first configuration file corresponding to a first operating system, and the second memory 23 is used for storing a second configuration file corresponding to a second operating system. The memory space of the first memory 22 is smaller than the memory space of the second memory 23.
The first Memory 22 is a Static Random-Access Memory (SRAM), and the second Memory 23 is a dynamic Random-Access Memory (Dynamic Random Access Memory, DRAM).
The first memory 22 uses the flip-flop as the storage unit to store 1 and 0, so that the access speed is high, and the content can be kept unchanged continuously as long as the power is not lost. While the basic memory circuit of the second memory 23 is the capacitance of the band drive transistor. The presence or absence of charge on the capacitor is considered to be logic 1 and 0. Over time, the charge on the capacitor gradually decreases and must be periodically refreshed to maintain its contents in order to maintain the data stored therein.
When the central processor 21 of the image processing chip in the embodiment of the application receives the first starting instruction, the first configuration file corresponding to the first operating system is read from the first memory 22 so as to quickly start the first operating system, so that the image processing chip under the operation of the first operating system can timely process the image data acquired by the image sensor.
It will be appreciated that since the first memory 22 is static random access memory, there is no need to initialize the first memory 22 prior to the start-up of the first operating system. Therefore, when the image processing chip receives the first start instruction, the first configuration file of the first operating system may be directly read from the first memory 22 to start the first operating system. Because the reading speed of the first memory 22 is faster and no initialization is required, the embodiment of the application uses the static random access memory to store the first configuration file of the first operating system, which can further shorten the starting time of the first operating system.
It should be noted that, since the first operating system is a real-time operating system, the real-time operating system has a small volume and short starting time, and is suitable for performing some simple configuration and calculation work, while the second operating system is a time-sharing operating system, which is relatively huge, and is suitable for multitasking and scheduling and complex calculation work. Therefore, when the image processing chip receives the multiplexed image processing job or when the central processor 21 of the image processing chip needs to perform the complex image processing calculation job, since the first operating system cannot normally process a large amount of calculation jobs, the image processing chip under the operation of the first operating system cannot normally process the image data.
Therefore, the image processing chip provided by the embodiment of the application can also run the second operating system, so that when the image processing chip works in the face of large calculation amount, the central processor 21 of the image processing chip can read the second configuration file of the second operating system in the second memory 23 to start the second operating system, thereby realizing the switching of the operating system of the image processing chip from the first operating system to the second operating system, solving the problem that the image processing chip under the first operating system cannot process a large amount of calculation work, and further improving the processing efficiency of the image processing chip.
It should be noted that, the image processing chip may receive the second start instruction at any time after the first operating system is started, and further start the second operating system according to the second start instruction. The above-mentioned that when the image processing chip receives a multitasking image processing job or when the central processor of the image processing chip needs to perform a complex image processing calculation job, the second operating system is only one embodiment of the present application.
S104, the image processing chip under the operation of the second operating system receives the image data and processes the image data.
After the first operating system is switched to the second operating system, the image processing chip under the operation of the second operating system continuously receives the image data acquired by the image sensor and performs enhancement processing on the image data.
From the above, the electronic device provided by the embodiment of the application is known. The image processing chip in the electronic equipment can run a first operating system and a second operating system, and the starting time of the first operating system is smaller than that of the second operating system; the image processing chip receives a first starting instruction and starts a first operating system; the image processing chip under the operation of the first operating system receives the image data and processes the image data; the image processing chip under the operation of the first operating system receives a second starting instruction in the process of processing image data, and starts the second operating system so as to switch the operating system from the first operating system to the second operating system; and the image processing chip under the operation of the second operating system continuously receives the image data and processes the image data. Therefore, the image processing chip can process the image data through the image processing chip under the operation of the first operating system during the starting period of the second operating system, so that the image processing chip during the starting period of the second operating system can normally process the image processing work, and the influence of long starting time of the second operating system on the image processing work of the image processing chip is reduced.
In some embodiments, please refer to fig. 4, fig. 4 is a schematic diagram illustrating a second structure of an image processing chip according to an embodiment of the present application. Specifically, the image Processing chip may further include an image signal processor (IMAGE SIGNAL Processing, ISP) 24, a neural network processor (Neural-network Processing Unit, NPU) 25, a digital signal processor (DIGITAL SIGNAL Processing, DSP) 26, and a system bus 27. The image processing chip realizes electrical connection between the image signal processor 24, the neural network processor 25, the digital signal processor 26, the central processing unit 21, the first memory 22 and the second memory 23 through the system bus. The image processing chip 20 can complete processing of image data by an image signal processor 24, a neural network processor 25, and a digital signal processor 26.
The image signal processor 24 is used for performing optimization processing on image data such as RAW data acquired by the image processing chip 20. The image signal processor 24 performs optimization processing on the original video image data, so that the problems of dead pixels and the like existing in the original image data can be solved. The optimized data is transmitted to the neural network processor 25 in the image processing chip 20 for processing, so that convergence of the neural network processor 25 can be quickened, and time for the neural network processor 25 to process one frame of image is prolonged, so that the neural network processor 25 can be ensured to process one frame of video image data within a preset time period. The preset time period is, for example, 33nm (milliseconds).
The image signal processor 24 performs an optimization process on the original video image data may include at least one of dead pixel compensation (Bad Pixel Correction, BPC), linearization (Linearization) processing, black level correction (Black Level Correction, BLC). The algorithm of the image signal processor 24 performing the optimization process on the video image data may include at least one of a black level correction algorithm, a dead pixel compensation algorithm, and a linearization process algorithm. The image signal processor 24 performs a black level correction algorithm to correct the black level of the original video image data, the image signal processor 24 performs a dead pixel compensation algorithm to compensate the dead pixel of the original video image data, and the image signal processor 24 performs a linearization algorithm to linearize the original video image data.
It should be noted that, the optimization of the original video image data by the image signal processor 24 is not limited thereto, and for example, the optimization of the original video image data by the image signal processor 24 may further include at least one of image cropping (loop) and image reduction (Bayerscaler). The algorithm for optimizing the video image data by the image signal processor 24 may include at least one of an image cropping algorithm and an image reduction algorithm.
The neural network processor 25 is used for performing enhancement processing on the image data optimally processed by the image signal processor 24, and can run an artificial intelligence training network processing image algorithm to improve the image quality under the condition of improving the efficiency of processing the image data. The neural network processor 25 processes the image data with high efficiency, and the improvement of the image quality is obvious.
In some embodiments, the neural network processor 25 may be a dedicated processor for processing images, which may be referred to simply as a dedicated processor. Hardening can be performed during hardware configuration such as circuit arrangement and programming, so that stability of the neural network processor 25 during processing of image data can be ensured, and power consumption and time required by the neural network processor 25 for processing of image data can be reduced. It will be appreciated that when the neural network processor 25 is a dedicated processor, it functions to process image data and it is not capable of processing other data such as text information.
The neural network processor 25 may process the image data in such a manner that the data blocks are read in a row manner and processed in a row manner. Such as the neural network processor 25, reads the data blocks in a multi-line manner and processes the data blocks in a multi-line manner. It will be appreciated that a frame of image may have multiple rows of data blocks, i.e., the neural network processor 25 may apply to a portion of a frame of image such asThe frame is processed where n is a positive integer such as 2, 4,5, etc. When the neural network processor 25 does not process all of one frame of image, the neural network processor 25 may have a built-in buffer (collectively referred to as OCM) to store data of a plurality of lines of data blocks processed by the neural network processor 25 during processing of one frame of image. Waiting for the neural network processor 25 to complete processing of one frame of image, the neural network processor 25 may write the processed data into a memory such as the second memory 23 of the image processing chip.
It should be noted that, the neural network processor 25 may complete processing in the Pipeline of the data stream according to a preset time. The preset time is, for example, 30 fps=33 ms (milliseconds). Or the time preset for the neural network processor 25 to process one frame of image is 33ms, so that the neural network processor 25 can realize real-time transmission of data on the basis of rapidly processing the image data.
It will be appreciated that if the neural network processor 25 is a general purpose processor, that is, the neural network processor 25 may process not only image data, but also other data such as text data. When the neural network processor 25 is a general-purpose processor, the image processing method is as follows: and loading a frame of image from a memory storing image data, and performing corresponding algorithm processing on the frame of image. Wherein temporary data calculated by a convolutional layer of the general neural network processor is often required to be saved to the memory during the processing of the general neural network processor. After the general neural network processor processes are completed, data is stored into the memory. Therefore, compared with the general neural network processor, the neural network processor 25 defined in some embodiments of the present application is a special neural network processor, which can accelerate the processing speed of the image data and ensure that the processing of one frame of image is completed within a preset time.
The neural network processor 25 may process still image data, such as still image data acquired by a user in a photographing mode. The neural network processor 25 may include algorithms that process static image data, such as a static image HDR algorithm, a static image night scene algorithm, a static image blurring algorithm, and the like.
The neural network processor 25 may also process moving image data, such as video image data acquired by a user in a shooting video mode. The neural network processor 25 may include algorithms that process video image data, such as a video night scene algorithm, a video HDR algorithm, a video blurring algorithm, a video noise reduction algorithm, a video super-resolution algorithm, and the like. The dynamic image data may include image data of recorded video and image data of video playing.
The digital signal processor 26 is mainly used for assisting the image signal processor 24 and the neural network processor 25. However, the digital signal processor 26 may process image data with a small calculation amount. For example, when performing zero-lag (ZSL) frame picking photographing through the image processing chip, the digital signal processor 26 may calculate a preset general algorithm to select a suitable target image from the multi-frame preview image as the photographed image. The preset general algorithm may include a blur detection algorithm, an image quality detection algorithm, and the like. It should be noted that in some cases, the neural network processor 25 cannot support some algorithms, such as for a camera with an ultra-wide angle, and if the deformity correction process is required, the neural network processor 25 may not be implemented, and the digital signal processor 26 may be used for processing.
It should be noted that the processor included in the image processor is not limited thereto, such as the image processor includes only the image signal processor 24 and the neural network processor 25.
Referring to fig. 5, fig. 5 is a schematic diagram of a second structure of an electronic device according to an embodiment of the application. The electronic device provided in this embodiment is different from the electronic device shown in fig. 1 in that the electronic device further includes an application processing chip, and the image processing chip 20 is electrically connected with the application processing chip 40, so that after the image is enhanced by the image processing chip 20, the enhanced image may be sent to the application processing chip 40 for further processing, and the application processing chip may display the processed image data on a display screen of the electronic device, so as to implement a camera preview function of the electronic device. Meanwhile, the application processing chip is also used for controlling the image processing chip to start the first operating system and the second operating system.
The application Processing chip 30 may include an application processor (Application Processor, AP), an image processor (IMAGE SIGNAL Processing, ISP), not shown.
The application processor is a control center of the electronic device 100, connects various parts of the entire electronic device 100 using various interfaces and lines, and performs various functions of the electronic device 100 and processes data by running or calling a computer program stored in a memory of the application processing chip and calling data stored in the memory, thereby performing overall monitoring of the electronic device 100. The image sensor 10 of the electronic device 100 is controlled to acquire raw image data, such as by the application processor 31, and the image processing chip 20 is controlled to perform image quality enhancement processing on the raw image data acquired by the image sensor 10 to obtain a preprocessing result. Then, the application processor controls the image processor of the application processing chip 30 to further process the pre-processing result, and then displays the result processed by the image processor as a preview image through the display screen of the electronic device 100.
The image processing chip 20 and the application processing chip 30 may be integrated on a circuit board, which may be a motherboard of the electronic device 100. It should be noted that, the image sensor in the camera 10 may be integrated on a circuit board together with the image processing chip 20 and the application processing chip 30, and of course, the image sensor in the camera 10 may also be separately disposed from the circuit board integrated with the image processing chip 20 and the application processing chip 30, and connected by using a wire.
The image processing chip 20 in the electronic device according to the embodiment of the present application may be any one of the image processing chips shown in fig. 1 to 4.
With continued reference to fig. 3, as shown in fig. 3, the first memory 22 and the second memory 23 of the image processing chip 20 shown in fig. 3 are volatile memories (Random Access Memory, RAM), and the RAM cannot retain data when the power is turned off. Therefore, before the image processing chip 20 starts the first operating system or starts the second operating system, it is necessary to receive the first configuration file corresponding to the first operating system or the second configuration file corresponding to the second operating system sent by the application processing chip 30, store the first configuration file into the first memory 22, and store the second configuration file into the second memory 23.
In some embodiments, before sending the first start instruction, the application processing chip 20 may send the first configuration file of the first operating system to the first memory 22 of the image processing chip 20, for example, when the application processing chip 30 receives the start instruction of the camera. Currently, the application processing chip 30 may also send the first configuration file of the first operating system to the first memory 22 of the image processing chip together with the first start instruction.
Further, since the second memory 23 is a dynamic random access memory, before the second configuration file is stored in the second memory 23, the cpu 21 of the image processing chip 20 also needs to initialize the memory space of the second memory 23, and in the initialization process, it is necessary to perform a memory parameter test on the second memory 23, so that the second memory 23 works more stably.
Therefore, after the first operating system is operated, the image processing chip in the electronic device provided by the embodiment of the application can initialize the second memory in advance, so that the initialized second memory can receive the second configuration file corresponding to the second operating system sent by the application processing chip.
In some embodiments, after detecting that the initialization of the second memory is completed, the central processor of the image processing chip may send a request for the second configuration file of the second operating system to the application processing chip, and the application processing chip sends the second configuration file of the second operating system to the second memory of the image processing chip according to the request.
Referring to fig. 6, fig. 6 is a schematic flow chart of processing image data by the electronic device shown in fig. 5. Specifically, the electronic device of fig. 5 processes the image data as follows:
s201, the image processing chip receives a first starting instruction sent by the application processing chip and starts a first operating system.
When the image processing chip receives a first starting instruction sent by the application processing chip, the image processing chip reads a first configuration file of a first operating system from a first memory to start the first operating system. After the first operating system is started, the application processing chip wakes up the image processing chip successfully, and the image processing chip processes the received image data to enhance the image quality of the image data.
In some embodiments, when the application processing chip receives the start instruction of the camera, a first start instruction is generated, and the first start instruction is sent to the image processing chip, so that the image processing chip can start the first operating system according to the first start instruction.
When the application processing chip receives an opening instruction of the camera, the user is indicated to start the camera function to prepare for photographing or taking a video. Therefore, the first starting instruction can be generated immediately when the starting instruction of the camera is received, so that the problem that the image processing chip cannot process image data in the process of starting the operating system is solved to the greatest extent when the first operating system of the image processing chip is simultaneously started in the process of starting hardware equipment such as an image sensor and the like by applying the processing chip after the camera is started by a user.
In some embodiments, the electronic device may also record usage information of each application in the electronic device by a user, and learn historical usage information of each application through a machine learning algorithm. And predicting time information, position information and a target application associated with the camera application of the user using the camera application through the historical use information. The application processing chip can generate a first starting instruction in advance according to the time information, the position information and the target application related to the camera application, so that the image processing chip can finish starting of an operating system before the camera is started, and once the application processing chip receives the camera starting instruction, the image processing chip can immediately perform enhancement processing on image data acquired by the image sensor so as to improve the processing efficiency of the image processing chip.
For example, the application processing chip obtains time information and position information according to prediction, and when any one of the current time information and position information of the electronic equipment meets the requirement, a first starting instruction is generated. Or after the application processing chip detects the starting instruction of the target application associated with the camera application, the application processing chip directly generates a first starting instruction. Wherein the target application associated with the camera application is obtained by analysis of historical usage information of the application, the application may be determined to be the target application associated with the camera application if the user opens the camera application multiple times during use of the application.
S202, the image processing chip under the operation of the first operating system receives the image data and processes the image data.
The image processing chip under the operation of the first operating system receives the image data collected by the image sensor and processes the image data. The image processing chip 20 may include an image processor that may process image data, such as RAW data, acquired by the image processing chip 20 to enhance the quality of the image data.
The process of processing the image data by the image processing chip under the operation of the first operating system is referred to the embodiment shown in fig. 4, and will not be described herein.
S203, the image processing chip under the operation of the first operating system sends the processed image data to the application processing chip, so that the application processing chip processes the processed image data again.
The image processing chip under the operation of the first operating system sends the processed image data to the application processing chip, the image processor in the application processing chip reprocesses the processed image data, and the image data processed by the image processor is displayed on a display screen of the electronic device as a preview image.
S204, receiving a second starting instruction sent by the application processing chip by the image processing chip in the process of processing the image data by the image processing chip under the operation of the first operating system, and starting the second operating system so as to switch the operating system from the first operating system to the second operating system.
The image processing chip receives a second starting instruction in the process of processing the image data under the operation of the first operating system, and reads a second configuration file corresponding to the second operating system from the second memory to start the second operating system. The starting time of the first operating system is smaller than that of the second operating system.
S205, the image processing chip under the operation of the second operating system receives the image data and processes the image data.
After the first operating system is switched to the second operating system, the image processing chip under the operation of the second operating system continuously receives the image data acquired by the image sensor and performs enhancement processing on the image data.
S206, the image processing chip under the operation of the second operating system sends the processed image data to the application processing chip, so that the application processing chip processes the processed image data again.
The image processing chip under the operation of the second operating system sends the processed image data to the application processing chip, the image processor in the application processing chip reprocesses the processed image data, and the image data processed by the image processor is displayed on a display screen of the electronic device as a preview image.
From the above, the electronic device provided by the embodiment of the application is known. The electronic device comprises an image processing chip and an application processing chip; the image processing chip receives a first starting instruction sent by the application processing chip and starts a first operating system; the image processing chip under the operation of the first operating system receives the image data and processes the image data; the image processing chip under the operation of the first operating system sends the processed image data to the application processing chip, so that the application processing chip processes the processed image data again; the image processing chip under the operation of the first operating system receives a second starting instruction sent by the application processing chip in the process of processing the image data, and starts the second operating system so as to switch the operating system from the first operating system to the second operating system; the image processing chip under the operation of the second operating system receives the image data and processes the image data; the image processing chip under the operation of the second operating system sends the processed image data to the application processing chip, so that the application processing chip processes the processed image data again; the starting time of the first operating system is smaller than that of the second operating system. Therefore, the image processing chip can process the image data through the image processing chip under the operation of the first operating system during the starting period of the second operating system, so that the image processing chip during the starting period of the second operating system can normally process the image processing work, and the influence of long starting time of the second operating system on the image processing work of the image processing chip is reduced.
In some embodiments, when the application processing chip receives the image signal of the first quality, a first start instruction is sent to the image processing chip, so that the image processing chip starts the first operating system according to the first start instruction; when the application processing chip receives the image signal with the second quality, a second starting instruction is sent to the image processing chip, so that the image processing chip starts a second operating system according to the second starting instruction.
When a user starts the camera, the application processing chip receives an image signal of a first quality in a normal photographing mode. When the user starts the camera and starts the advanced photographing mode, the application processing chip receives the image signal of the second quality. The photographing parameters in the normal photographing mode are preset, the image processing chip only adopts a common algorithm to process the received image data in the normal photographing mode, and the calculated amount in the image processing process is small. However, in the advanced photographing mode, the image processing chip needs to perform enhancement processing on the image data by adopting an advanced algorithm, for example, a preset Super-Resolution (SR) neural network is used to process the target image, so as to improve the Resolution of the image. Or Noise Reduction (NR) neural network is adopted to perform Noise Reduction processing on the target image so as to eliminate Noise points in the target image and improve the quality of the image. Or a preset High dynamic light rendering (High-DYNAMIC RANGE, HDR) neural network is adopted to process the target image so as to improve the dynamic range and the image detail of the target image. Or processing the target image by adopting a preset Seg-Depth neural network so as to improve the quality of the image. Or performing object Detection processing on the target image by adopting a Detection neural network, processing the target image by adopting a saliency neural network to obtain the remarkable information of the target image, and performing enhancement processing on the remarkable information to enhance the visual effect of the image. It will be appreciated that the neural network processor may employ one or more predetermined processing algorithms to enhance the target image.
Therefore, the calculated amount of the image processing chip central processing unit in the advanced photographing mode for image data processing is increased, and the image processing chip in the first operating system running cannot normally process the acquired image data of the second quality in the advanced photographing mode due to low processing efficiency of the first operating system for a large task with the calculated amount. At this time, the application processing chip sends a second start instruction to the image processing chip, so that the image processing chip starts the second operating system according to the second start instruction.
The image quality of the preview image finally displayed on the display screen in the normal photographing mode is inferior to that of the preview image displayed on the display screen in the advanced photographing mode. The advanced photographing mode comprises one or more modes of video extreme darkness mode, video super night scene, video zooming clear video blurring and field Jing Jiangzao and brightness enhancement real-time processing in a photographing scene.
Therefore, when the user starts the advanced photographing mode, the calculated amount of the image processing chip for processing the image data increases, and at this time, the processing efficiency of the image processing chip for processing the image data under the operation of the first operating system decreases.
When the application processing chip receives the image signal with the first quality, the image sensor is controlled to acquire the image data with the first quality; when the application processing chip receives the image signal with the second quality, the image sensor is controlled to acquire the image data with the second quality, wherein the image quality of the image data with the second quality is higher than that of the image data with the first quality.
In some embodiments, when the image sensor 10 of the electronic device 100 starts to collect image data, the application processing chip 30 is further configured to detect whether the operating system of the image processing chip is started successfully, where the operating system is a first operating system or a second operating system;
when the success of the starting of the operating system of the image processing chip 20 is detected, the application processing chip controls the image sensor to send the acquired image data to the image processing chip for processing;
The image processing chip 20 transmits the processed image data to the application processing chip 30, so that the application processing chip 30 processes the processed image data again;
When it is detected that any of the operating systems of the image processing chip 20 is not started successfully, the application processing chip controls the image sensor 10 to send the acquired image data to the application processing chip 30, and the image data is directly processed through the application processing chip 30.
When the application processing chip controls the image sensor to collect image data, whether an operating system (namely a first operating system or a second operating system) of the image processing chip is started successfully is detected, and when the operating system of the image processing chip is detected to be started successfully, the image processing chip after the operating system is operated has the image data processing capability. The application processor controls the image sensor to send the acquired image data to the image processing chip for processing.
However, when the application processing chip detects that any operation system of the image processing chip is not started successfully, the image processing chip cannot normally process data at the moment. At this time, in order to enable the image data collected by the image sensor to be displayed on the display screen in time, the application processing chip in the embodiment of the application controls the image sensor to send the collected image data to the application processing chip, processes the image data directly through the application processing chip, and displays the processed image data on the display screen.
The image sensor can send the collected image data to the application processing chip through a bypass path between the image sensor and the application processing chip. bypass is a bypass function, that is, two networks can be directly and physically conducted through a specific trigger state (power-off or dead halt) without passing through a system of network security equipment, so that after a bypass passage is provided, after the network security equipment fails, the networks connected to the network security equipment can be conducted mutually.
In some embodiments, the image processing chip 20 is further configured to detect interrupt data during the running process of the first operating system when the first operating system is running;
When the image processing chip 20 detects that the interrupt data in the running process of the first operating system is larger than a preset value, the second operating system is started so as to switch the operating system from the first operating system to the second operating system.
The starting of the real-time operating system of the image processing chip needs to establish management information such as memory information, clock information, thread information, interrupt information and the like. For example, during a real-time operating system boot process, a memory management unit (MMU, memory Management Unit) page table needs to be established, and information such as a system thread unit, a clock management unit, an interrupt unit, etc. needs to be established.
When the image processing chip detects that interrupt data collected by an interrupt unit of a real-time operating system of the image processing chip is abnormal, for example, when the interrupt data reported by the interrupt unit exceeds a preset value, the image processing chip indicates that the real-time operating system of the image processing chip is abnormal in operation. The image processing chip under the operation of the real-time operating system cannot normally process the image data, and at the moment, the central processor of the image processing chip reads the second operating system (namely the time-sharing operating system) and processes the received image data through the image processing chip under the operation of the second operating system.
In some embodiments, the image processing chip 20 may also detect interrupt data during the running of the first operating system by applying the processing chip 30 while running the first operating system;
When the application processing chip 30 detects that the interrupt data in the running process of the first operating system is greater than a preset value, a second starting instruction is generated, so that the image processing chip starts the second operating system according to the second starting instruction, and the operating system is switched from the first operating system to the second operating system.
In some embodiments, the image processing chip 20 sends a start request of the second operating system to the application processing chip 30 after the first operating system starts;
The application processing chip 30 sends a second start instruction to the image processing chip 20 according to the start request of the second operating system, so that the image processing chip 20 starts the second operating system according to the second start instruction to switch the operating system from the first operating system to the second operating system.
The image processing chip completes enhancement processing of the received image data during the period of starting the second operating system according to the second starting instruction or through the image processing chip operated by the first operating system. And only after the second operating system is started, the processing work of the image data is transmitted to the image processing chip under the operation of the second operating system for processing.
In some embodiments, referring to fig. 7, fig. 7 is a schematic diagram of a third structure of an electronic device according to an embodiment of the application. The electronic device further includes: radio frequency circuitry 401, touch screen 402, control circuitry 403, input unit 404, audio circuitry 405, sensors 406, touch circuitry 407, and power supply 408. The first control chip 501 and the second control chip 502 are electrically connected to the radio frequency circuit 401, the touch screen 402, the control circuit 403, the input unit 404, the audio circuit 405, the sensor 406, the touch circuit 407, and the power supply 408, respectively.
The radio frequency circuit 401 is used to transmit and receive radio frequency signals to communicate with a network device or other electronic device through wireless communication.
The touch screen 402 may be used to display information entered by a user or provided to a user as well as various graphical user interfaces of an electronic device, which may be composed of images, text, icons, video, and any combination thereof.
The control circuit 403 is electrically connected to the touch screen 402, and is used for controlling the touch screen 402 to display information.
The input unit 404 may be used to receive input numbers, character information, or user characteristic information (e.g., a fingerprint), and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control. The input unit 404 may include a fingerprint recognition module.
The audio circuitry 405 may provide an audio interface between the user and the electronic device through a speaker, microphone. Wherein the audio circuit 405 comprises a microphone. The microphone is electrically connected to the processor 501. The microphone is used for receiving voice information input by a user.
The sensor 406 is used to collect external environmental information. The sensor 406 may include one or more of an ambient brightness sensor, an acceleration sensor, a gyroscope, and the like.
The power supply 408 is used to power the various components of the electronic device. In some embodiments, the power supply 408 may be logically connected to the processor 501 through a power management system, so as to perform functions of managing charging, discharging, and power consumption management through the power management system.
Although not shown in fig. 7, the electronic device may further include a bluetooth module, etc., and will not be described herein.
As can be seen from the above, the embodiment of the present application provides an electronic device, which includes an image processing chip and an application processing chip; the image processing chip receives a first starting instruction sent by the application processing chip and starts a first operating system; the image processing chip under the operation of the first operating system receives the image data and processes the image data; the image processing chip under the operation of the first operating system sends the processed image data to the application processing chip, so that the application processing chip processes the processed image data again; the image processing chip under the operation of the first operating system receives a second starting instruction sent by the application processing chip in the process of processing the image data, and starts the second operating system so as to switch the operating system from the first operating system to the second operating system; the image processing chip under the operation of the second operating system receives the image data and processes the image data; the image processing chip under the operation of the second operating system sends the processed image data to the application processing chip, so that the application processing chip processes the processed image data again; the starting time of the first operating system is smaller than that of the second operating system. Therefore, the image processing chip can process the image data through the image processing chip under the operation of the first operating system during the starting period of the second operating system, so that the image processing chip during the starting period of the second operating system can normally process the image processing work, and the influence of long starting time of the second operating system on the image processing work of the image processing chip is reduced.
The embodiment of the application also provides a storage medium, wherein a computer program is stored in the storage medium, and when the computer program runs on a computer, the computer executes the following steps:
the image processing chip receives a first starting instruction and starts a first operating system;
The image processing chip under the operation of the first operating system receives the image data and processes the image data;
the image processing chip under the operation of the first operating system receives a second starting instruction in the process of processing image data, and starts the second operating system so as to switch the operating system from the first operating system to the second operating system;
The image processing chip under the operation of the second operating system receives the image data and processes the image data.
It should be noted that, those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments may be implemented by a computer program, which may be stored in a computer readable storage medium, and the storage medium may include, but is not limited to: read Only Memory (ROM), random access Memory (RAM, random Access Memory), magnetic or optical disk, and the like.
Furthermore, the terms "first," "second," and "third," and the like, herein, are used for distinguishing between different objects and not for describing a particular sequential order. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or modules is not limited to the particular steps or modules listed and certain embodiments may include additional steps or modules not listed or inherent to such process, method, article, or apparatus.
The image processing method, the storage medium and the electronic device provided by the embodiment of the application are described in detail. The principles and embodiments of the present application have been described herein with reference to specific examples, the description of which is intended only to assist in understanding the methods of the present application and the core ideas thereof; meanwhile, as those skilled in the art will vary in the specific embodiments and application scope according to the ideas of the present application, the present description should not be construed as limiting the present application in summary.

Claims (10)

1. An image processing chip, wherein the image processing chip can run a first operating system and a second operating system, and the starting time of the first operating system is less than the starting time of the second operating system;
the image processing chip receives a first starting instruction and starts a first operating system;
the image processing chip under the operation of the first operating system receives image data and processes the image data;
The image processing chip under the operation of the first operating system receives a second starting instruction in the process of processing image data, and starts the second operating system so as to switch the operating system from the first operating system to the second operating system;
and the image processing chip under the operation of the second operating system receives the image data and processes the image data.
2. The image processing chip of claim 1, wherein the image processing chip comprises a central processor and a first memory;
the first memory is used for storing a first configuration file corresponding to the first operating system;
The central processing unit is used for receiving a first starting instruction, and loading the first configuration file according to the first starting instruction so as to start the first operating system.
3. The image processing chip of claim 2, further comprising a second memory, wherein the second memory is configured to store a second configuration file corresponding to the second operating system, and a memory space of the second memory is larger than a memory space of the first memory;
And the central processing unit is also used for receiving a second starting instruction in the process of processing the image data by the image processing chip operated by the first operating system, and starting the second operating system according to the second starting instruction so as to switch the operating system from the first operating system to the second operating system.
4. An electronic device, characterized in that the electronic device comprises an image processing chip and an application processing chip;
The image processing chip receives a first starting instruction sent by the application processing chip and starts a first operating system;
the image processing chip under the operation of the first operating system receives image data and processes the image data;
The image processing chip under the operation of the first operating system sends the processed image data to the application processing chip, so that the application processing chip carries out reprocessing on the processed image data;
The image processing chip under the operation of the first operating system receives a second starting instruction sent by the application processing chip in the process of processing image data, and starts a second operating system so as to switch the operating system from the first operating system to the second operating system;
The image processing chip under the operation of the second operating system receives the image data and processes the image data;
The image processing chip under the operation of the second operating system sends the processed image data to the application processing chip, so that the application processing chip carries out reprocessing on the processed image data;
the starting time of the first operating system is smaller than that of the second operating system.
5. The electronic device of claim 4, wherein when the application processing chip receives the image signal of the first quality, a first start instruction is sent to the image processing chip, so that the image processing chip starts a first operating system according to the first start instruction;
And when the application processing chip receives the image signal with the second quality, a second starting instruction is sent to the image processing chip, so that the image processing chip starts a second operating system according to the second starting instruction.
6. The electronic device of claim 5, further comprising an image sensor, wherein the application processing chip is configured to control the image sensor to collect image data of a first quality when receiving the image signal of the first quality;
when the application processing chip receives the image signal with the second quality, the image sensor is controlled to acquire the image data with the second quality, wherein the image quality of the image data with the second quality is higher than that of the image data with the first quality.
7. The electronic device of claim 6, wherein when the image sensor starts to collect image data, the application processing chip is further configured to detect whether the image processing chip is started successfully, and the operating system is a first operating system or a second operating system;
when the successful starting of the operating system of the image processing chip is detected, the application processing chip controls the image sensor to send the acquired image data to the image processing chip for processing;
The image processing chip sends the processed image data to the application processing chip, so that the application processing chip processes the processed image data again;
When the fact that any operation system of the image processing chip is not started successfully is detected, the application processing chip controls the image sensor to send collected image data to the application processing chip, and the image data is processed directly through the application processing chip.
8. The electronic device of claim 4, wherein the image processing chip is further configured to detect interrupt data during operation of the first operating system when the first operating system is running;
when the image processing chip detects that the interrupt data in the running process of the first operating system is larger than a preset value, the second operating system is started so as to switch the operating system from the first operating system to the second operating system.
9. The electronic device of claim 4, wherein the image processing chip sends a start-up request of a second operating system to the application processing chip after the first operating system is started up;
the application processing chip sends a second starting instruction to the image processing chip according to the starting request of the second operating system, so that the image processing chip starts the second operating system according to the second starting instruction, and the operating system is switched from the first operating system to the second operating system.
10. The electronic device of any of claims 4-9, wherein the image processing chip comprises a first memory and a second memory;
the first memory is used for receiving a first configuration file corresponding to a first operating system sent by the application processing chip;
the second memory is used for receiving a second configuration file corresponding to a second operating system sent by the application processing chip.
CN202010478357.6A 2020-05-29 2020-05-29 Image processing chip and electronic device Active CN113744116B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010478357.6A CN113744116B (en) 2020-05-29 2020-05-29 Image processing chip and electronic device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010478357.6A CN113744116B (en) 2020-05-29 2020-05-29 Image processing chip and electronic device

Publications (2)

Publication Number Publication Date
CN113744116A CN113744116A (en) 2021-12-03
CN113744116B true CN113744116B (en) 2024-07-16

Family

ID=78724885

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010478357.6A Active CN113744116B (en) 2020-05-29 2020-05-29 Image processing chip and electronic device

Country Status (1)

Country Link
CN (1) CN113744116B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107870823A (en) * 2016-09-27 2018-04-03 富士施乐株式会社 Electronic installation and image processing apparatus
CN108932160A (en) * 2017-10-10 2018-12-04 北京猎户星空科技有限公司 Multiple operating system control method, device, electronic equipment and computer storage medium

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2015150872A1 (en) * 2014-04-02 2015-10-08 Continental Automotive Gmbh Early logo display in a multiprocessor architecture
CN108932213A (en) * 2017-10-10 2018-12-04 北京猎户星空科技有限公司 The means of communication, device, electronic equipment and storage medium between multiple operating system
CN109324903B (en) * 2018-09-21 2021-03-02 深圳前海达闼云端智能科技有限公司 Display resource scheduling method and device for embedded system

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107870823A (en) * 2016-09-27 2018-04-03 富士施乐株式会社 Electronic installation and image processing apparatus
CN108932160A (en) * 2017-10-10 2018-12-04 北京猎户星空科技有限公司 Multiple operating system control method, device, electronic equipment and computer storage medium

Also Published As

Publication number Publication date
CN113744116A (en) 2021-12-03

Similar Documents

Publication Publication Date Title
CN112717370B (en) Control method and electronic equipment
CN110493538B (en) Image processing method, image processing device, storage medium and electronic equipment
US11611715B2 (en) System and method for event camera data processing
CN109167931B (en) Image processing method, device, storage medium and mobile terminal
KR20140128885A (en) Power efficient image sensing apparatus, method of operating the same and eye/gaze tracking system
CN107615745B (en) Photographing method and terminal
CN111479059B (en) Photographing processing method and device, electronic equipment and storage medium
US10769416B2 (en) Image processing method, electronic device and storage medium
CN111768352A (en) Image processing method and device
CN112422798A (en) Photographing method and device, electronic equipment and storage medium
CN115633262A (en) Image processing method and electronic device
CN113923461A (en) Screen recording method and screen recording system
CN108259767B (en) Image processing method, image processing device, storage medium and electronic equipment
CN113744139B (en) Image processing method, device, electronic equipment and storage medium
CN107180417B (en) Photo processing method and device, computer readable storage medium and electronic equipment
CN113744116B (en) Image processing chip and electronic device
WO2018217334A1 (en) Frame buffering technology for camera-inclusive devices
CN114422686A (en) Parameter adjusting method and related device
CN111880661A (en) Gesture recognition method and device
JP2001238189A (en) Image processing apparatus, and operation control method for the same
CN108495038B (en) Image processing method, image processing device, storage medium and electronic equipment
WO2023011302A1 (en) Photographing method and related apparatus
CN113784036B (en) Equipment focusing method and device, storage medium and electronic equipment
CN114286004A (en) Focusing method, shooting device, electronic equipment and medium
CN112163985B (en) Image processing method, image processing device, storage medium and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant