CN113873158A

CN113873158A - Electronic equipment and control method thereof

Info

Publication number: CN113873158A
Application number: CN202111159893.0A
Authority: CN
Inventors: 刘金; 于海
Original assignee: Lenovo Beijing Ltd
Current assignee: Lenovo Beijing Ltd
Priority date: 2021-09-30
Filing date: 2021-09-30
Publication date: 2021-12-31

Abstract

The application provides an electronic device, including: the camera module is used for acquiring original image data; the first processor at least can control the camera module to collect the original image data; the second processor is in signal connection with the first processor and is at least used for processing the obtained original image data into target image data; the second processor obtains the original image data through a corresponding first data input channel, and the first data input channel does not pass through a memory of the electronic device. Meanwhile, the application also provides a control method.

Description

Electronic equipment and control method thereof

Technical Field

The present disclosure relates to control methods, and particularly to an electronic device and a control method thereof.

Background

When real-time video processing is performed using a Personal Computer (PC), if high-resolution and high-frame-rate video data is processed, the processing delay tends to increase significantly.

Disclosure of Invention

In view of the above, embodiments of the present application are directed to providing an electronic device.

The technical scheme of the application is realized as follows:

according to an aspect of the present application, there is provided an electronic apparatus including:

the camera module is used for acquiring original image data;

the first processor at least can control the camera module to collect the original image data;

the second processor is in signal connection with the first processor and is at least used for processing the obtained original image data into target image data;

the second processor obtains the original image data through a corresponding data input channel, and the data input channel does not pass through a memory of the electronic device.

In the above solution, the electronic device further includes a south bridge chip in signal connection with the first processor, the camera module includes a first camera, the first camera can collect the original image data, the first camera is connected to the south bridge chip through a first interface, the second processor is in signal connection with the first processor through a communication bus or a second interface, and the second processor can obtain the original image data sent from the south bridge chip to the first processor through the communication bus or the second interface.

In the above scheme, the camera module includes a second camera, the second camera can collect the original image data, the second camera is connected to the first processor through a third interface, the second processor is in signal connection with the first processor through a communication bus or a second interface, and the second processor can obtain the original image data through a data channel formed by the communication bus and the third interface; or, the second processor can obtain the original image data through a data channel formed by the second interface and the third interface.

In the above scheme, the camera module includes a third camera, the third camera is connected to the second processor through a fourth interface, and the second processor obtains the original image data collected by the third camera through the fourth interface.

In the above scheme, the first processor sends the target image data to a data sharing module through a corresponding data output channel, so as to send the target image data to a corresponding receiving end device through the data sharing module; or, the first processor sends the target image data to a target application through a corresponding data output channel so as to output the target image data in a window of the target application;

and the data output channel passes through or does not pass through the memory.

According to another aspect of the present application, there is provided a control method including:

if a video stream acquisition instruction is obtained, controlling a camera module of the electronic equipment to acquire original image data;

the original image data is sent to at least one second processor of the electronic equipment through a corresponding data input channel to be processed, and target image data are obtained;

the data input channel does not pass through a memory of the electronic equipment.

In the above scheme, the method further comprises:

and sending the target image data to a target application through a corresponding data output channel so as to display the target image data on a window of the target application, wherein the target application runs on the electronic equipment and/or a receiving terminal device which establishes target communication connection with the electronic equipment.

In the above scheme, if the camera module includes at least two cameras, access information of the cameras is obtained, and a data input channel formed between each camera and the second processor is determined at least based on the access information, so that original image data acquired by different cameras can be sent to the corresponding second processor through the corresponding data input channels.

In the above solution, the sending the original image data to at least one second processor of the electronic device through a corresponding data input channel for processing includes:

acquiring attribute information of each camera, and sending target original image data acquired by a target camera to a corresponding at least one second processor for processing through a corresponding target data input channel at least based on the attribute information, wherein the target camera is a camera which determines that image parameters of the acquired original image data meet a first condition at least based on the attribute information; or the like, or, alternatively,

and obtaining the use information of the electronic equipment, and determining a target data input channel at least based on the use information so as to send the original image data collected by the corresponding camera to the corresponding at least one second processor for processing.

if the original image data is acquired through a first camera in the camera module, the original image data is sent to a first image processor of the electronic equipment through a first data input channel which does not pass through a first processor of the electronic equipment for processing, wherein the first image processor is one of the at least one second processor; or the like, or, alternatively,

if the original image data is acquired through a second camera in the camera module, the original image data is sent to at least one second processor of the electronic equipment through a second data input channel passing through a first processor of the electronic equipment for processing; or the like, or, alternatively,

and if the original image data is acquired by a first camera and a second camera in the camera module, the original image data is sent to a first image processor and a second image processor for processing through a corresponding first data input channel and a corresponding second data input channel, and the second image processor is one of the at least one second processor different from the first image processor.

The application provides an electronic device and a control method thereof, which can reduce the delay caused by repeated transfer, calling or forwarding of original image data between a memory and a second processor and improve the processing efficiency of the original image data by controlling a camera module to collect the original image data and processing the original image data into target image data by at least one second processor through a corresponding data input channel without passing through the memory of the electronic device.

Drawings

FIG. 1 is a first schematic structural component diagram of an electronic device according to the present application;

FIG. 2 is a schematic structural diagram of an electronic device according to the present application;

fig. 3 is a schematic structural composition diagram of an electronic device in the present application;

FIG. 4 is a fourth schematic structural component diagram of an electronic device according to the present application;

fig. 5 is a schematic structural composition diagram of an electronic device in the present application;

fig. 6 is a schematic structural composition diagram six of the electronic device in the present application;

fig. 7 is a seventh structural component schematic diagram of the electronic device in the present application;

fig. 8 is a schematic structural composition diagram eight of the electronic device in the present application;

fig. 9 is a schematic structural composition diagram nine of an electronic device in the present application;

fig. 10 is a schematic view of a flow implementation of the control method in the present application.

Detailed Description

The technical solution of the present application is further described in detail with reference to the drawings and specific embodiments of the specification.

Fig. 1 is a schematic structural composition diagram of an electronic device in the present application, as shown in fig. 1, the electronic device includes: the image processing system comprises a first processor 10, a camera module 20 and a second processor 30, wherein the first processor 10 may specifically be a Central Processing Unit (CPU), and at least can control the camera module 20 to acquire original image data according to a control instruction sent by an upper application; the camera module 10 is in signal connection with the first processor 10, and at least can receive a control instruction of the first processor 10 to collect original image data according to the control instruction of the first processor 10; at least one second processor 30 is in signal connection with the first processor 10, and is at least used for processing the obtained original image data into target image data; the second processor 30 obtains the original image data collected by the camera module 20 through a corresponding data input channel, and the data input channel does not pass through the memory of the electronic device.

Here, the second processor 30 may be a Graphic Processing Unit (GPU) or a Video Processing Unit (VPU). The GPU is also called a display core, a visual processor, and a display chip (including an integrated display card and an independent display card), and is a microprocessor dedicated to image and graphic-related operations on a personal computer, a workstation, a game machine, and some mobile devices (such as a tablet computer, a smart phone, and the like), and the GPU may also include a VPU. The VPU is a brand-new core engine of the video processing platform, and has the capability of hard decoding function and reducing the load of a CPU.

Fig. 2 is a schematic structural diagram of an electronic device in the present application, and as shown in fig. 2, the electronic device further includes a south bridge chip 40 in signal connection with the first processor 10 and a memory 60 in signal connection with the first processor, the camera module 20 may include a first camera 201, and the first camera 201 can collect the original image data under the control of the first processor 10. Specifically, the first camera 201 may be connected to the south bridge chip 40 through a first interface 001, the second processor 30 may be in signal connection with the first processor 10 through a communication bus 002 or a second interface 002, and the second processor 30 may obtain raw image data given from the south bridge chip 40 to the first processor 10 through a data channel formed by the communication bus 002 or the second interface 002 with the south bridge chip 40. Since the second processor 30 obtains the original image data sent from the south bridge chip 40 through the communication bus 002 or the second interface 002 without passing through the memory 60 of the electronic device, the scheme of the present application can reduce the delay of the image data stream and increase the processing speed of the image data.

Here, the first interface 001 and the second interface 002 may be a Universal Serial Bus (USB) interface, a Video Graphics Array (VGA) interface, or the like. The south bridge chip 40 may be specifically connected to the first processor 10 through a Direct Media Interface (DMI) to transmit the original image data to the first processor 10 through the DMI.

Fig. 3 is a schematic structural diagram of an electronic device in the present application, and as shown in fig. 3, the camera module 20 further includes a second camera 202, and the second camera 202 can collect the original image data under the control of the first processor 10. The second camera 202 can be connected to the first processor 10 through the third interface 003, the second processor 30 can be in signal connection with the first processor 10 through the communication bus 002 or the second interface 002, and the second processor 30 can obtain the original image data through the data input channel formed by the communication bus 002 or the second interface 002 and the third interface 003.

Here, the third interface 003 includes, but is not limited to, a lightning interface, a USB interface, and a VGA interface. The USB interface comprises but is not limited to a B-5Pin interface, a B-4Pin interface, a B-8Pin-2 x 4 interface, a Micro USB interface and a Type-C interface.

In the present application, the second processor 30 does not pass through the memory 60 of the electronic device when obtaining the original image data through the communication bus 002 or the data input channel formed by the second interface 002 and the third interface 003, so that the time for transferring, calling or forwarding the original image data between the memory 60 and the second processor 30 can be reduced, thereby improving the processing efficiency of the image data.

Fig. 4 is a schematic view illustrating a structural composition of an electronic device in the present application, and as shown in fig. 4, the camera module 20 further includes a third camera 203, the third camera 203 may be connected to the second processor 30 through a fourth interface 004, and the second processor 30 may directly obtain original image data acquired by the third camera 203 through the fourth interface 004. According to the image processing method and device, the fourth interface 004 is arranged on the second processor 30, so that the second processor 30 directly obtains original image data acquired by the third camera 203 through the fourth interface 004, the original image data do not need to pass through the first processor 10 and the memory 60 of the electronic device, and the processing speed of the image data is greatly improved.

Here, the fourth interface 004 includes, but is not limited to, a thunder interface, a USB interface, a VGA interface, wherein the USB interface includes, but is not limited to, a B-5Pin interface, a B-4Pin interface, a B-8Pin-2 × 4 interface, a Micro USB interface, and a Type-C interface.

In the present application, the first processor 10 may also control the second processor 30 to obtain the raw image data in different data channels according to the current network operation parameters.

In an implementation, the first processor 10 may compare the network operating parameter with a preset network parameter when acquiring the current network operating parameter, and determine that the current network environment is poor if the comparison result indicates that the current network operating parameter is smaller than the preset network parameter, and when receiving, through the DMI interface, the raw image data acquired by the south bridge chip 40 from the first camera 201, the first processor 10 may control the second processor 30 to acquire the raw image data, which is provided from the south bridge chip 40 to the first processor 10, through a data channel formed between the communication bus 002 and the south bridge chip 40, where the data channel does not pass through the memory 60 of the electronic device. If the comparison result indicates that the current network operating parameter is greater than the preset network parameter, it is determined that the current network environment is better, and when the first processor 10 receives the original image data sent by the south bridge chip 40 through the DMI interface, the second processor 30 may be controlled to obtain the original image data sent from the south bridge chip 40 to the first processor 10 through a data channel formed between the first processor 10 and the memory 60.

In another implementation, when the first processor 10 obtains the current network operation parameter, it may further determine a stable condition of the current network operation parameter within a preset time period, and if the current network operation parameter is within a preset parameter range within a preset time (for example, 2 minutes), it is determined that the current network operation parameter is stable, the second processor 30 is controlled to obtain the original image data sent from the south bridge chip 40 to the first processor 10 through a data channel formed between the first processor 10 and the memory 60. If the current network operation parameter frequently exceeds the preset parameter range within the preset time (such as 2 minutes), it is determined that the current network operation parameter is not stable, and the second processor 30 is controlled to obtain the raw image data from the south bridge chip 40 to the first processor 10 through the communication bus 002 or the data channel formed between the second interface 002 and the south bridge chip 40, where the data channel does not pass through the memory 60 of the electronic device. Thus, the processing efficiency of the image data can be improved.

In the present application, the first processor 10 may also control the second processor 30 to obtain the raw image data in different data channels according to the type of the current application.

Specifically, when the first processor 10 receives a control instruction sent by the current application, if the current application is the target application, the second processor 30 is controlled to obtain the raw image data through the data channel that does not pass through the memory 60.

Here, the first processor 10 may search the preset list for the packet name of the current application when receiving the control instruction of the current application, and if the search result indicates that the packet name of the current application is found in the preset list, it is determined that the current application is the target application, the second processor 30 is controlled to obtain the original image data through a data channel that does not pass through the memory 60 (for example, obtain the original image data through a data channel formed between the communication bus 002 and the south bridge chip 40). Conversely, if the query result indicates that the packet name of the current application is not found in the preset list, and it is determined that the current application is a non-target application, the second processor 30 is controlled to obtain the raw image data through the data channel of the memory (for example, the raw image data is obtained through the data channel formed between the communication bus 002 and the memory 60). In this way, free switching of the second processor between different data input channels can be achieved.

According to the method and the device, the camera module is controlled by the first processor to collect the original image data, and under the condition that the original image data do not pass through the memory of the electronic equipment, the original image data are obtained by the at least one second processor through the corresponding data input channel and are processed into the target image data, so that the delay caused by repeated transfer, calling or forwarding of the original image data between the memory and the second processor can be reduced, and the processing efficiency of the original image data is improved.

Fig. 5 is a schematic structural composition diagram of an electronic device in the present application, as shown in fig. 5, the electronic device includes a plurality of cameras (e.g., a first camera 201, a second camera 202, and a third camera 203) and at least two second processors (e.g., a second processor 3A and a second processor 3B), and the at least two cameras are both in an image capturing state (e.g., the first camera 201 and the second camera 202 are in the image capturing state).

For example, when the first processor 10 receives a control instruction sent by the video conference application, the second camera 202 may be controlled to collect original image data based on the control instruction, and since the video conference application has a high requirement on the transmission rate of the image data, at this time, the first processor 10 may send the original image data to the second processor 30A based on a data channel formed between the communication bus 002 and the third interface 003, and at this time, the data channel does not pass through the electronic device memory 60. Therefore, the processing speed of the image data can be improved, and the picture is not blocked in the video conference process. When the first processor 10 receives a control instruction sent by a non-video conference video application (such as a video recording application, an image capturing application, etc.), based on the control instruction, the first camera 201 may be controlled to capture raw image data, and the second processor 30B may be controlled to capture the raw image data from the first processor 10 to the south bridge 40 to the first processor 10 through a data channel formed between the communication bus 002 and the memory 60. The data channel then passes through the memory 60. Of course, when the first processor 10 receives a control instruction sent by a video application other than the video conference, based on the control instruction, the first camera 201 may also be controlled to capture raw image data, and the second processor 30B may be controlled to capture the raw image data sent to the first processor 10 from the south bridge chip 40 through the data channel formed between the communication bus 002 and the south bridge chip 60. The data channel does not go through the memory 60 at this time.

Here, the performance and image processing speed of the second processor 30A are higher than those of the second processor 30B. For example, the second processor 30A may be a stand-alone graphics card, and the second processor 30B may be an integrated graphics card. Of course, the second processor 30A and the second processor 30B may also be independent display cards.

As shown in fig. 5, if at least two cameras of the electronic device are currently in an image capturing state, for example, the first camera 201 and the third camera 203 are simultaneously in an image capturing state, in another implementation scheme, the first processor 10 may further determine which second processor of the at least two second processors 30 processes the original image data of which camera according to a connection object of the at least two cameras.

For example, the connection object of the first camera 201 is the south bridge chip 40, the connection object of the third camera 203 is the second processor 30A, when the first processor 10 receives a control instruction issued by the first application currently in the running state, the third camera 203 may be controlled to acquire original image data based on the control instruction, and since the connection object of the third camera 203 is the second processor 30A, the second processor 30A may directly acquire the original image data acquired by the third camera 203 through the fourth interface 004. If a second application is running at the same time, the first processor 10 may control the first camera 201 to collect the original image data based on the control instruction when receiving the control instruction sent by the second application, and control the second processor 30B to obtain the original image data sent to the first processor 10 from the south bridge chip 40 through the data channel formed between the communication bus 002 and the memory 60 from the first processor 10. The data channel then passes through the memory 60.

Of course, when receiving the control instruction sent by the second application, the first processor 10 may control the first camera 201 to capture the original image data based on the control instruction, and control the second processor 30B to obtain the original image data sent from the south bridge 40 to the first processor 10 from the first processor 10 through the data channel formed between the communication bus 002 and the south bridge 40. The data channel does not go through the memory 60 at this time. Thus, the processing speed of the image data can be increased.

According to the method and the device, different second processors are selected through different strategies to acquire the original image data through different data channels, the requirements of different users and different application scenes can be met, meanwhile, under the condition that the memory of the electronic equipment is not needed, the original image data can be acquired through the corresponding data input channel by at least one second processor and processed into the target image data, and therefore the delay caused by repeated transfer, calling or forwarding of the original image data between the memory and the second processors can be reduced, and the processing efficiency of the original image data is improved.

In the present application, after the second processor 30, such as a Graphic Processing Unit (GPU), processes the raw image data into the target image data, the second processor 30 may also store the target image data in a video memory.

Here, the video memory may also provide the target image data to the data sharing module 50 through the corresponding data channel, so as to provide the target image data to the corresponding receiving end device 70 through the data sharing module 50.

For example, the south bridge chip 40 is connected to the sink device 70 through the data sharing module 50, the video memory is connected to the first processor 10 through the data interface, and after receiving the target image data, the video memory may provide the target image data to the data sharing module 50 through a data channel formed between the data interface and the south bridge chip 40, so as to provide the target image data to the corresponding sink device 70 through the data sharing module 50. Here, the data channel does not pass through the memory 60.

For another example, the first processor 10 is connected to the receiving end device 70 through the data sharing module 50, the video memory may be connected to the first processor 10 through the data interface, and after the video memory receives the target image data, the target image data may be sent to the data sharing module 50 through a data channel formed between the data interface and the data sharing module 50, so as to send the target image data to the corresponding receiving end device 70 through the data sharing module 50. Here, the data channel does not pass through the memory 60.

Here, if the video memory is located in the first processor 10, the video memory may also directly provide the target image data to the corresponding receiving end device 70 through the data sharing module 50.

In this application, the first processor 10 may further receive the target image data sent by the second processor 30, and send the target image data to the data sharing module 50 through the corresponding data output channel, so as to send the target image data to the corresponding receiving end device 70 through the data sharing module 50.

Here, the data sharing module 50 may be a network card, a display Interface (DP), a High Definition Multimedia Interface (HDMI), a Video Graphics Array (VGA) Interface, a Universal Serial Bus Interface (USB PHY), a network card, a PCI-Express (peripheral component interconnect) Interface, or the like. The receiving end device may be a display screen of the electronic device or an external display outside the electronic device.

For example, when an electronic device (e.g., a computer) and a data sink (e.g., a television, a projector) are connected via an HDMI interface, the first processor 10 may provide the target image data to the HDMI interface via a corresponding data output channel, so as to provide the target image data to the sink device via the HDMI interface, thereby achieving the effect of data projection.

For another example, when the electronic device (e.g., a computer host) is connected to a receiving device (e.g., a computer monitor) through a DP interface, the first processor 10 may send the target image data to the DP interface through a corresponding data output channel, so as to send the target image data to the receiving device through the DP interface, so as to achieve the effect of image data output display.

Here, the data output channel may or may not pass through the memory of the electronic device.

In one implementation, as shown in fig. 2, the south bridge chip 40 is connected to the sink device 70 through the data sharing module 50, and the first processor 10 may directly provide the target image data to the data sharing module 50 through a data channel formed by the DMI interface and the data sharing module 50, so as to provide the target image data to the corresponding sink device 70 through the data sharing module 50. Here, the data channel does not pass through the memory 60.

In another implementation, referring to fig. 2, the south bridge chip 40 is connected to the sink device 70 through the data sharing module 50, and when receiving the target image data sent by the second processor 30, the first processor 10 may further send the target image data to the memory 60 through the communication bus 005, and send the target image data to the data sharing module 50 through a data channel formed by the communication bus 005 and the DMI interface, so as to send the target image data to the corresponding sink device through the data sharing module 50. Here, the data channel passes through the memory.

In another implementation, as shown in fig. 3, the first processor 10 is connected to the receiving device 70 through the data sharing module 50, and when receiving the target image data sent by the second processor 30, the first processor 10 directly sends the target image data to the receiving device 70 through the data sharing module 50.

In another implementation, referring to fig. 3, the first processor 10 is connected to the receiving device 70 through the data sharing module 50, and when receiving the target image data sent by the second processor 30, the first processor 10 may further send the target image data to the memory 60 through the communication bus 005, receive the target image data sent by the memory 60 through the communication bus 005, and send the target image data to the receiving device 70 through the data sharing module 50.

In another implementation, as shown in fig. 4, the second processor 30 is connected to the receiving device 70 through the data sharing module 50, and after the second processor 30 processes the original image data, the second processor 30 directly sends the target image data of the original image data to the receiving device through the data sharing module 50. During which time the target image data is not passed through the memory 60, which may improve the efficiency of rendering the target image data on the receiving device 70.

In this application, the first processor 10 may also send the target image data sent by the second processor 30 to the target application through a corresponding data output channel, so as to output the target image data in the window of the target application. The data output channel passes through or does not pass through the memory of the electronic equipment.

In one implementation, the south bridge chip 40 is connected to the data sharing module 50, and the first processor may directly send the target image data to the data sharing module 50 through a data channel between the DMI interface and the data sharing module 50, so as to send the target image data to the target application through the data sharing module 50, and output the target image data in the window of the target application. Here, the data channel does not pass through the memory.

In another implementation, the south bridge chip 40 is connected to the data sharing module 50, and the first processor sends the target image data to the memory through the communication bus 005, receives the target image data sent by the memory through the communication bus 005, and sends the target image data to the data sharing module 50 through the DMI interface, so as to send the target image data to the target application through the data sharing module 50, and output the target image data in the window of the target application. The data channel at this time passes through the memory.

In another implementation, the first processor 10 is connected to the data sharing module 50, and when the first processor 10 receives the target image data sent by the second processor 30, the target image data is directly sent to the target application through the data sharing module 50, so as to output the target image data in the window of the target application. The data channel at this time passes through the memory.

According to the method and the device, the target image data are sent to the receiving terminal device or the target application without a memory, so that the interface display speed of the target image data on the receiving terminal device or the target application can be increased.

Fig. 6 is a schematic structural composition diagram six of the electronic device in the present application, as shown in fig. 6, including: the image processing system comprises a CPU, a memory, a south bridge chip (PCH), an independent display card, a thunder interface and a camera, wherein the USB camera is plugged in the USB interface on the PCH, and when the camera acquires original image data, the independent display card can directly acquire the original image data through a data channel (i) without the memory. After the independent display card processes the original image data into the target image data, the target image data can be sent to the memory through the data channel II, and after the target image data is temporarily stored in the memory, the target image data is sent to the network card through the data channel III.

Specifically, the camera may send original image data to the PCH through a USB Interface (e.g., USB 2.0/3.0), the PCH receives the original image data, and then the PCH sends the original image data to the CPU through a Direct Media Interface (DMI), the CPU receives the original image data, and then the CPU sends the original image data to the independent graphics card through a communication bus (16Lans), and after the independent graphics card receives the original image data, the independent graphics card processes the original image data through a Graphics Processing Unit (GPU) to obtain target image data, and then stores the target image data in the graphics memory. Or after obtaining the target image data, the independent video card sends the target image data to the CPU through a communication bus (16Lans), a memory controller in the CPU sends the target image data to a memory through a Quick Path Interconnect (QPI), after the target image data is stored in the memory, the memory sends the target image data to the CPU through the QPI bus, after receiving the target image data, the CPU sends the target image data to the PCH through a DMI interface, and the PCH outputs the target image data to the network card through a physical interface transceiver (PYH) or a PCIe interface.

Here, the independent video card can directly acquire the original image data without the memory, so that delay caused by repeated transfer, calling or forwarding between the memory and the independent video card is reduced, and the processing efficiency of the original image data is improved.

Fig. 7 is a schematic diagram illustrating a structural configuration of an electronic device in the present application, where as shown in fig. 7, a USB camera is plugged into a lightning interface (hunderbolt) of a PCIe controller, an independent graphics card can directly acquire original image data without an internal memory through a data path (i), after the original image data is processed into target image data by the independent graphics card, the target image data can be sent to the internal memory through a data channel (ii), and after the target image data is temporarily stored in the internal memory, the target image data is sent to a network card through a data channel (iii).

Specifically, the camera is plugged in a lightning interface of a PCIe controller, wherein the lightning interface is connected with the PCIe controller on the CPU through a communication bus (4Lans), the PCIe controller acquires original image data acquired by the camera through the lightning interface and then sends the original image data to the independent display card through the communication bus (16Lans), and after receiving the original image data, the independent display card processes the original image data through the GPU to obtain target image data, and then stores the target image data in the video memory. Or, after obtaining the target image data, the independent video card sends the target image data to the CPU through the communication bus (16Lans), the memory controller in the CPU sends the target image data to the memory through the QPI bus, after the target image data is stored in the memory, the memory sends the target image data to the memory controller in the CPU through the QPI bus, after the memory controller receives the target image data, the CPU sends the target image data to the PCH through the DMI interface, and the PCH outputs the target image data to the network card through the physical interface transceiver (PYH) or the PCIe interface.

Fig. 8 is an eighth schematic structural component diagram of the electronic device in the present application, as shown in fig. 8, a USB interface is added to an independent graphics card, a camera is plugged into the USB interface of the independent graphics card, when the camera acquires original image data, the independent graphics card can directly acquire the original image data through a data path (i), after the original image data is processed into target image data by the independent graphics card, the target image data can be provided to a memory through a data channel (ii), and after the target image data is temporarily stored in the memory, the target image data is provided to a network card through the data channel (iii).

Specifically, the camera is plugged into a USB interface of the independent graphics card, the independent graphics card can directly acquire the original image data acquired by the camera through the USB interface, the original image data is processed by the GPU to obtain target image data, and then the target image data is stored in the video memory. Or, after obtaining the target image data, the independent video card sends the target image data to the CPU through the communication bus (16Lans), the memory controller in the CPU sends the target image data to the memory through the QPI bus, after the target image data is stored in the memory, the memory sends the target image data to the memory controller in the CPU through the QPI bus, after the memory controller receives the target image data, the CPU sends the target image data to the PCH through the DMI interface, and the PCH outputs the target image data to the network card through the physical interface transceiver (PYH) or the PCIe interface.

Of course, the target image data may also be sent to the lightning interface on the PCIe controller through the CPU, so as to transmit the target image data to the network card through the lightning interface.

Fig. 9 is a schematic diagram illustrating a structural composition of an electronic device in this application, where as shown in fig. 9, a USB interface is added to an independent graphics card, a camera is plugged into the USB interface of the independent graphics card, when the camera acquires original image data, the independent graphics card can directly acquire the original image data through a data path (i), and after the original image data is processed into target image data by the independent graphics card, the target image data can be provided to a PCIe controller on a memory through a data channel (ii), so that the target image data is provided to a network card through PYH on the PCIe controller or the PCIe interface. Here, the communication bus between the network card and the PCIe controller may be 1/2/4 Lans.

Because the independent display card can directly acquire the original image data without the memory and can transmit the target image data to the network card without the memory, the delay caused by repeated unloading, calling or forwarding between the memory and the independent display card is reduced, and the processing efficiency of the original image data and the transmission speed of the target image data are improved.

FIG. 10 is a schematic view of a flowchart implementation of the control method in the present application, as shown in FIG. 10, including

1001, if a video stream acquisition instruction is obtained, controlling a camera module of the electronic equipment to acquire original image data;

in the application, the method can be applied to electronic equipment with image acquisition capability, and the electronic equipment can be terminals such as computers, notebooks, mobile phones and the like. The electronic equipment can be provided with a plurality of applications, and the control instruction sent by the target application is received by the electronic equipment, so that the camera module on the electronic equipment can be controlled to collect original image data based on the control instruction. Here, the target application may be a video conference application and a non-video conference video application.

Step 1002, the original image data is sent to at least one second processor of the electronic device through a corresponding data input channel to be processed, so as to obtain target image data;

in one implementation, if the camera module is connected to a south bridge chip of the electronic device, and the south bridge chip is connected to the first processor (e.g., CPU), when the camera module acquires original image data, the electronic device may acquire the original image data acquired by the camera module through the south bridge chip of the electronic device, and send the original image data to the second processor for processing through a data input channel formed between the second processor and the south bridge chip to obtain target image data. The data input channel does not pass through the memory of the electronic equipment. This can improve the processing speed of the image data.

Here, the second processor may be specifically a graphics card including a stand-alone graphics card and an integrated graphics card. The second processor and the first processor may be connected through a data interface or a communication bus.

In another implementation, if the camera module is connected to a first processor (CPU) on the electronic device through a data interface (e.g., a lightning interface or a USB interface), when the camera module acquires original image data, the electronic device may acquire the original image data through the data interface, and send the original image data to a second processor for processing through a data input channel formed between the data interface and the second processor, so as to obtain target image data. The data input channel does not pass through the memory of the electronic equipment. This can improve the processing speed of the image data.

In another implementation scheme, if the camera module is connected to the second processor through a data interface (such as a lightning interface or a USB interface), when the camera module acquires original image data, the second processor may directly acquire the original image data through the data interface and process the original image data to obtain target image data, during which the target image data does not pass through a memory of the electronic device. This can improve the processing speed of the image data.

In the application, the camera module may include at least two cameras, the electronic device may further obtain access information of each camera at a self-checking stage of a Basic Input Output System (BIOS) in a starting process of the electronic device, and determine a data input channel formed between each camera and the second processor based on the access information of each camera at least, so that original image data acquired by different cameras can be provided to the corresponding second processor through the corresponding data input channels.

For example, the electronic device determines, according to the access information of each camera, that the first camera is connected to the south bridge chip of the electronic device through the USB interface, and the second camera is connected to the lightning interface of the electronic device, because the data transmission speed of the lightning interface is higher than that of the USB interface, when the electronic device acquires the original image data through the USB interface on the south bridge chip, the electronic device may send the original image data to the second processor through a data channel formed between the south bridge chip and the memory. When the electronic equipment acquires the original image data through the lightning interface, the electronic equipment can send the original image data to the second processor through a data channel formed between the lightning interface and the second processor, and the original image data does not pass through a memory of the electronic equipment in the period, so that the processing speed of the image data is greatly improved.

In this application, the second processors may include at least two, and the two second processors may both be independent display cards, may also both be integrated display cards, and may also be one of the independent display cards and one of the integrated display cards.

Specifically, when the electronic device includes at least two second processors, the electronic device may also provide the original image data to the corresponding second processors in different data input channels according to the current network parameters. And if the current network parameters represent that the current network is stable, the original image data is sent to the second processor A through the data channel of the memory, and if the current network parameters represent that the current network is unstable, the original image data is sent to the second processor B through the data channel without the memory. Reference may be made in particular to the above description of the electronic device.

In this application, if the electronic device has a plurality of cameras, the electronic device may further obtain attribute information of each camera at a self-checking stage of the BIOS, and send target original image data acquired by the target camera to the corresponding at least one second processor through the corresponding target data input channel for processing based on at least the attribute information, where the target camera is a camera that determines that image parameters of the acquired original image data meet a first condition based on at least the attribute information.

Here, the attribute information of the camera includes, but is not limited to, a resolution parameter, a pixel parameter, a frame rate parameter, and the like of the camera.

Determining that image parameters of original image data acquired by the target camera meet a first condition if the resolution parameter of the target camera meets a preset parameter based on the attribute information of the target camera;

or determining that the image parameters of the original image data acquired by the target camera meet a first condition if the pixel parameters of the target camera meet preset parameters based on the attribute information of the target camera;

or determining that the image parameters of the original image data collected by the target camera meet a first condition if the frame rate parameters of the target camera meet preset parameters based on the attribute information of the target camera.

Here, the target camera may be one or a plurality of cameras.

When the target camera is one, the electronic device may send the raw image data collected by the target camera to the second processor through a data channel that does not pass through the memory.

When a plurality of target cameras are provided, the electronic device may determine the second processor corresponding to each camera according to the correspondence between the identifier of each camera stored in the preset database and each second processor, so as to provide the original image data acquired by each camera to the corresponding second processor through the corresponding data channel.

For example, the first camera and the second camera are both target cameras satisfying the first condition, the first camera corresponds to the second processor a, and the second camera corresponds to the second processor B, so that the electronic device may send the original image data to the second processor a through a data channel formed between the second processor a and the first camera, and send the original image data to the second processor B through a data channel formed between the second processor B and the second camera. In the period, the two data channels do not pass through the CPU and the memory or pass through the CPU but do not pass through the memory.

In this application, the electronic device may further obtain usage information of the electronic device, and determine a target data input channel based on at least the usage information, so as to send the raw image data collected by the corresponding camera to the corresponding at least one second processor for processing.

Specifically, the electronic device may obtain power information of the electronic device, and if it is determined that the power information is smaller than the preset power, the corresponding original image data acquired by the camera is sent to the corresponding second processor through a data input channel that does not pass through the internal memory for processing. Therefore, the time for data to be stored, called or forwarded back and forth between the memory and the second processor is shortened, and the power consumption of the electronic equipment can be reduced.

Or the electronic device can also obtain load information of the electronic device, and if the load of the CPU exceeds the preset parameter, the original image data acquired by the camera is sent to the corresponding second processor through a data input channel that does not pass through the memory for processing. Therefore, the data between the memory and the CPU is reduced from being transferred back and forth, called or forwarded, and the load of the CPU can be reduced.

In the application, if the original image data is collected by the first camera in the camera module, the original image data is sent to the first image processor of the electronic device through the first data input channel which does not pass through the first processor of the electronic device for processing, and the first image processor is one of the at least one second processor.

Here, the first data input channel may be characterized as a data channel formed between the second processor and the first camera. For example, the second processor is a video card, the first camera is connected to the second processor through a USB interface, and the electronic device may send the original image data to the second processor through the USB interface between the second processor and the first camera for processing. The period does not go through the memory and the CPU.

Or if the original image data is acquired through a second camera in the camera module, the original image data is sent to at least one second processor of the electronic equipment through a second data input channel of a first processor of the electronic equipment for processing.

Here, the second data input channel may be characterized as a data channel formed between the second processor and the second camera via the first processor. For example, the second camera is connected to the first processor through the south bridge chip, the second processor is connected to the first processor through the communication bus, and the electronic device sends the raw image data to the second processor for processing through the data channel formed between the communication bus between the second processor and the first processor and the south bridge chip, without passing through the memory. For another example, the second camera is connected to the first processor through the lightning interface, the second processor is connected to the first processor through the communication bus, and the electronic device sends the original image data to the second processor for processing through the data channel formed between the communication bus between the second processor and the first processor and the lightning interface, without passing through the memory.

Or if the original image data is collected through a first camera and a second camera in the camera module, the original image data is sent to a first image processor and a second image processor through a corresponding first data input channel and a corresponding second data input channel for processing, and the second image processor is one of the at least one second processor different from the first image processor.

Here, the image processing speeds of the first image processor and the second image processor may be the same or different, for example, the first image processor is a separate graphic card and the second image processor is an integrated graphic card. Or the first image processor and the second image processor are both independent display cards or are both integrated display cards.

In one example, the electronic device may determine, according to a connection object of the first camera and the second camera, that the first camera is connected to the first image processor, and determine that the second camera is connected to the south bridge chip, then send the original image data of the first camera to the first image processor for processing through a first data input channel that does not pass through the memory, and send the original image data of the second camera to the second image processor for processing through a second data input channel that passes through the memory.

In this application, the electronic device may further send the target image data to a target application through a corresponding data output channel, so as to display the target image data on a window of the target application, where the target application runs on the electronic device and/or a receiving end device that establishes a target communication connection with the electronic device.

Here, when the receiving end device is an external device such as an external display and a projector, the receiving end device may be connected to the electronic device in a wired (e.g., cable or USB interface) or wireless (e.g., WIFI, hotspot, bluetooth) manner.

In one implementation, if the target application runs on the electronic device itself, the electronic device may send the target application through a data output channel formed by a communication interface between the CPU in the electronic device and the target application when receiving the target image data sent by the second processor. The data output channel does not pass through the memory of the electronic equipment, so that the data transmission speed can be improved.

In another implementation, if the target application is running on the electronic device itself, when receiving the target image data sent by the second processor, the electronic device may further send the target image data to the memory, and then send the target image data to the CPU through the memory, so as to send the target image data to the target application through a data output channel formed between the memory and the communication interface of the target application. Where the data output channel passes through memory.

In this application, if the target application is running on the receiving end device connected to the electronic device, and if the receiving end device is connected to the south bridge chip of the electronic device through the data sharing module, when the electronic device receives the target image data sent by the second processor, the electronic device may also send the target image data to the data sharing module through the communication bus between the second processor and the CPU or the data output channel formed between the communication interface and the south bridge chip, so as to send the target image data to the receiving end device through the data sharing module. Wherein the data output channel does not pass through the memory.

Here, the data sharing module includes, but is not limited to, a DP interface, a network card interface, a USB network card interface, an HDMI interface, and a VGA interface.

In another implementation, if the target application is running on a receiving end device connected to the electronic device, and if the receiving end device is connected to the lightning interface of the electronic device through the data sharing module, when the electronic device receives the target image data sent by the second processor, the electronic device may also send the target image data to the data sharing module through a communication bus between the second processor and the CPU or a data output channel formed between the communication interface and the lightning interface, so as to send the target image data to the receiving end device through the data sharing module. Here the data output channel does not go through memory.

In another implementation, if the receiving end device is connected to the second processor of the electronic device through the data sharing module, the second processor may also directly send the target image data to the data sharing module when obtaining the target image data, so as to send the target image data to the receiving end device through the data sharing module. During which the data output channel does not pass through memory. The process of memory is saved, so the speed of image output can be greatly improved,

it should be noted that: the control method provided by the above embodiment and the electronic device embodiment provided by the above embodiment belong to the same concept, and specific implementation processes thereof are described in the electronic device embodiment and are not described herein again.

In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described device embodiments are merely illustrative, for example, the division of the unit is only a logical functional division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or other forms.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

The methods disclosed in the several method embodiments provided in the present application may be combined arbitrarily without conflict to obtain new method embodiments.

Features disclosed in several of the product embodiments provided in the present application may be combined in any combination to yield new product embodiments without conflict.

The features disclosed in the several method or apparatus embodiments provided in the present application may be combined arbitrarily, without conflict, to arrive at new method embodiments or apparatus embodiments.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An electronic device, comprising:

the camera module is used for acquiring original image data;

2. The electronic device of claim 1, further comprising a south bridge chip in signal communication with the first processor, wherein the camera module comprises a first camera, wherein the first camera is capable of capturing the raw image data, wherein the first camera is connected to the south bridge chip via a first interface, wherein the second processor is in signal communication with the first processor via a communication bus or a second interface, and wherein the second processor is capable of obtaining the raw image data from the south bridge chip to the first processor via the communication bus or the second interface.

3. The electronic device of claim 1, wherein the camera module includes a second camera, the second camera is capable of acquiring the original image data, the second camera is connected to the first processor through a third interface, the second processor is in signal connection with the first processor through a communication bus or a second interface, and the second processor is capable of obtaining the original image data through a data channel formed by the communication bus and the third interface, or the second processor is capable of obtaining the original image data through a data channel formed by the second interface and the third interface.

4. The electronic device of claim 1, wherein the camera module comprises a third camera, the third camera is connected to the second processor through a fourth interface, and the second processor obtains original image data collected by the third camera through the fourth interface.

5. The electronic device of any one of claims 1 to 4, wherein the first processor sends the target image data to a data sharing module through a corresponding data output channel, to send the target image data to a corresponding receiving end device through the data sharing module, or,

the first processor sends the target image data to a target application through a corresponding data output channel so as to output the target image data in a window of the target application;

and the data output channel passes through or does not pass through the memory.

6. A control method, comprising:

7. The method of claim 6, further comprising:

8. The method of claim 6, wherein,

and if the camera module comprises at least two cameras, acquiring access information of the cameras, and determining a data input channel formed between each camera and the second processor at least based on the access information so that the original image data collected by different cameras can be sent to the corresponding second processor through the corresponding data input channels.

9. The method of claim 8, wherein processing the raw image data to at least a second processor of an electronic device via a corresponding data input channel comprises:

10. The method of claim 6 or 8, wherein the processing the raw image data to at least a second processor of an electronic device via a corresponding data input channel comprises: