WO2022143059A1

WO2022143059A1 - Terminal and terminal application method

Info

Publication number: WO2022143059A1
Application number: PCT/CN2021/136437
Authority: WO
Inventors: 刘达林
Original assignee: 中兴通讯股份有限公司
Priority date: 2020-12-31
Filing date: 2021-12-08
Publication date: 2022-07-07
Also published as: CN114697600A

Abstract

The embodiments of the present disclosure provide a terminal and a terminal application method. The terminal comprises a multimedia collector, a multimedia encoding processing chip, a switch chip and a main control chip; the multimedia encoding processing chip and the main control chip establish a network communication channel by means of the switch chip; and the multimedia encoding processing chip encodes multimedia data collected by the multimedia collector to obtain encoded multimedia data, and then transmits same to the main control chip by means of the network communication channel, so that the main control chip achieves a multimedia service.

Description

A terminal and terminal application method

CROSS-REFERENCE TO RELATED APPLICATIONS

The present disclosure claims the priority of Chinese patent application CN202011621594.X, filed on December 31, 2020, entitled "A Terminal and Terminal Application Method", the entire contents of which are incorporated into the present disclosure by reference.

technical field

The embodiments of the present disclosure relate to, but are not limited to, the field of electronic devices, and specifically relate to, but are not limited to, a terminal and a terminal application method.

Background technique

In the relevant multimedia conference system, the multimedia data acquisition terminal (such as camera, microphone) is usually connected to the conference terminal through a USB cable or an HDMI (High Definition Multimedia Interface, high-definition multimedia interface) cable, and the audio and video collected by the multimedia data acquisition terminal The data is transmitted to the conference terminal through the USB interface or HDMI, so that the conference terminal and the external terminal can realize the video conference service. However, limited by the transmission capability of the USB cable and the HDMI cable, the transmission rate and transmission stability of the data transmission from the multimedia data collection end to the conference terminal are still unsatisfactory. Therefore, there is an urgent need to provide a multimedia conference product.

SUMMARY OF THE INVENTION

The terminal and the terminal application method provided by the embodiments of the present disclosure mainly solve the technical problem: how to improve the speed and stability of data transmission between the multimedia data collection terminal and the conference terminal.

In order to solve the above technical problems, an embodiment of the present disclosure provides a terminal, which includes a multimedia collector, a multimedia encoding processing chip, a switch chip, and a main control chip; the multimedia encoding processing chip and the main control chip establish a network communication channel through the switch chip, so as to realize Communication connection; multimedia collector, used to collect multimedia data, and output the collected multimedia data to the multimedia encoding processing chip; multimedia encoding processing chip, used to encode and process the multimedia data, obtain the encoded multimedia data, and encode the multimedia data. It is transmitted to the main control chip through the network communication channel; the main control chip is used to realize multimedia services with external devices according to the received encoded multimedia data.

An embodiment of the present disclosure also provides a terminal application method, which is applied to the above-mentioned terminal. The terminal application method includes: a main control chip establishes a network communication channel with a multimedia encoding processing chip through a switch chip; the main control chip receives the multimedia encoding processing through the network communication channel The encoded multimedia data transmitted by the chip is obtained by encoding and processing the multimedia data collected by the multimedia collector by the multimedia encoding processing chip; the main control chip realizes the multimedia business according to the encoded multimedia data and external equipment.

Other features of the present disclosure and corresponding benefits are set forth in later portions of the specification, and it is to be understood that at least some of the benefits will become apparent from the recitation of the present disclosure.

Description of drawings

FIG. 1 is a schematic diagram of a hardware structure of a terminal provided in Embodiment 1 of the present disclosure;

Fig. 2 is a kind of schematic diagram of the internal and external network communication of the terminal in Fig. 1;

3 is a schematic diagram of another hardware structure of the terminal provided in Embodiment 1 of the present disclosure;

Fig. 4 is a kind of schematic diagram of the internal and external network communication of the terminal in Fig. 3;

FIG. 5 is a flowchart of the terminal application method provided in Embodiment 2 of the present disclosure;

6 is a schematic diagram of a hardware structure of the terminal provided in Embodiment 3 of the present disclosure;

FIG. 7 is a flowchart of the terminal application method provided in Embodiment 3 of the present disclosure.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present disclosure more clear, the embodiments of the present disclosure will be further described in detail below through specific implementations in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present disclosure, but not to limit the present disclosure.

Example 1

In common video conferencing solutions in related technologies, the camera and microphone are usually independent from the conference terminal. The camera and microphone are connected to the conference terminal through a USB cable, and the image data and audio data collected by the two are transmitted through the USB cable. In some examples, the USB cable can be replaced by an HDMI cable. However, whether it is a communication connection based on a USB cable or a communication connection based on an HDMI cable, there are comparisons in terms of multimedia data transmission rate and transmission stability. It is a big problem, which can easily affect the conference effect of the conference terminal. In this regard, this embodiment provides a terminal, please refer to a schematic diagram of a hardware structure of the terminal shown in FIG. 1 .

The terminal 10 includes a multimedia collector 11 , a multimedia encoding processing chip 12 , a switch chip 13 and a main control chip 14 .

The multimedia collector 11 is used to collect multimedia data. In some examples of this embodiment, the multimedia data collected by the multimedia collector 11 may include image data, so the multimedia collector 11 may have an image collection function. In one example, the multimedia collector 11 includes a camera sensor. In other examples of this embodiment, the multimedia collector 11 may have an audio collection function, and therefore, in these examples, the multimedia collector 11 includes a microphone. In some other examples, the multimedia collector 11 includes a camera sensor and a microphone at the same time.

The multimedia encoding processing chip 12 is used for encoding and processing the multimedia data collected by the multimedia collector 11 to obtain encoded multimedia data. If the multimedia collector 11 includes a camera sensor, the multimedia encoding processing chip 12 can encode the image data. For example, when the multimedia collector 11 is a camera sensor, the multimedia encoding processing chip 12 can be an ISP (image signal processing) If the multimedia collector 11 includes a microphone, the multimedia encoding processing chip 12 can encode and process the audio data, which can be an audio signal processing chip. In some examples of this embodiment, a multimedia server application may also run on the multimedia encoding processing chip 12 , and the multimedia server application may receive an instruction of a multimedia client application on the main control chip 14 .

The switch chip 13 can provide a network communication path for the multimedia encoding processing chip 12 and the main control chip 14. The multimedia encoding processing chip 12 and the main control chip 14 establish a network communication channel based on the switch chip 13. After the network communication channel is established, the multimedia encoding processing is completed. The chip 12 can transmit the encoded multimedia data obtained by encoding to the main control chip 14 through the network communication channel. In some examples of this embodiment, the main control chip 14 may also send an instruction to the multimedia encoding processing chip 12 through the network communication channel to control the multimedia encoding processing chip 12 .

The main control chip 14 is actually the CPU of the terminal 10, which can run various business software, thereby realizing the processing of various transactions. For example, in this embodiment, a multimedia client application can run on the main control chip 14, and the client application can cooperate with the multimedia server application running on the multimedia encoding processing chip 12 to realize the multimedia encoding by the main control chip 14. Control of the processing chip 12 . In addition, the main control chip 14 can communicate with external devices, and after acquiring encoded multimedia data through a network communication channel with the multimedia encoding processing chip 12, the main control chip 14 can implement multimedia services with the external devices. It can be understood that the main control chip 14 can also run service software other than multimedia client applications, such as video conference software, video telephony software, video editing software (including but not limited to video editing, mosaic processing) and the like.

In some examples of this embodiment, the network communication channel established by the multimedia encoding processing chip 12 and the main control chip 14 through the switch chip 13 is the first network communication channel. The main control chip 14 communicates with the external device through a second network communication channel, and the first network communication channel and the second network communication channel are isolated from each other. Based on the isolation between the second network communication channel and the first network communication channel, external devices that interact with the main control chip 14 through the second network communication channel cannot access the multimedia encoding processing chip 12 .

In some examples of this embodiment, the second network communication channel for communication between the main control chip 14 and the external device may also be established based on the switch chip 13. In this case, in order to avoid the main control chip 14 and the multimedia encoding processing chip 12 and external devices are in the same local area network, causing the multimedia encoding processing chip 12 to be freely accessed by external devices, the switch chip 13 can isolate the first network communication channel and the second network communication channel through different VLANs (Virtual Local Area Networks). , for example, in an example, as shown in FIG. 2 , after the terminal 10 is powered on, the main control chip 14 can virtualize two virtual network cards, namely the first virtual network card and the second virtual network card, the main control chip 14. Bind the first virtual network card to VLAN1, and bind the second virtual network card to VLAN0. The first virtual network card establishes a first network communication channel with the multimedia encoding processing chip 12 based on the first communication address, and the second virtual network card establishes a second network communication channel with an external device based on the second communication address. In one example, the first communication address and the second communication address may be IP addresses. In some examples, the first communication address and the second communication address are both statically configured IP addresses, and in other examples, the first communication address and the second communication address are both dynamically configured IP addresses through DHCP (Dynamic Host Configuration Protocol). ; In some examples, one of the two communication addresses is statically configured, and the other is dynamically configured. For example, in an example, the first communication address is a statically configured IP address, and the second communication address is a dynamically configured IP address. It can be seen that the first network communication channel established based on the first virtual network card is in VLAN1, and the second network communication channel established based on the second virtual network card is in VLAN0. In FIG. 2 , the first network communication channel is represented by a thick solid line, and the second network communication channel is represented by a thick dashed line. Through the isolation of different virtual local area networks, even if the external communication device is connected to the same switch chip as the multimedia encoding processing chip 12 , it cannot access the multimedia encoding processing chip 12 .

In an example, the communication address of the multimedia encoding processing chip 12 may be set based on a static configuration, or may be acquired based on a dynamic configuration. For example, in some examples of this embodiment, the communication address of the multimedia encoding processing chip 12 is a static IP address. In other examples, after the multimedia encoding processing chip 12 is started, it can dynamically apply for an IP address from the DHCP service on the main control chip 14 through the switch chip 13 .

In other examples of this embodiment, the terminal may use different ways to isolate the first network communication channel and the second network communication channel, as shown in FIG. 3 , in these examples, the terminal 30 includes the multimedia collector 11 in addition to the In addition to the multimedia encoding processing chip 12, the switch chip 13 and the main control chip 14, it also includes a network switch chip 15. The function of the network switch chip 15 and the switch chip 13 is similar, and both are used to realize the communication connection between the two communication terminals. . In this embodiment, the reason for the difference between "switch chip" and "network switch chip" is mainly to distinguish two chips with network switch functions. In fact, in some examples, the network switch chip The structure and model of the switch chip 15 and the switch chip 13 can be exactly the same. Of course, it is also feasible that the network switch chip 15 and the switch chip 13 are not identical. For example, the switch chip 13 and the network switch chip 15 are designed and produced by different manufacturers and have the same characteristics. A chip with network switching functions.

In the terminal 30, on the one hand, the main control chip 14 can establish a first network communication channel with the multimedia encoding processing chip 12 through the switch chip 13, and on the other hand, the main control chip 14 can establish a second network with external devices through the network switch chip 15 Communication channels, eg, a first network communication channel is shown in bold solid lines and a second network communication channel is shown in bold dashed lines in FIG. 4 . Since the switch chip 13 and the network switch chip 15 are physically isolated, the first network communication channel and the second network communication channel are isolated from each other, which prevents the external device from directly interacting with the multimedia encoding processing chip 12 to obtain the encoded multimedia data. Possibly, the security of the encoded multimedia data is improved.

It is understandable that when the conference terminal communicates with the camera and the microphone through the USB interface, it is difficult for the conference terminal to find the failure of the multimedia data collection end such as the camera. Moreover, even if the user finds that the multimedia data collection end fails, it can only be restarted. Only the conference terminal and the camera can realize fault recovery, which greatly affects the user's multimedia conference experience. In this embodiment, the main control chip 14 and the multimedia encoding processing chip 12 can realize fault detection based on the heartbeat mechanism: the main control chip 14 One end of the two ends of the multimedia encoding processing chip 12 can periodically send a heartbeat message to the other end, and the other end also returns a heartbeat message to the opposite end after receiving the heartbeat message.

In some examples of this embodiment, the main control chip 14 and the multimedia encoding processing chip 12 can not only implement fault detection based on the heartbeat mechanism, but also can automatically perform fault processing according to the detection results after fault detection:

In some examples, the main control chip 14 can detect the reachability of the multimedia encoding processing chip 12 based on the heartbeat mechanism. It can be understood that the reachability of the multimedia encoding processing chip 12 actually refers to whether the message sent by the main control chip 14 is It can reach the multimedia encoding processing chip 12, or whether the message sent by the multimedia encoding processing chip 12 can reach the main control chip 14. Therefore, the detection of the reachability of the multimedia encoding processing chip 12 is actually aimed at the connection between the main control chip 14 and the main control chip 12. In the detection of the availability of the network communication channel between the multimedia encoding processing chips 12, after determining that the network communication channel is unavailable based on the heartbeat mechanism, the main control chip 14 can control the switch chip 13 to initialize. In some examples of this embodiment, after determining that the network communication channel is unavailable, the main control chip may also restart the multimedia client application running by itself.

In some examples, the main control chip 14 can detect the availability of the multimedia encoding processing chip 12 based on the heartbeat mechanism, and when it is determined that the multimedia encoding processing chip 12 is abnormal, send a fault indication instruction to the multimedia encoding processing chip 12 to indicate If the multimedia encoding processing chip 12 is faulty, it is convenient for the multimedia encoding processing chip 12 to perform corresponding fault processing after knowing the fault.

In some other examples, the multimedia encoding processing chip 12 can control to restart the multimedia server application running by itself after determining that the heartbeat is abnormally timed out based on the heartbeat detection result. On the other hand, the multimedia encoding processing chip 12 can also control the multimedia collector 11 to initialize .

It can be understood that, because the main control chip 14 communicates with the multimedia encoding processing chip 12 based on the network, and TCP (Transmission Control Protocol, Transmission Control Protocol) provides a heartbeat mechanism, therefore, in some examples of this embodiment, the main The control chip 14 and the multimedia encoding processing chip 12 can implement fault detection directly based on the TCP heartbeat mechanism. Of course, in some other examples of this embodiment, the main control chip 14 and the multimedia encoding processing chip 12 may also implement fault detection based on the heartbeat mechanism under other protocols.

The terminal provided by this embodiment can not only improve the data transmission performance between the multimedia data collection terminal and the main control chip through network technology, but also because the multimedia collector, multimedia encoding processing chip and switch chip are integrated inside the terminal, and at the same time based on VLAN Technology or two different chips with network switching function prevent the multimedia encoding processing chip from being directly accessed by external devices, and improve the security of multimedia data. On the other hand, the main control chip and the multimedia encoding processing chip can realize fault detection based on the heartbeat mechanism, and automatically realize fault processing, which is beneficial to the reliability of the terminal.

Embodiment 2

This embodiment provides a terminal application method, and the terminal application method is mainly applied to the aforementioned terminal. Please refer to a flowchart of the terminal application method shown in FIG. 5 below.

S502: The main control chip establishes a network communication channel through the switch chip and the multimedia encoding processing chip.

In this embodiment, the main control chip can establish a network communication channel with the multimedia encoding processing chip through the switch chip. It can be understood that if there is only one switch chip in the terminal, the communication between the main control chip and the external device also needs to be based on this Switch chip implementation. If there are other chips with network switching function in the terminal, the main control chip can implement communication with external devices based on other chips with network switching function. Of course, those skilled in the art can understand that even if there are two or more chips with network switching functions in the terminal, for example, the terminal has both switch chips and network switching chips, the main control chip can also choose to use only One chip realizes the communication with the multimedia encoding processing chip and with the external equipment.

In some examples of this embodiment, the main control chip implements communication with the multimedia encoding processing chip and with external devices through the same switch chip: the main control chip establishes a first network communication channel with the multimedia encoding processing chip based on the switch chip, and A second network communication channel is established with the external device based on the switch chip. In order to prevent external devices from directly accessing the multimedia encoding processing chip to obtain encoded multimedia data, the main control chip can control the switch chip to isolate the second network communication channel from the first network communication channel through VLAN technology, and the main control chip establishes the first network communication channel in a VLAN. Network communication channel, establish a second network communication channel in another VLAN. In this way, the external device and the multimedia encoding processing chip are in different virtual local area networks. Therefore, the external device cannot access the multimedia encoding processing chip. The multimedia encoding processing chip encodes Whether the obtained encoded multimedia data can be sent to an external device, and which external device is sent to can be controlled by the main control chip itself, which improves the security of the encoded multimedia data.

In other examples of this embodiment, the main control chip implements communication with the multimedia encoding processing chip and the external device through two different chips with a network switching function: the terminal includes a switch chip and a network switch chip, two with network The chip for switching functions, in an example, the switch chip and the network switching chip may be the same chip or different chips. The terminal can establish a first network communication channel with the multimedia encoding processing chip through the switch chip, and establish a second network communication channel with the external device through the network switching chip. Because the switch chip and the network switch chip are two different physical entities, the first network communication channel and the second network communication channel are located under two different local area networks respectively, which realizes mutual isolation between the two.

S504: The main control chip receives the encoded multimedia data transmitted by the multimedia encoding processing chip through the network communication channel.

After the main control chip establishes a network communication channel (eg, the first network communication channel) with the multimedia encoding processing chip, it can receive encoded multimedia data sent by the multimedia encoding processing chip. In this embodiment, the multimedia encoding processing chip may perform encoding processing on multimedia data collected by a multimedia collector, such as a camera sensor, so as to obtain encoded multimedia data. In an example, a multimedia client application can run on the main control chip, and a multimedia server application matching the multimedia client application can run on the multimedia encoding processing chip. In cooperation with each other, the main control chip can control the multimedia encoding processing chip, for example, indicating that the multimedia encoding processing chip is faulty, etc. At the same time, the main control chip can also collect encoded multimedia data from the multimedia encoding processing chip through the multimedia client application.

S506: The main control chip realizes the multimedia service according to the encoded multimedia data and the external device.

After the main control chip acquires the encoded multimedia data from the multimedia encoding processing chip, it can implement multimedia services with external devices based on the encoded multimedia data. In some examples of this embodiment, after the main control chip determines that the encoded multimedia data is to be sent to the external device, after receiving the encoded multimedia data through the first network communication channel, the encoded multimedia data can be processed by the service software running by itself. , and then sent to the external device through the second network communication channel isolated from the first network communication channel, thereby ensuring the security of the data on the side of the multimedia encoding processing chip.

In some examples of this embodiment, the main control chip and the multimedia encoding processing chip may also implement fault detection based on a heartbeat mechanism. Even in some examples, the main control chip and the multimedia encoding processing chip can not only implement fault detection based on the heartbeat mechanism, but also automatically perform fault processing according to the detection results after fault detection:

For example, the main control chip can detect the reachability of the multimedia encoding processing chip based on the heartbeat mechanism. After determining that the network communication channel is unavailable based on the heartbeat mechanism, the main control chip can control the initialization of the switch chip. In some examples, the main control chip can also On the basis of the initialization of the switch chip, the multimedia client application running by itself is further restarted.

For another example, the main control chip can detect the availability of the multimedia encoding processing chip based on the heartbeat mechanism, and when it is determined that the multimedia encoding processing chip is abnormal, send a fault indication instruction to the multimedia encoding processing chip to instruct the multimedia encoding processing chip that it occurs. It is convenient for the multimedia encoding processing chip to perform corresponding fault processing after knowing the fault.

In some other examples, the multimedia encoding processing chip can control to restart the multimedia server application running by itself after determining that the heartbeat is abnormally timed out based on the heartbeat detection result, and the multimedia encoding processing chip can also control the initialization of the multimedia collector.

The terminal application method provided by this embodiment improves the data transmission rate between the multimedia encoding processing chip and the main control chip, enhances the stability of data transmission, and is beneficial to improving the service effect of the terminal multimedia service. Moreover, in the terminal application method, the main control chip can detect the reachability and availability of the multimedia encoding processing chip based on the heartbeat mechanism, find faults in time, and eliminate the faults, thereby improving the reliability of the terminal multimedia service.

Embodiment 3

In order to make the terminal and the terminal application method provided in the foregoing embodiment more clear to those skilled in the art, this embodiment will further describe the foregoing solution with reference to an example.

This embodiment provides a terminal, which can be applied to various remote audio and video call scenarios, such as business conferences, remote classes, and the like. In this embodiment, the multimedia data in the terminal includes image data, therefore, the multimedia collector may include a camera sensor, and correspondingly, the multimedia encoding processing chip may be an ISP chip. Please refer to FIG. 6 , the terminal 60 includes a camera sensor 61 , an ISP chip 62 , a switch chip 63 and a main control chip 64 , wherein the camera sensor 61 is communicatively connected to the ISP chip 62 , and the ISP chip 62 receives the image data transmitted by the camera sensor 61 . , and perform encoding processing to obtain encoded image data. The communication between the main control chip 64 and the ISP chip 62 and the communication between the main control chip 64 and the external devices are all realized through the switch chip 63 . In this embodiment, the main control chip 64 and the ISP chip 62 establish a first network communication channel with each other in VLAN1 based on the switch chip 63 , on the other hand, the main control chip 64 is based on the switch chip 63 between itself and the external device. A second network communication channel in VLAN0 is established. It should be understood that the numbers of the two VLANs "VLAN1" and "VLAN0" do not limit the implementation of the solution in this embodiment, and the two numbers are only used for exemplary description here. In some other examples of this embodiment , VLANs can have other numbers.

In this embodiment, the multimedia server application running on the ISP chip 62 is a camera server application, and the main control chip 64 runs a camera client application corresponding to the camera server application.

In this embodiment, the main control chip 64 and the ISP chip 62 implement fault detection based on the heartbeat mechanism, and the main control chip 64 and the ISP chip 62 can also automatically process the fault according to the detection result after the fault detection: the main control chip 64 can The reachability of the ISP chip 62 is detected based on the heartbeat mechanism. After it is determined based on the heartbeat mechanism that the first network communication channel is unavailable, the main control chip 64 can control the switch chip 63 to initialize, or the main control chip can further restart its own operation. webcam client application. In addition, the main control chip 64 can detect whether the ISP chip 62 is faulty based on the heartbeat mechanism. If the detection result indicates that the ISP chip 62 is abnormal, the main control chip 64 can send a fault indication instruction to the ISP chip 62 to instruct the ISP chip 62 that it has occurred. It is convenient for the ISP chip 62 to perform corresponding fault processing after knowing the fault. In some other examples, the ISP chip 62 can control to restart the camera server application running by itself after determining that the heartbeat is abnormally timed out based on the heartbeat detection result, and at the same time, the ISP chip 62 can also control the initialization of the camera sensor 61 .

The application method of the terminal in FIG. 6 will be described below with reference to the flowchart shown in FIG. 7 .

S702: The terminal is powered on and turned on.

S704: The ISP chip initializes its network IP address as a static IP address within the range of the network segment corresponding to VLAN1.

S706: The ISP chip initializes the camera sensor and starts the camera server application.

After the ISP chip starts the camera server application, the camera client can access the encoded image data collected by the camera sensor and encoded by the ISP chip.

S708: The main control chip initializes the switch chip, and uses the VLAN technology to divide the network interface of the switch chip.

The main control chip can divide the network interface used by the switch chip to establish the first network communication channel to VLAN1, and the network interface used by the switch chip to establish the second network communication channel to VLAN0, that is, the switch chip is connected to the network interface of the ISP chip. And the network interface of the switch chip connected to the main control chip is divided into VLAN1, and the network interface of the switch chip connected to the main control chip and the network interface of the main control chip connected to the external network are divided into VLAN0.

S710: The main control chip virtualizes two virtual network cards and performs IP configuration on the virtual network cards.

The main control chip can bind one of the virtual network cards to VLAN0, and configure its IP acquisition method as DHCP dynamic acquisition method, so that it can dynamically obtain IP from the external network through the switch chip; the main control chip binds the other virtual network card to VLAN1 , and configure its IP address as a static IP address in the network segment corresponding to VLAN1. It can be understood that the IP acquisition methods of the two virtual network cards are not fixed, and both can be selected from static configuration methods and dynamic configuration methods.

S712: The main control chip initializes the service software and starts the camera client application.

S714: The main control chip and the ISP chip establish a first network communication channel, and the camera client controls the camera sensor through the first network communication channel.

The main control chip establishes a first network communication channel with the ISP chip through the IP address in VLAN1, and then the camera client on the main control chip controls the camera sensor by sending instructions to the camera client on the ISP chip.

S716: The main control chip and the ISP chip start heartbeat detection, and perform troubleshooting after a fault is detected.

When the camera sensor is found to be abnormal, the camera client on the main control chip will send a fault indication instruction to the camera server on the ISP chip. If the network is unreachable, the main control chip reinitializes the switch chip, and can also restart the camera client application.

After the camera server on the ISP chip establishes a connection with the camera client on the main control chip, the heartbeat mechanism is turned on, and the instruction request of the camera client is monitored, and the work corresponding to the instruction is completed according to the instruction of the camera client. When the heartbeat expires abnormally, the ISP chip reinitializes the camera sensor and restarts the camera server application.

S718: The camera client on the main control chip accesses the camera server on the ISP chip, obtains encoded image data, and transmits the encoded image data to the service software for processing.

It should be understood that the sequence obtained by each step in FIG. 7 is not unique. For example, in some other examples, S708-S712 may be performed before S704-S706, or the two processes may be performed in parallel. For another example, S718 and S716 may be performed simultaneously, or S718 may be performed first and then S716 may be performed.

The present embodiments also provide a computer-readable storage medium embodied in any method or technology for storing information, such as computer-readable instructions, data structures, computer program modules, or other data Volatile or nonvolatile, removable or non-removable media. Computer-readable storage media include, but are not limited to, RAM (Random Access Memory, random access memory), ROM (Read-Only Memory, read-only memory), EEPROM (Electrically Erasable Programmable read only memory, electrically erasable programmable read only memory) ), flash memory or other memory technology, CD-ROM (Compact Disc Read-Only Memory), digital versatile disk (DVD) or other optical disk storage, magnetic cartridges, magnetic tape, magnetic disk storage or other magnetic storage devices, Or any other medium that can be used to store the desired information and that can be accessed by a computer.

One or more computer programs that can be read, compiled and executed by one or more processors may be stored in the computer-readable storage medium, and in this embodiment, the computer-readable storage medium may be stored with terminal application programs , the terminal application program can be used by one or more processors to execute the process of implementing any one of the terminal application methods described in the foregoing embodiments.

In this embodiment, the camera and the terminal are integrated together, and a switch chip is arranged between the camera and the main control of the terminal, so that the ISP chip on the camera side and the main control chip use network communication, which improves the speed and speed of data transmission between the two. Transmission stability. In addition, through the VLAN technology of the switching chip, the camera side is isolated from the external network to ensure the data security of the camera side. Moreover, an effective heartbeat mechanism is established between the main control chip and the ISP chip to implement fault detection through TCP, which can ensure long-term stable operation of terminal services.

In the terminal and the terminal application method provided by the embodiments of the present disclosure, the terminal includes a multimedia collector, a multimedia encoding processing chip, a switch chip, and a main control chip, wherein the multimedia encoding processing chip and the main control chip establish a network communication channel through the switch chip to realize communication. connect. The multimedia collector is used to collect multimedia data and output the collected multimedia data to the multimedia encoding processing chip. The multimedia encoding processing chip encodes and processes the multimedia data to obtain encoded multimedia data. The multimedia encoding processing chip is also used to pass the encoded multimedia data through the network. The communication channel is transmitted to the main control chip, so that the main control chip can realize multimedia services with external devices according to the received encoded multimedia data. In the embodiment of the present disclosure, because the multimedia encoding processing chip and the main control chip are connected through a network based on a switch chip, compared with the solution of connecting through a USB cable or an HDMI cable in the related art, the multimedia encoding processing chip is connected to the above-mentioned The network communication channel established based on the switch chip between the main control chips has a higher transmission rate and a more stable transmission performance, which is beneficial to improve the service effect of the terminal multimedia service. Moreover, because in the embodiments of the present disclosure, a multimedia collector for realizing multimedia data collection, a multimedia encoding processing chip for realizing multimedia data encoding, a switch chip, and a main control chip are integrated into one terminal device, which can reduce the cost of multimedia conference products. .

It can be seen that those skilled in the art should understand that all or some of the steps in the methods disclosed above, the functional modules/units in the system, and the device can be implemented as software (which can be implemented by computer program codes executable by a computing device). ), firmware, hardware, and their appropriate combination. In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be composed of several physical components Components execute cooperatively. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit .

In addition, communication media typically embodies computer readable instructions, data structures, computer program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and can include any information delivery, as is well known to those of ordinary skill in the art medium. Therefore, the present disclosure is not limited to any particular combination of hardware and software.

The above content is a further detailed description of the embodiments of the present disclosure in conjunction with specific implementations, and it cannot be considered that the specific implementation of the present disclosure is limited to these descriptions. For those of ordinary skill in the technical field to which the present disclosure pertains, without departing from the concept of the present disclosure, some simple deductions or substitutions can also be made, all of which should be regarded as belonging to the protection scope of the present disclosure.

Claims

A terminal includes a multimedia collector, a multimedia encoding processing chip, a switch chip and a main control chip;

The multimedia encoding processing chip and the main control chip establish a network communication channel through the switch chip to realize communication connection; the multimedia collector is used for collecting multimedia data and outputting the collected multimedia data to the multimedia encoding processing chip;

The multimedia encoding processing chip is used for encoding and processing the multimedia data to obtain encoded multimedia data, and transmitting the encoded multimedia data to the main control chip through the network communication channel;

The main control chip is used for implementing multimedia services with an external device according to the received encoded multimedia data.
The terminal according to claim 1, wherein the multimedia encoding processing chip and the main control chip establish a first network communication channel through the switch chip, and the main control chip communicates with the external device through a second network A channel implements communication, and the first network communication channel and the second network communication channel are isolated from each other.
The terminal according to claim 2, wherein the main control chip and the external device establish the second network communication channel through the switch chip, the first network communication channel and the second network communication channel They are in different virtual local area networks.
The terminal according to claim 3, wherein the main control chip is configured to configure a first virtual network card and a second virtual network card, and the first virtual network card is configured to establish the first virtual network card based on a statically configured first communication address A network communication channel, where the second virtual network card is configured to establish the second network communication channel based on the dynamically allocated second communication address.
The terminal according to claim 2, wherein the terminal further comprises a network switch chip, and the main control chip and the external device establish the second network communication channel through the network switch chip.
The terminal according to any one of claims 1-5, wherein the multimedia data includes image data, the multimedia collector includes a camera sensor, and the multimedia encoding processing chip includes an image signal processing ISP chip.
The terminal according to any one of claims 1-5, wherein the main control chip and the multimedia encoding processing chip implement fault detection based on a heartbeat mechanism, and perform fault processing according to the detection result.
The terminal according to claim 7, wherein a multimedia client application runs on the main control chip, a multimedia server application runs on the multimedia encoding processing chip, and the multimedia client application is connected to the multimedia server. The application cooperates with each other to realize the interaction between the main control chip and the multimedia encoding processing chip; the main control chip and the multimedia encoding processing chip realize fault detection based on the heartbeat mechanism, and perform fault processing according to the detection result. The method includes at least one of the following:

The main control chip detects the availability of the network communication channel based on the heartbeat mechanism, and controls the switch chip to initialize when the network communication channel is unavailable;

The main control chip detects the availability of the multimedia encoding processing chip based on the heartbeat mechanism, and sends a fault indication instruction to the multimedia encoding processing chip when the multimedia encoding processing chip is abnormal;

The multimedia encoding processing chip restarts the multimedia server application and controls the initialization of the multimedia collector when the heartbeat is abnormally timed out.
The terminal according to any one of claims 1-5, wherein the communication address of the multimedia encoding processing chip is obtained by static configuration.
A terminal application method, applied to the terminal according to any one of claims 1-9, the terminal application method comprising:

The main control chip establishes a network communication channel with the multimedia encoding processing chip through the switch chip;

The main control chip receives the encoded multimedia data transmitted by the multimedia encoding and processing chip through the network communication channel, and the encoded multimedia data is encoded and processed by the multimedia encoding and processing chip on the multimedia data collected by the multimedia collector. get;

The main control chip implements a multimedia service with the external device according to the encoded multimedia data.