CN111181835B

CN111181835B - Message monitoring method, system and server

Info

Publication number: CN111181835B
Application number: CN201910989247.3A
Authority: CN
Inventors: 潘林锋; 方昭潭
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2019-10-17
Filing date: 2019-10-17
Publication date: 2021-07-27
Anticipated expiration: 2039-10-17
Also published as: CN111181835A

Abstract

The invention provides a message monitoring method, a system and a server, wherein after an enterprise instant messaging message is received, the message type of the enterprise instant messaging message is identified, if the message type indicates that the enterprise instant messaging message corresponds to a picture, text content and object content in the picture are identified, words and phrases associated with the picture and confidence degrees of the words and phrases are obtained, at least one target word and phrase are selected from the words and phrases based on the confidence degrees of the words and phrases, if the at least one target word and sensitive words in a preset sensitive word bank are matched, the invisible content corresponding to the sensitive words in the enterprise instant messaging message is controlled, so that the sensitive content (such as the content corresponding to the sensitive words) can be invisible when the sensitive content exists, the leakage of the sensitive content is reduced, the enterprise safety is improved, and the monitoring of the enterprise instant messaging message can be realized. And after receiving the instant messaging message of the enterprise, the system can immediately monitor whether the sensitive words exist, so as to realize real-time monitoring.

Description

Message monitoring method, system and server

Technical Field

The invention belongs to the technical field of enterprise instant messaging, and particularly relates to a message monitoring method, a message monitoring system and a server.

Background

Currently, Enterprise employees can communicate online through an EIM (Enterprise Instant Messaging), for example, an EIM application can be installed in electronic devices such as a mobile phone and a computer, and online communication is performed by enabling the EIM application and inputting contents in an interface of the EIM application.

When online communication is performed through the EIM application, various contents can be communicated, and some contents are prohibited from being communicated, so that it is necessary to provide a message monitoring method and apparatus for monitoring contents.

Disclosure of Invention

In view of the above, the present invention provides a method, a system and a server for monitoring an instant messaging message of an enterprise. The technical scheme is as follows:

the invention provides a message monitoring method, which comprises the following steps:

receiving an enterprise instant messaging message, and identifying the message type of the enterprise instant messaging message;

if the message type indicates that the enterprise instant messaging message corresponds to a picture, identifying text content and object content in the picture to obtain each word associated with the picture and confidence of each word;

selecting at least one target word from the respective words based on the confidence of the respective words;

and if the at least one target word is matched with a sensitive word in a preset sensitive word bank, controlling the content corresponding to the sensitive word in the picture to be invisible.

Preferably, the identifying the text content and the object content in the picture to obtain each word associated with the picture and a confidence of each word includes:

identifying text contents in the pictures to obtain words included in the text contents;

identifying the object content in the picture to obtain the text description content of the scene displayed by the object content;

identifying the text description content to obtain words included in the text description content;

calculating a confidence level for the word.

Preferably, the controlling that the content corresponding to the sensitive word in the picture is invisible includes:

carrying out layer covering treatment on the content corresponding to the sensitive words in the picture, and/or carrying out special effect treatment on the content corresponding to the sensitive words in the picture, wherein the content corresponding to the sensitive words is invisible through the layer covering treatment and the special effect treatment;

or

And prohibiting the sending of the enterprise instant messaging message to the receiver of the enterprise instant messaging message.

Preferably, if the at least one target word is matched with a sensitive word in a preset sensitive word bank, the method further includes:

adjusting the message sending authority of the sender of the enterprise instant messaging message;

and/or

Storing the enterprise instant messaging message in a sensitive message list, wherein the sensitive message list is used for updating a preset sensitive word bank which is compared with the target word;

and/or

Sending prompt information, wherein the prompt information is used for indicating the current enterprise instant messaging message of sending the presence-sensitive words;

and/or

And storing the enterprise instant messaging message in a sensitive message list, and outputting the sensitive message list to a server for monitoring the enterprise instant messaging message.

Preferably, the method further comprises: acquiring a sender identifier of the enterprise instant messaging message; if the sender identification indicates that the sender belongs to a monitored object, executing a step of identifying the message type of the enterprise instant messaging message;

and/or

And the sensitive words in the preset sensitive word library correspond to sender marks, and the sender marks are used for indicating senders for sending the enterprise instant messaging messages.

Preferably, if the message type indicates that the instant messaging message of the enterprise corresponds to a picture, identifying the text content and the object content in the picture includes:

and if the message type indicates that the enterprise instant communication message corresponds to a picture, acquiring the picture based on index data in the enterprise instant communication message, and identifying text content and object content in the picture.

Preferably, if the at least one target word is matched with a sensitive word in a preset sensitive word bank, the controlling that the content corresponding to the sensitive word in the picture is invisible includes:

and if at least two target words in the target words are matched with sensitive words in a preset sensitive word bank, controlling the contents corresponding to the sensitive words in the picture to be invisible, wherein the at least two target words are different.

The invention also provides a message monitoring system, which comprises:

the message management module is used for receiving the enterprise instant messaging message and identifying the message type of the enterprise instant messaging message;

the image message monitoring module is used for identifying text contents and object contents in an image if the message type indicates that the enterprise instant messaging message corresponds to the image, and obtaining words related to the image and confidence degrees of the words;

the sensitive word comparison module is used for selecting at least one target word from the words based on the confidence of each word and comparing the at least one target word with the sensitive words in the preset sensitive word bank;

and the monitoring result processing module is used for controlling the content corresponding to the sensitive word in the picture to be invisible if the at least one target word is matched with the sensitive word in a preset sensitive word bank.

Preferably, if the at least one target word is matched with a sensitive word in a preset sensitive word bank, the monitoring result processing module is further configured to:

and/or

The present invention also provides a server, comprising: a communication interface and a processor;

the communication interface is used for receiving the enterprise instant communication message;

the processor is used for identifying the message type of the enterprise instant messaging message; if the message type indicates that the enterprise instant messaging message corresponds to a picture, identifying text content and object content in the picture to obtain each word associated with the picture and confidence of each word; selecting at least one target word from the respective words based on the confidence of the respective words; and if the at least one target word is matched with a sensitive word in a preset sensitive word bank, controlling the content corresponding to the sensitive word in the picture to be invisible.

By the technical scheme, after the instant messaging message of the enterprise is received, the message type of the instant messaging message of the enterprise is identified, if the message type indicates that the instant messaging message of the enterprise corresponds to a picture, recognizing text contents and object contents in the picture to obtain words related to the picture and confidence degrees of the words, selecting at least one target word from the words based on the confidence degrees of the words, controlling contents corresponding to sensitive words in the enterprise instant messaging message to be invisible if the at least one target word is matched with the sensitive words in a preset sensitive word bank, therefore, when sensitive content (such as content corresponding to sensitive words) exists in the enterprise instant messaging message, the sensitive content can be invisible, leakage of the sensitive content is reduced, enterprise safety is improved, and monitoring on the enterprise instant messaging message related to the picture can be achieved. And after receiving the instant messaging message of the enterprise, the system can immediately monitor whether the sensitive word exists, thereby realizing the real-time monitoring of the instant messaging message of the enterprise.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.

FIG. 1 is a system architecture diagram of an online communication system provided by an embodiment of the present invention;

fig. 2 is a flowchart of a message monitoring method according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of the online communication system of FIG. 1 employing a message monitoring method;

FIG. 4 is a diagram illustrating a processing comparison of EIM messages according to an embodiment of the present invention;

fig. 5 is a flowchart of another message monitoring method according to an embodiment of the present invention;

fig. 6 is a schematic structural diagram of a message monitoring system according to an embodiment of the present invention;

fig. 7 is a schematic structural diagram of another message monitoring system according to an embodiment of the present invention;

fig. 8 is a schematic structural diagram of a server according to an embodiment of the present invention.

Detailed Description

At present, a system architecture diagram of an online communication system corresponding to an online communication system performed through an EIM is shown in fig. 1, and the system architecture diagram includes a first terminal 10, a server 20, and a second terminal 30, where the first terminal 10 and the second terminal 30 are terminals used by both parties performing online communication through the EIM, the first terminal 10 and the second terminal 30 may be any device capable of installing an EIM application, such as a computer, a mobile phone, and a wearable device, the EIM application provides an input interface for one means of online communication or at least for online communication, the first terminal 10 and the second terminal 30 can respectively display the input interface of the EIM application, obtain contents input by both parties of online communication through the input interface, and send the input contents as an EIM message (enterprise instant messaging message).

The server 20 serves as a message relay party between the first terminal 10 and the second terminal 30, so that messages can be transmitted between the first terminal 10 and the second terminal 30 through the server 20, and online communication can be achieved. But there is a problem with the current online communication: the server 20 is only used as a message relay party, and no matter whether the content corresponding to the EIM message (such as the content carried in the EIM message and/or the content that can be found through the EIM message) has sensitive content, such as salary, personal privacy, and the eight diagrams, which are not allowed by enterprise culture and enterprise regulation (considered as sensitive content), the server 20 forwards the EIM message, based on which the embodiment provides a message monitoring method for monitoring the EIM message, and in the process of monitoring the EIM message, if the sensitive content exists, the sensitive content is specially processed to reduce the leakage of the sensitive content, the message monitoring method can be applied to any one of the server, the sender of the EIM message and the receiver of the EIM message are the first terminal and the second terminal, the specific which is the sender and which is the receiver depends on which terminal the EIM message is sent from, and this embodiment will not be described.

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Referring to fig. 2, a flowchart of a message monitoring method according to an embodiment of the present invention is shown, where the method includes the following steps:

201: an EIM message is received, identifying a message type of the EIM message. Wherein the message types of the EIM message comprise: the EIM message is a picture related type, for example, if the EIM message is a picture message (such as a screenshot, a static picture and a dynamic picture), the EIM message is a video message, and a picture can be downloaded through the EIM message, the message type of the EIM message is a picture related type; if a picture cannot be obtained through the EIM message, for example, the EIM message is a text message (e.g., plain text, adding emoticons to plain text), the EIM message is a voice message (the voice message does not point to any picture), and the message type of the EIM message is a text-related type.

The identification opportunity of the message type of the EIM message can be started immediately after the EIM message is received so as to achieve the purpose of monitoring the EIM message in real time, and compared with delayed monitoring, the EIM message can be monitored in the interaction process of the EIM message so as to prevent sensitive content from being leaked in the interaction process of the EIM message, wherein the delayed monitoring is to process the EIM message after the EIM message is sent to a receiver, and although the processing mode can also monitor, the sensitive content is leaked, so that the real-time monitoring can improve enterprise safety.

202: and if the message type indicates that the EIM message corresponds to a picture, identifying text contents and object contents in the picture to obtain each word associated with the picture and the confidence coefficient of each word. The text content represents characters in the picture, the object content indicates contents in the picture except the text content, for example, the object content is a person, an animal, a building, and the like in the picture, privacy, confusion, and the like may be involved in the object content, and therefore, for the object content, the embodiment also needs to identify the object content, so as to obtain words (i.e., words associated with the picture) corresponding to the two contents through identification of the text content and the object content in the picture, and increase the monitoring range.

In this embodiment, the process of obtaining each word associated with the picture and the confidence of each word is as follows:

step 1, recognizing text contents in the picture to obtain words included in the text contents, such as words included in the text contents obtained by but not limited to ocr (optical character recognition) character recognition technology.

Step 2, identifying the object content in the picture to obtain the character description content of the scene displayed by the object content; and identifying the text description content to obtain words included in the text description content. One mode describes each object contained in the object content in a text mode to obtain text description content; the other method is to identify the relationship among all things besides the objects to obtain the scene shown by the object content, and describe the scene in a text mode to obtain the text description content. For example, a picture is a perusal to a leader character a, and the text description contents obtained by the two methods are: the leader character A and the pernicious leader character A are obtained, and then the obtained words are the leader character A, the leader character A and the pernicious, and after the character description content is obtained, the words can be obtained through but not limited to an OCR character recognition technology.

In this embodiment, the content of the object in the picture can be identified by, but not limited to, Tencent best picture identification technology, and this embodiment will not be described in detail.

And 3, calculating the confidence coefficient of the words. In the embodiment, the word and the picture can be simultaneously input into a machine model, the confidence of the word is calculated through the machine model, and the machine model is obtained by training the existing picture and the word marked from the existing picture.

Of course, the embodiment can also train another machine model in advance, the machine model has two functions of content recognition and confidence calculation, and the machine model can recognize and obtain each word and can also obtain the confidence of each word.

The points to be explained here are: if the EIM message is a picture message or a video message, the picture can be directly extracted from the picture message or the video message for identification, and for the video message, the first frame picture in the video message or the N frames of pictures in the video message can be identified, where N is greater than 1 but which frames of the N frames of pictures are specifically not limited in this embodiment. If a picture can be downloaded through the EIM message, for example, the EIM message includes index data indicating the picture, the picture is obtained through the index data, and then text content and object content in the picture are identified, for example, a form of the index data is a Uniform Resource Locator (URL), and the picture is downloaded through the URL.

203: and selecting at least one target word from each word based on the confidence of each word, and screening out the target word which is compared with the sensitive words in the preset sensitive word bank according to the confidence of each word.

One way to select a target word is to: sequencing the words according to the mode that the confidence of each word is reduced from high to low, selecting the first M words as target words, wherein M is a natural number which is more than or equal to 1, and the specific value can be determined according to the actual requirement, so that the confidence of the selected target words is greater than the confidence of the words which are not selected; the other mode is as follows: and setting a threshold value for selecting the target word, and if the confidence coefficient of the word is greater than the threshold value, taking the word as the target word. In addition to these two ways, the present embodiment can select the target word in other ways, and this embodiment is not described one by one.

204: and if at least one target word is matched with the sensitive words in the preset sensitive word bank, controlling the contents corresponding to the sensitive words in the picture to be invisible.

One point is pointed out here: if the EIM message is a picture message or a video message, controlling that the content corresponding to the sensitive word in the picture is invisible means that the content corresponding to the sensitive word in the picture message or the video message is invisible, and if a picture can be downloaded through the EIM message, controlling that the content corresponding to the sensitive word in the downloaded picture is invisible means controlling that the content corresponding to the sensitive word in the downloaded picture is invisible.

The target word is matched with the sensitive words in the preset sensitive word bank to represent that: the target words are the same as or similar to the sensitive words, and the target words are matched with the sensitive words in the preset sensitive word bank one by one in the implementation process so as to determine whether the target words are matched with the sensitive words. For the sensitive words in the preset sensitive word bank, the embodiment mainly monitors the EIM message, and the sensitive words in the preset sensitive word bank are words related to enterprise culture, enterprise regulations, and the like, for example, the sensitive words may be words related to enterprise names, words related to competitive enterprises, personal problems of employees of enterprises, and the like, and for example, the sensitive words in the preset sensitive word bank include: wages, compensation, job level, promotion, jumping slot, hunting head, and many spellings.

In this embodiment, if at least one target word is matched with a sensitive word in a preset sensitive word bank, the content corresponding to the sensitive word in the control picture is invisible. Or if at least two target words in the target words are matched with the sensitive words in the preset sensitive word bank, controlling the contents corresponding to the sensitive words in the picture to be invisible, wherein the at least two target words are different, namely if the at least two different target words are matched with the sensitive words, controlling the contents corresponding to the sensitive words in the picture to be invisible.

If one word in the text content is found to be matched with the sensitive word through comparison, the word is controlled to be invisible, and if one object in the object content is found to be matched with the sensitive word through comparison, the object is controlled to be invisible, wherein the mode of controlling the content corresponding to the sensitive word in the picture to be invisible includes but is not limited to the following modes:

one way is as follows: carrying out layer covering processing on the content corresponding to the sensitive words in the picture, wherein the layer covering processing enables the content corresponding to the sensitive words to be invisible;

in another mode: performing special effect processing on the content corresponding to the sensitive word in the picture, where the special effect processing makes the content corresponding to the sensitive word invisible, for example, but not limited to, adding a mosaic to the content corresponding to the sensitive word in the picture to cover the content corresponding to the sensitive word, or covering the content corresponding to the sensitive word in the picture with other content irrelevant to the sensitive word, or performing fuzzification processing on the content corresponding to the sensitive word in the picture, where the layer covering processing and the special effect processing may be performed at the same time, and this embodiment is not limited;

for the two ways, the processed picture may be sent to the receiver of the EIM message after the content corresponding to the sensitive word in the control picture is invisible. Of course, in addition to invisible content corresponding to the sensitive word, the embodiment may perform special effect processing on the entire content of the picture, such as performing blurring processing on the entire content, where the blurred picture enables a user to know where the content is located in the picture but the content is unknown, or may perform layer covering processing and/or special effect processing on the content connected to the content in addition to invisible content corresponding to the sensitive word, so as to prevent that the content corresponding to the sensitive word can be pushed out through the content connected to the content corresponding to the sensitive word, for example, if "my month wage is carried in the EIM message," wage is content corresponding to the sensitive word ", then invisible processing may be performed on" wage is. Preventing the push out of my month's payroll is.

In another way: and prohibiting the EIM message from being sent to a recipient of the EIM message to make the EIM message invisible to the recipient by prohibiting the sending such that content in the picture corresponding to the sensitive word is invisible to the recipient.

Referring to the online communication system shown in fig. 1, a process of implementing the message monitoring method by the online communication system is described below with reference to the online communication system shown in fig. 1, and as shown in fig. 3, a preset sensitive word bank is added in advance in the server 20, where the preset sensitive word bank includes the following sensitive words:

salary, job level, promotion, jumping slot, hunting head and more pieces;

the first terminal 10 sends an EIM message, and the server 20 recognizes that the EIM message corresponds to a picture after receiving the EIM message sent by the first terminal 10, such as a picture shown in fig. 4, where the picture is a salary of an employee in an enterprise;

the server 20 identifies the text content and the object content in the picture to obtain each word and the confidence of each word in the following table 1; the last two recognition results in table 1 indicate the confidence that the content carried in the EIM message is a text and the confidence that the content is a picture;

TABLE 1 identification results of EIM messages

Recognition result	Confidence level
		Basic	0.999968588
Attendance at work	0.972433031
		Reward promotion for working age	0.888525367
Payroll	0.997067988
		Number of days	0.997722924
7500	0.999279201
		28	0.999806523
13842	0.999568284
		500	0.999121606
Text	0.73
		Screenshot	0.13

The server 20 selects a target word with a confidence level greater than a threshold value from the words shown in table 1 according to a preset threshold value and the confidence level of each word, if the preset threshold value can be but is not limited to 0.85, at least target words such as wages, reward and the like are selected based on the threshold value, the selected target words are respectively compared with the sensitive words in the preset sensitive word bank, the two target words such as the wages and the reward are found to be matched with the sensitive words, the two target words "payroll" and "bonus" in the screenshot shown in figure 4 and the values under the two target words are not visible, as the arrows in fig. 4 point to the pictures, it is known from the two pictures shown in fig. 4 that the sensitive content is not visible after processing, i.e. the content corresponding to the sensitive word is not visible, and then a picture, in which the sensitive content is not visible, is sent by the server 20 to the second terminal 30.

The message monitoring method provided in this embodiment can control the content corresponding to the sensitive word in the picture to be invisible, and can execute at least one of the following processing modes when at least one target word is matched with the sensitive word in the preset sensitive word bank:

a processing mode is as follows: and adjusting the message sending authority of the sender of the EIM message, wherein the message sending authority is used for indicating whether the sender of the EIM message can send the EIM message or not, if the message sending authority can be sent, the message sending authority is also used for indicating the message type of the EIM message sent by the sender, and the like. In this embodiment, if at least one target word is matched with a sensitive word in a preset sensitive word bank, adjusting the message sending permission may be to reduce the message sending permission, so that the sender can only send a limited EIM message, for example, only send an EIM message of a text-related type, and in addition, adjusting the message sending permission may also prohibit the sender from sending the EIM message, for example, after monitoring that the same sender sends an xth EIM message having a content corresponding to the sensitive word, the sender is prohibited from sending the EIM message within a preset time period from the time of monitoring the EIM message, where a value of X and a value of the preset time period may be determined according to actual requirements.

The other treatment method comprises the following steps: and storing the EIM message in a sensitive message list, wherein the sensitive message list is used for updating a preset sensitive word bank which is compared with the target word. The sensitive message list may be a data table, and each monitored EIM message having a content corresponding to a sensitive word is stored in the data table, or the sensitive message list may be a queue, and each monitored EIM message having a content corresponding to a sensitive word may be stored in a first position in the queue, and other EIM messages are sequentially moved down, and the EIM message located in a last position in the queue is removed from the queue.

The other processing mode is as follows: sending a prompt message, where the prompt message is used to indicate that an EIM message of the presence-sensitive word is currently sent, and the prompt message may be at least one of text, picture, and sound, and for example, the prompt message is sent in a form of a short message or an EIM message, and the form of the prompt message is not limited in this embodiment.

The other processing mode is as follows: storing the EIM message in a sensitive message list, and outputting the sensitive message list to a server for monitoring the EIM message, such as the server 20 shown in fig. 1, where the server 20 is capable of implementing the message monitoring method provided in this embodiment.

Referring to fig. 5, a flowchart of another message monitoring method according to an embodiment of the present invention is shown, where the method includes the following steps:

501: and receiving the EIM message, and acquiring the sender identification of the EIM message. In this embodiment, the sender identifier may be, but is not limited to, a user account for logging in the EIM, and the sender identifier may be carried in the EIM message.

502: and judging whether the sender indicated by the sender identification belongs to the monitored object, if so, executing the step 503, and if not, executing the step 507. In this embodiment, a monitored object table may be preset, and sender identifiers of senders (i.e., monitored objects) to be monitored are recorded in the monitored object table, so that after an to-be-sent EIM message is obtained, whether the to-be-sent EIM message belongs to a monitored object can be determined by the sender identifier corresponding to the to-be-sent EIM message and the sender identifier in the monitored object table.

In this embodiment, the monitored object may be a sender that sends Y EIM messages having contents corresponding to the sensitive words within a certain time, so that the monitored object can be automatically determined and automatically updated in this manner. If the sender indicated by the sender identifier belongs to the monitored object, the EIM message sent by the sender is monitored in real time through the steps 503 to 506, and if the sender indicated by the sender identifier does not belong to the monitored object, the step 507 may be executed to directly send the EIM message to the receiver.

The above is only described with reference to the monitored object table, and other manners may also be adopted when the message monitoring method provided by this embodiment is applied, which is not described in this embodiment one by one.

503: the message type of the EIM message is identified.

504: and if the message type indicates that the EIM message corresponds to a picture, identifying text contents and object contents in the picture to obtain each word associated with the picture and the confidence coefficient of each word.

505: at least one target word is selected from the respective words based on the confidence of the respective words.

506: and if at least one target word is matched with the sensitive words in the preset sensitive word bank, controlling the contents corresponding to the sensitive words in the picture to be invisible.

In this embodiment, steps 503 to 506: the same as the above steps 201 to 204, this embodiment will not be described again.

507: the EIM message is sent to a recipient of the EIM message.

By means of the technical scheme, after the EIM message is received, the sender identification of the EIM message is obtained, if the sender identification indicates that the sender belongs to the monitored object, the EIM message is monitored in real time, and real-time monitoring on the EIM message sent by a part of senders is achieved through the sender identification.

For the message monitoring method, the sensitive words in the preset sensitive word library can correspond to the sender identification, and the sensitive words can be configured for the sender in a manner of corresponding the sensitive words to the sender identification, so that after any EIM message is received, if the message type of the EIM message indicates that the EIM message corresponds to a picture, the target words are compared with the sensitive words corresponding to the sender identification of the EIM message after the target words are selected, and the personalized setting and monitoring personalization of the sensitive words are realized.

While, for purposes of simplicity of explanation, the foregoing method embodiments have been described as a series of acts or combination of acts, it will be appreciated by those skilled in the art that the present invention is not limited by the illustrated ordering of acts, as some steps may occur in other orders or concurrently with other steps in accordance with the invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required by the invention.

Corresponding to the foregoing method embodiment, an embodiment of the present invention further provides a message monitoring system, whose structure is shown in fig. 6, and may include: the system comprises a message management module 10, a picture message monitoring module 20, a sensitive word comparison module 30 and a monitoring result processing module 40.

The message management module 10 is configured to receive the EIM message, identify a message type of the EIM message, and please refer to the method embodiment for describing the message type of the EIM message, which is not described in this embodiment.

And the picture message monitoring module 20 is configured to identify text contents and object contents in the picture if the message type indicates that the EIM message corresponds to a picture, so as to obtain words associated with the picture and confidence levels of the words. The text content represents characters in the picture, the object content indicates contents in the picture except the text content, for example, the object content is a person, an animal, a building, and the like in the picture, privacy, confusion, and the like may be involved in the object content, and therefore, for the object content, the embodiment also needs to identify the object content, so as to obtain words (i.e., words associated with the picture) corresponding to the two contents through identification of the text content and the object content in the picture, and increase the monitoring range.

In this embodiment, one way for the picture message monitoring module 20 to obtain each word associated with the picture and the confidence level of each word is as follows: identifying text contents in the picture to obtain words included in the text contents; identifying the object content in the picture to obtain the character description content of the scene displayed by the object content; identifying the text description content to obtain words included in the text description content; the confidence of the words is calculated. For a detailed description of this manner, reference is made to method embodiments, which are not described again.

The points to be explained here are: if the EIM message is a picture message or a video message, the picture message monitoring module 20 may directly extract a picture from the picture message or the video message for identification, and for the video message, the picture message monitoring module 20 may identify a first frame of picture in the video message or N frames of pictures in the video message, where N is greater than 1 but which frames of the N frames of pictures are specifically not limited in this embodiment. If a picture can be downloaded through the EIM message, for example, the EIM message includes index data indicating the picture, the picture message monitoring module 20 first obtains the picture through the index data, and then identifies text content and object content in the picture, for example, a form of the index data is a URL through which the picture is downloaded.

The sensitive word comparison module 30 is configured to select at least one target word from the words based on the confidence of each word, and compare the at least one target word with a sensitive word in a preset sensitive word bank.

And the monitoring result processing module 40 is configured to control the content corresponding to the sensitive word in the picture to be invisible if at least one target word is matched with the sensitive word in the preset sensitive word bank.

in another mode: the method includes the steps that special effect processing is conducted on contents corresponding to sensitive words in the picture, the contents corresponding to the sensitive words are made invisible through the special effect processing, in the embodiment, layer covering processing and the special effect processing can be conducted at the same time, and the embodiment is not limited;

for the above two ways, the monitoring result processing module 40 may send the processed picture to the receiver of the EIM message after the content corresponding to the sensitive word in the control picture is invisible.

In this embodiment, the monitoring result processing module 40, besides that the control content is invisible, executes at least one of the following processing modes when at least one target word is matched with a sensitive word in a preset sensitive word bank:

a processing mode is as follows: adjusting the message sending authority of the sender of the EIM message; the other treatment method comprises the following steps: storing the EIM message in a sensitive message list, wherein the sensitive message list is used for updating a preset sensitive word bank which is compared with a target word; the other processing mode is as follows: sending prompt information, wherein the prompt information is used for indicating that the EIM information of the sensitive words is sent currently; the other processing mode is as follows: and storing the EIM message in a sensitive message list, and outputting the sensitive message list to a server for monitoring the EIM message.

For the description of the processing manner, refer to the above method embodiment, which is not described again.

On the basis of the message monitoring system shown in fig. 6, the message monitoring system provided in this embodiment may further include: the sensitive word stock management module 50 is, as shown in fig. 7, configured to set each sensitive word in the preset sensitive word stock, and further configured to update each sensitive word in the preset sensitive word stock, for example, based on the sensitive message list.

The sensitive words in the preset sensitive word library can also correspond to sender identifications, so that the sensitive words can be configured for the senders in a manner of corresponding the sensitive words to the sender identifications, and after any EIM message is received, if the message type of the EIM message indicates that the EIM message corresponds to a picture, the target words are compared with the sensitive words corresponding to the sender identifications of the EIM message after the target words are selected, and personalized setting and monitoring personalization of the sensitive words are achieved.

For the above message monitoring system, the picture message monitoring module 20 is further configured to obtain a sender identifier of the EIM message, and if the sender indicated by the sender identifier belongs to a monitored object, identify a message type of the EIM message, so as to implement real-time monitoring on the EIM message sent by a part of senders through the sender identifier.

An embodiment of the present invention further provides a server, a structure of which is shown in fig. 8, and the server may include: a communication interface 100 and a processor 200.

A communication interface 100 for receiving EIM messages;

a processor 200 for identifying a message type of the EIM message; if the message type indicates that the EIM message corresponds to a picture, identifying text contents and object contents in the picture to obtain words related to the picture and confidence coefficients of the words; selecting at least one target word from each word based on the confidence of each word; and if at least one target word is matched with the sensitive words in the preset sensitive word bank, controlling the contents corresponding to the sensitive words in the picture to be invisible.

Optionally, the apparatus may further include a memory 300, an input unit 400, a display 500, and a communication bus 600.

The communication interface 100, the processor 200, the memory 300, the input unit 400, and the display 500 all communicate with each other via a communication bus 600.

In this embodiment, the processor 200 may be a Central Processing Unit (CPU), an application specific integrated circuit, a digital signal processor, an off-the-shelf programmable gate array, or other programmable logic device.

The processor may call a program stored in the memory 300. The memory 300 is used for storing one or more programs, and the program may include program codes including computer operation instructions, and in this embodiment, at least the program for implementing the message monitoring method is stored in the memory.

In one possible implementation, the memory 300 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function (such as preset sensitive word stock update, etc.), and the like; the storage data area can store data involved in the use process of the server, such as sensitive words and the like.

Further, the memory 300 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device or other volatile solid state storage device.

The communication interface 100 may be an interface of a communication module, such as an interface of a GSM module.

Of course, the structure of the server shown in fig. 8 does not limit the device in the present embodiment, and in practical applications, the server may include more or less components than those shown in fig. 8, or some components in combination.

The embodiment of the invention also provides a storage medium, wherein the storage medium stores computer executable instructions, and the computer executable instructions are loaded and executed by the processor to realize the message monitoring method.

It should be noted that, in the present specification, the embodiments are all described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments may be referred to each other. For the system class embodiment and the server class embodiment, since they are basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.

Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.

Claims

1. A message monitoring method, the method comprising:

if the message type indicates that the enterprise instant messaging message corresponds to a picture, identifying text content and object content in the picture to obtain words associated with the picture and confidence degrees of the words, wherein the object content indicates content in the picture except the text content;

2. The method of claim 1, wherein the identifying the text content and the object content in the picture, and obtaining the words associated with the picture and the confidence of the words comprises:

calculating a confidence level for the word.

3. The method of claim 1, wherein the controlling that content in the picture corresponding to the sensitive word is invisible comprises:

or

4. The method according to any one of claims 1 to 3, wherein if the at least one target word matches a sensitive word in a preset sensitive lexicon, the method further comprises:

and/or

5. The method of claim 1, further comprising: acquiring a sender identifier of the enterprise instant messaging message; if the sender identification indicates that the sender belongs to a monitored object, executing a step of identifying the message type of the enterprise instant messaging message;

and/or

6. The method of claim 1, wherein if the message type indicates that the instant messaging business message corresponds to a picture, identifying text content and object content in the picture comprises:

7. The method according to any one of claims 1, 2, 3, 5 and 6, wherein the controlling that the content corresponding to the sensitive word in the picture is invisible if the at least one target word matches the sensitive word in a preset sensitive word bank comprises:

8. A message monitoring system, the system comprising:

a picture message monitoring module, configured to identify text content and object content in a picture if the message type indicates that the enterprise instant messaging message corresponds to the picture, to obtain words associated with the picture and confidence levels of the words, where the object content indicates content in the picture other than the text content;

the sensitive word comparison module is used for selecting at least one target word from the words based on the confidence of each word and comparing the at least one target word with the sensitive words in a preset sensitive word bank;

9. The system of claim 8, wherein if the at least one target word matches a sensitive word in a preset sensitive word bank, the monitoring result processing module is further configured to:

and/or

10. A server, characterized in that the server comprises: a communication interface and a processor;

the processor is used for identifying the message type of the enterprise instant messaging message; if the message type indicates that the enterprise instant messaging message corresponds to a picture, identifying text content and object content in the picture to obtain words associated with the picture and confidence degrees of the words, wherein the object content indicates content in the picture except the text content; selecting at least one target word from the respective words based on the confidence of the respective words; and if the at least one target word is matched with a sensitive word in a preset sensitive word bank, controlling the content corresponding to the sensitive word in the picture to be invisible.