JP2024504884A - SYSTEMS, METHODS AND COMPUTER-READABLE MEDIA FOR IMAGE RECOGNITION - Google Patents

Abstract

The present invention provides a system, method, and computer-readable medium for image recognition. The present invention relates to a system, method, and computer-readable medium for image recognition, and the method includes the steps of: obtaining a first pattern to be displayed on a user terminal; The present invention improves interactions during conference calls or group calls and synchronizes online communications, including the steps of comparing portions of users displayed on user terminals and updating the results of the comparison. make it easier. [Selection diagram] Figure 2C

Description

The present invention relates to image or video processing in live video streaming or video teleconferencing.

Various techniques are known that allow users to participate in online communications with each other. Its applications include live streaming, live conference calls, and more. As these applications become more widespread, users are increasingly demanding improved interaction experiences and smoother synchronization during communication.

A method according to one embodiment of the present invention is an image recognition method. The method of the present invention includes a step of obtaining a first pattern to be displayed on a user terminal, a step of comparing the first pattern with a portion of the user displayed on the user terminal, and updating the result of the comparison. and a step of doing so.

A system according to an embodiment of the present invention is a system for image recognition including one or more processors, the one or more processors executing machine-readable instructions to obtain a first pattern to be displayed on a user terminal. comparing the first pattern with a portion of the user displayed on the user terminal; and updating a result of the comparison.

A computer-readable medium according to an embodiment of the present invention is a non-transitory computer-readable medium that includes a program for image recognition, and the computer-readable medium is a non-transitory computer-readable medium that includes a program for image recognition. the first pattern, the step of comparing the first pattern with the user's portion displayed on the user terminal, and the step of updating the result of the comparison.

FIG. 2 is a schematic diagram showing an example of a group call. FIG. 2 is a schematic diagram illustrating exemplary interactions in a group call in accordance with some embodiments of the present invention. FIG. 2 is a schematic diagram illustrating an example interaction in a group call in accordance with some embodiments of the present invention. FIG. 2 is a schematic diagram illustrating exemplary interactions in a group call in accordance with some embodiments of the present invention. FIG. 2 is a schematic diagram illustrating exemplary interactions in a group call in accordance with some embodiments of the present invention. FIG. 2 is a schematic diagram illustrating exemplary interactions in a group call in accordance with some embodiments of the present invention. FIG. 2 is a schematic diagram illustrating exemplary interactions in a group call in accordance with some embodiments of the present invention. FIG. 2 is a schematic diagram illustrating exemplary interactions in a group call in accordance with some embodiments of the present invention. FIG. 2 is a schematic diagram illustrating exemplary interactions in live streaming according to some embodiments of the present invention. 1 is a schematic diagram illustrating a configuration of a communication system according to some embodiments of the present invention. FIG. 1 is an exemplary sequence diagram illustrating the operation of a communication system according to some embodiments of the present invention. FIG. 1 is an exemplary functional block diagram of a communication system according to some embodiments of the present invention. FIG.

Some live streaming services, applications (APPs) or platforms allow multiple users (streamers, viewers, broadcasters, anchors, etc.) to participate in group call mode or conference call mode, of which: Images of multiple users are displayed simultaneously on the screen of the user terminal to indicate their group call or participation in the group call. The user terminal can be a smartphone, tablet, personal computer, or laptop, and is used by one of the users to participate in the group call.

FIG. 1 shows a schematic diagram illustrating an example of a group call. S1 is a screen of the user terminal that displays the group call. RA is a display area within the screen S1, and displays user A's live video. RB is a display area within the screen S1, and displays user B's live video. RC is a display area within the screen S1, and displays user C's live video. RD is a display area within the screen S1, and displays live video of user D. The live video of the user A may be captured and provided by a video capture device such as a camera placed near the user A. The live video of the user B may be captured and provided by a video capturing device such as a camera placed near the user B. The live video of the user C may be captured and provided by a video capture device such as a camera placed near the user C. The live video of the user D may be captured by a video capturing device such as a camera placed near the user D, and may be provided.

Traditionally, users A, B, C, and D operate in their own display areas, lacking collective interaction across different display areas. In order to increase the motivation of users to participate in group calls, more collective or interesting interactions are desired.

Traditionally, when problems such as delay, lag, or desync occur in the connection between users A, B, C, and D during a group call, one user may not notice the connection problem immediately, and other users may Sometimes I keep talking without being able to receive (audio or video) in sync or smoothly. Or, if you have concerns or suspicions that an out-of-sync problem is occurring, there is no convenient way for users to test or uncover it. Therefore, it would be desirable to be able to initiate a convenient method for users to test the synchronization status of group call connections.

FIGS. 2A, 2B, and 2C provide schematic diagrams illustrating exemplary interactions in a group call in accordance with some embodiments of the present invention.

As shown in FIG. 2A, a trajectory or contour O1 is displayed on the screen S1. The outline O1 is displayed across the display areas RA, RB, RC, and RD. A portion of the outline O1 is displayed in each display area. In this embodiment, the display areas RA, RB, RC, and RD are separated from each other. In some embodiments, the display areas RA, RB, RC, and RD may partially overlap. In this embodiment, the contour O1 is heart-shaped. In some embodiments, the contour can be any shape, such as circular, oval, square, etc. The contour O1 may also be referred to as a first pattern. The contour O1 may be obtained from a server or a user terminal.

As shown in FIG. 2B, users A, B, C, and D try to match the contour O1. The contour O1 is compared with the user A, user B, user C, and user D portions. User A uses his or her hands or arms (which may be partially covered by clothing) to fit or fit the part of the outline O1 displayed within the display area RA. User B uses his or her hands or arms (which may be partially covered by clothing) to fit or fit the part of the outline O1 displayed within the display area RB. The user C uses his or her hands or arms (which may be partially covered with clothing) to fit or fit the part of the outline O1 displayed within the display area RC. User D uses his or her hands or arms (which may be partially covered by clothing) to fit or match the part of the outline O1 displayed within the display area RD.

As shown in FIG. 2C, a special effect O11 is displayed on the screen S1. The special effect O11 can function as an update of the comparison result between the contour O1 and the user A, user B, user C, and user D portions. The special effect O11 may be a graphic object, an animation object, or a materialization object of the contour O1. In some embodiments, the special effect O11 is such that an aggregate shape or an aggregate pattern formed or constituted by the part of the user (such as the body parts or non-body parts of users A, B, C, D) is Displayed when matching or conforming to O1. The set pattern is formed across display areas RA, RB, RC, and RD.

In some implementations, the aggregate pattern formed by the users may be recognized by a system servicing the group call that has a user terminal or pattern recognition process. In some embodiments, the pattern recognition process may include a gesture recognition process, a skin recognition process, a contour recognition process, a shape recognition process, or an object recognition process. In some implementations, the pattern recognition process may include an image comparison process (such as comparing the series of images to a predetermined initial background image for each user's display area) or a moving object detection process. In some implementations, motion estimation (ME) or motion compensation (MC) methods may be used. In some implementations, the aggregate pattern may be referred to as a second pattern.

The determination as to whether or not the set pattern matches the contour O1 (or the comparison between the contour O1 and the user's part) may be performed by image or pattern comparison/matching processing. It may also include calculating a similarity coefficient (correlation value, etc.) between the set pattern and the contour O1. In some embodiments, the calculation of the similarity coefficient may include a correlation calculation process, a trajectory superimposition process, a normalization process, or a minimum distance determination process. The image or pattern comparison process may be performed by a user terminal or a system providing the group call service. In some embodiments, the special effect O11 is displayed when the similarity coefficient is greater than or equal to a predetermined value.

3A and 3B depict other schematic diagrams illustrating exemplary interactions in a group call in accordance with some embodiments of the present invention.

As shown in FIG. 3A, the contour O1 continues to move due to the offset, and the position of the user's part (such as the hand) is adjusted in real time. In this embodiment, the contour O1 is moved laterally. In another embodiment, the contour O1 may be moved vertically, diagonally or in any direction. In some embodiments, the contour O1 may be moved in the front-back direction. In some implementations, the contour O1 may be moved periodically in time periods such as 1 second, 2 seconds, or 3 seconds. In some implementations, the contour O1 may be moved intermittently at different time periods.

As shown in FIG. 3B, the special effect O11 is displayed on the screen S1. The special effect O11 may be a graphic object, an animation object, or a materialization object of the contour O1. In some embodiments, the special effect O11 is such that an aggregate shape or an aggregate pattern formed or constituted by the part of the user (such as the body parts or non-body parts of users A, B, C, D) is Displayed when matching or conforming to O1. In this embodiment, the contour O1 is a moving contour or a moving object. Therefore, users A, B, C, and D need to move their parts in accordance with the movement of the contour O1 so that the collective pattern continuously or periodically matches or fits the contour O1. .

The aggregation pattern may be recognized continuously or periodically by the user terminal or the system providing the group call service with pattern recognition processing. The determination as to whether or not the aggregate pattern matches the outline O1 may be performed by image or pattern comparison processing, and this processing is performed using a similarity coefficient between the moving aggregate pattern and the moving outline O1. (correlation value, etc.) may be calculated continuously or periodically. The image or pattern comparison process may be performed by a user terminal or a system providing the group call service. In some embodiments, the special effect O11 is displayed when the similarity coefficient is continuously or periodically equal to or greater than a predetermined value. For example, the special effect O11 is such that the similarity coefficient is equal to or greater than a predetermined value for a predetermined period (such as 5 seconds or 10 seconds) or a predetermined number of cycles (such as the number of operations of the contour O1 is 3 or 5 times). Displayed in some cases.

FIG. 4 depicts another schematic diagram illustrating exemplary interactions in a group call in accordance with some embodiments of the present invention.

As shown in FIG. 4, the contour O1 is rotated continuously, periodically or intermittently at an offset angle. In some embodiments, the contour O1 may be rotated in the front-back direction. In some implementations, the contour O1 may be rotated periodically in time periods such as 1 second, 2 seconds, or 3 seconds. In some implementations, the contour O1 may be rotated intermittently at different time periods.

The special effect O11 is displayed on the screen S1. The special effect O11 may be a graphic object, an animation object, or a materialization object of the contour O1. In some embodiments, the special effect O11 is such that an aggregate shape or an aggregate pattern formed or constituted by the part of the user (such as the body parts or non-body parts of users A, B, C, D) is Displayed when matching or conforming to O1. In this embodiment, the contour O1 is a moving contour or a moving object (here, "movement" includes "rotation"). Therefore, users A, B, C, and D move their parts in accordance with the movement (or rotation) of the contour O1 so that the collective pattern continuously or periodically matches or adapts to the contour O1. It is necessary to do so.

The aggregation pattern may be recognized continuously or periodically by the user terminal or the system providing the group call service with pattern recognition processing. The determination as to whether or not the set pattern matches the outline O1 may be performed by image or pattern comparison processing, and this processing is performed by comparing the moving/rotating set pattern and the moving/rotating outline O1. The method may include continuously or periodically calculating a similarity coefficient (correlation value, etc.) of . The image or pattern comparison process may be performed by a user terminal or a system providing the group call service. In some embodiments, the special effect O11 is displayed when the similarity coefficient is continuously or periodically equal to or greater than a predetermined value. For example, the special effect O11 is such that the similarity coefficient is equal to or greater than a predetermined value for a predetermined period (such as 5 seconds or 10 seconds) or a predetermined number of cycles (such as the number of operations of the contour O1 is 3 or 5 times). Displayed in some cases.

In some embodiments, continuously or periodically matching the contour O1 with the aggregate pattern formed by the users tests the synchronization state or synchronization level of the connections between the users of the group call. It can also be used as a method. For example, if a user feels that the communication is not smooth, the user may display the outline O1 through the user terminal, periodically move the outline O1, and display the set pattern formed by the user. A series of steps can be started, including a step of periodically recognizing the pattern and a step of periodically comparing the set pattern and the contour O1. A synchronization level may be determined to be acceptable if periodic matching (eg, matching for a predetermined period of time or a predetermined number of cycles) is achieved.

For example, if the similarity coefficient between the contour O1 and the set pattern is greater than or equal to a predetermined value for a predetermined period of time, a message may be displayed on the user terminal indicating that the synchronization level is acceptable. The message may be a special effect, such as special effect O11, or any other form. Conversely, if the similarity coefficient between the contour O1 and the set pattern is less than a predetermined value, a message indicating that the synchronization level is unacceptable may be displayed on the user terminal. In this case, the user can know that the connection state is not good, and can slow down the pace of communication or change to another communication method. In some implementations, the synchronization test described above may be performed before starting an online game or any event that requires a smooth connection between users.

FIG. 5 depicts another schematic diagram illustrating exemplary interactions in a group call in accordance with some embodiments of the present invention.

In this embodiment, users A, B, C, and D use the object or tool T1 to match or fit the contour O1. The tool T1 includes a flexible material and can be bent or deformed to fit a particular shape. For example, the user A can bend the tool T1 to match the part of the outline O1 displayed in the display area RA. In some embodiments, information such as the color and shape of the tool T1 can be taught to the user terminal or the system servicing the group call. The user terminal or the system can recognize the aggregation pattern formed by users A, B, C, D using the information of the tool T1.

FIG. 6 depicts another schematic diagram illustrating exemplary interactions in live streaming (which may be in the form of a group call or other than a group call) in accordance with some embodiments of the present invention.

In this embodiment, users A, B, C, and D are displayed within a single display area on the screen S1 of the user terminal. Users A, B, C, and D may actually be in the same space. Alternatively, users A, B, C, and D may be in separate spaces and their respective images or videos may be transferred to one another through image processing or video processing performed by the user terminal or the system participating in or providing the streaming service in question. may be combined into one display area. Similar to the embodiment described above, users A, B, C, and D attempt to form a set pattern that matches the contour O1. If a match is achieved, special effects may occur.

In some implementations, a special effect may also be considered as a reward upon successful matching of a collective pattern (formed by the user's parts) and a predetermined contour. Further, the matching mechanism may be considered as a method to enliven the atmosphere and interaction of live streaming in which multiple users participate. For example, a successful match may initiate or trigger a donation, or a gift may be sent by the user who initiated the matching process. The user who initiates the matching process may be a streamer (or anchor, broadcaster) or a viewer (or fan) of the streaming. Therefore, more collective or interesting interaction between users involved in streaming or live distribution is realized. This could lead to increased gift sending and increased revenue for streaming service providers or streamers, as well as improved platform performance and content production. In some implementations, the sending of a gift is triggered or realized based on the comparison between the contour O1 and the part of the user.

FIG. 7 shows a schematic diagram showing the configuration of a communication system based on some embodiments of the present invention. The communication system 1 can provide live streaming services with interaction through content. "Content" here refers to digital content that can be played back on a computer device. The communication system 1 allows users to participate in real-time interactions with other users online. The communication system 1 includes a plurality of user terminals 10, a backend server 30, and a streaming server 40. The user terminal 10, the backend server 30, and the streaming server 40 are connected via a network 90 (for example, the Internet may be used). Backend server 30 may be a server that synchronizes interactions between user terminals and/or streaming server 40 . In some implementations, backend server 30 may be an application (APP) provider's origin server. Streaming server 40 is a server for handling or providing streaming data or video data. In some implementations, backend server 30 and streaming server 40 may be independent servers. In some implementations, backend server 30 and streaming server 40 may be integrated into one server. In some implementations, user terminal 10 is a client device for live streaming. In some implementations, user terminal 10 may be referred to as a viewer, streamer, anchor, podcaster, audience, listener, etc. Each of the user terminals 10, the back-end servers 30, and the streaming servers 40 are examples of information processing devices. In some implementations, streaming can be live streaming or video playback. In some implementations, streaming can be audio streaming and/or video streaming. In some implementations, streaming can include content such as online shopping, talk shows, talent shows, entertainment events, sporting events, music videos, movies, comedy, concerts, group calls, conference calls, and the like.

FIG. 8 shows an exemplary sequence diagram illustrating the operation of a communication system according to some embodiments of the present invention.

The processing unit set 120 may include components or devices of the user terminal 10A used by the user A. For example, the processing unit set 120 may include image/video capture devices such as renderers, encoders, decoders, CPUs, GPUs, controllers, processors, and/or cameras.

The display 122 may refer to the display of the user terminal 10A used by user A.

The processing unit set 140 may include components or devices of the user terminal 10B used by user B. For example, the processing unit set 140 may include image/video capture devices such as renderers, encoders, decoders, CPUs, GPUs, controllers, processors, and/or cameras.

The display 142 may refer to the display of the user terminal used by user B.

The message unit 32 is configured to communicate messages or signals with devices such as user terminals, backend servers or streaming servers. In some implementations, the message unit 32 may be implemented within a backend server. In some implementations, the message unit 32 may be implemented independently of the backend server.

Shape recognition unit 50 is configured to perform shape/pattern detection/recognition processes, including gesture recognition processes, skin recognition processes, contour recognition processes, and/or object recognition processes. The shape recognition unit 50 may be configured to perform a pattern comparison process that includes calculating a similarity coefficient or correlation value between two patterns. According to another embodiment, the shape recognition unit 50 may be implemented in a user terminal, a backend server, or independently. In this embodiment, both the pattern recognition process and the pattern comparison process are executed by the shape recognition unit 50. In some implementations, these two processes may be performed by different units, and each may be implemented in the user terminal, backend server, or independently based on the actual application. .

In step S800, the processing unit set 120 renders video data for display on the display 122, which may include an image of user A captured by a video capture device.

In step S802, the processing unit set 120 transmits the encoded video data to the streaming server 40. The encoded video data may include user A's video encoded by the encoder.

In step S804, the processing unit set 140 renders video data for display on the display 142, which may include the video of user B captured by the video capture device.

In step S806, the processing unit set 140 transmits the encoded video data to the streaming server 40. The encoded video data may include user B's video encoded by the encoder.

At step S808, user B's video data is retrieved from the streaming server 40 and displayed on the display 122. Note that some processing such as decoding, processing, and rendering of the extracted video data is omitted here.

At step S810, user A's video data is retrieved from the streaming server 40 and displayed on the display 142. Note that some processing such as decoding, processing, and rendering of the extracted video data is omitted here.

In step S812, shape or pattern information is sent from the message unit 32 to the processing unit set 120.

In step S814, shape or pattern information is sent from the message unit 32 to the processing unit set 140. Step S812 and step S814 may be performed simultaneously.

In some implementations, the transmission of shape information may be triggered by user interaction via a user terminal. The user may be a streamer, broadcaster, anchor, or viewer. In some implementations, the transmission of shape information may be triggered by a communications or streaming service provider.

In step S816, the processing unit set 120 renders a shape or pattern corresponding to the received or acquired shape information and displays it on the display 122. Therefore, user A can confirm on the screen the pattern that needs to be matched (eg, the portion of the pattern in user A's display area).

In step S818, the processing unit set 140 renders a shape or pattern corresponding to the received or acquired shape information and displays it on the display 142. Therefore, user B can see on the screen the pattern that needs to be matched (eg, the portion of the pattern in user B's display area).

In step S820, user A attempts to fill in, match, or fit the shape. For example, user A may attempt to behave so that user A or a portion of a tool used by user A (displayed on the display 122) conforms to the portion of the shape displayed in user A's display area. do.

In step S822, the user attempts to fill, match, or fit the shape. For example, user B may attempt to behave such that user B or a portion of a tool used by user B (displayed on the display 142) conforms to the portion of the shape displayed in user B's display area. do.

In step S824, the shape recognition unit 50 detects or recognizes a set shape or a set pattern formed by user A and user B. The recognized aggregate shape may be displayed on the display 122. In this embodiment, the shape recognition unit 50 recognizes the initial shape (or a predetermined shape). The shape recognition unit 50 may calculate a similarity coefficient or a correlation value between the set shape and the predetermined shape, and determine whether matching is established using a predetermined threshold.

In step S828, the matching result is sent from the shape recognition unit 50 to the message unit 32.

In step S830, the message unit 32 notifies the processing unit set 120 of the matching result.

In step S832, the message unit 32 notifies the processing unit set 140 of the matching result. Step S830 and step S832 may be performed simultaneously.

In step S834, if the matching is successful, special effects are rendered on the display 122. The special effect may include a message indicating that communication between user A and user B has been smoothly synchronized.

In step S836, if the matching is successful, special effects are rendered on the display 142. The special effect may include a message indicating that communication between user A and user B has been smoothly synchronized.

In some implementations, the message unit 32 may periodically send shape information to each user terminal and include an offset in the shape information each time. Therefore, a moving or rotating shape can be displayed on the display of each user terminal. The subsequent pattern recognition processing and pattern matching processing may also be executed periodically and used as a communication synchronization level check. For example, the user can visually know the synchronization status by looking at the display.

Regarding the screen S1 in FIG. 1 (or FIG. 2A), the screen S1 can be implemented on the user terminal of user A, user B, user C, or user D. In some implementations, the communication system may be configured to swap the display areas for any two of the users. The communication system may receive a replacement instruction from one of the user terminals. This increases the degree of freedom in arrangement and further improves interaction between users.

In some implementations, the screen S1 of FIG. 1 (or FIG. 2A) may be implemented on a streamer's user terminal. In this case, the streamer instructs the users A to D what to do, thereby deepening their interaction. This strengthens the sense of unity between the streamer and the users A to D. Any of the users A to D can become a streamer or a viewer.

In some implementations, the trajectory or said contour O1 is selected by one of the users or by a vote from a list of candidate contours, each of which is associated with a respective gift amount or with a value amount. tied to each gift. The amount (or amount of value) may be set such that the larger the amount, the more difficult it is to match the respective contour. In some implementations, the gift value corresponds to the contour/pattern matching difficulty. Alternatively, the contour O1 may be arbitrarily selected from a list of candidate contours, or the contour O1 may be determined based on the profiles of the participating users A to D.

FIG. 9 shows an exemplary functional block diagram of a communication system according to some embodiments of the present invention. In this embodiment, the shape recognition unit 50 may be implemented independently rather than within the backend server 30, streaming server 40 or user terminal. In another embodiment, the shape recognition unit 50 may be implemented within a backend server, a streaming server, a cache server, or a user terminal. In this embodiment, the message unit 32 is implemented within the backend server 30. In another embodiment, the message unit 32 may be implemented external to the backend server 30. For example, the message unit 32 may be implemented independently, within a streaming server, a cache server, or a user terminal.

The user terminal 10A is a user terminal used by user A. The user terminal 10A includes a processing unit set 120 and a display 122. The processing unit set 120 may include an image/video capture device such as a renderer, an encoder, a decoder, a CPU, a GPU, a controller, a processor, and/or a camera.

User terminal 10B is a user terminal used by user B. The user terminal 10B includes a processing unit set 140 and a display 142. The processing unit set 120 may include an image/video capture device such as a renderer, an encoder, a decoder, a CPU, a GPU, a controller, a processor, and/or a camera.

As shown in FIGS. 8 and 9, the message unit 32 may transmit shape or pattern information to the processing unit set 120 of the user terminal 10A and the processing unit set 140 of the user terminal 10B. The transmission of the pattern information described above may be triggered by a user terminal (such as a streamer's or viewer's user terminal), a server in the system, or an application supporting communication, of which the processing unit set 120 may receive or A shape or pattern corresponding to the acquired shape information is rendered and displayed on the display 122. Therefore, user A can confirm on the screen the pattern that needs to be matched (eg, the portion of the pattern in user A's display area).

The processing unit set 140 renders a shape or pattern corresponding to the received or acquired shape information and displays it on the display 142. Therefore, user B can see on the screen the pattern that needs to be matched (eg, the portion of the pattern in user B's display area).

Thereafter, the processing unit set 120 and the processing unit set 140 collect video/image data that users A and B want to match to their respective parts of the pattern, and send the data to the shape recognition unit 50.

The shape recognition unit 50 detects or recognizes a collective shape or a collective pattern formed by user A and user B. In this embodiment, the shape recognition unit 50 displays the information displayed on the display 122 (for example, in step S816 in FIG. 8) in response to the collective shape and the shape information received in step S812 in FIG. Compare with the initial shape (or predetermined shape). The shape recognition unit 50 may calculate a similarity coefficient or a correlation value between the set shape and the predetermined shape, and determine whether matching is established using a predetermined threshold. The shape recognition unit 50 then sends the matching result to the message unit 32.

The message unit 32 notifies the processing unit set 120 and the processing unit set 140 of the matching result. If the matching is successful, the processing unit set 120 and the processing unit set 140 may render special effects for display on the display 122 and the display 142. The special effect may include a message indicating that communication between user A and user B has been smoothly synchronized.

The present invention improves interaction during conference calls or group calls, facilitates synchronization of online communications, and increases revenue for online communication service providers.

The processes and procedures described in this invention, in addition to those explicitly described, may be implemented by software, hardware, or any combination thereof. For example, the processes and procedures described herein implement logic corresponding to the processes and procedures in a medium such as an integrated circuit, volatile memory, non-volatile memory, non-transitory computer-readable medium, magnetic disk, etc. This can be achieved by Further, the processes and procedures described in this specification can be realized as a computer program corresponding to the processes and procedures, and can be executed by various computers.

The systems or methods described in the above embodiments may be integrated into programs stored on non-transitory computer-readable media, such as solid state storage, optical disk storage, magnetic disk storage, and the like. Alternatively, the program may be downloaded from a server via the Internet and executed by the processor.

Although the technical contents and features of the present invention have been described above, those with ordinary knowledge in the technical field to which the present invention pertains will appreciate that many variations and modifications can be made without departing from the teachings and disclosure of the present invention. It can be performed. Therefore, the scope of the invention is not limited to the embodiments already disclosed, but includes other variations and modifications that do not depart from the invention and are included in the scope of the claims.

S1 Screen RA Area RB Area RC Area RD Area O1 Object O11 Special effect T1 Tool 1 System 10 User terminal 120 Processing unit set 122 Display 140 Processing unit set 142 Display 30 Back-end server 32 Message unit 40 Streaming server 50 Shape detection unit 90 Network S800, S802, S804, S806, S808, S810, S812, S814, S816, S818, S820, S822, S824, S828, S830, S832, S834, S836 Step 10A User terminal 10B User terminal

Claims (17)

An image recognition method,
obtaining a first pattern displayed on a user terminal;
comparing the first pattern with a portion of the user displayed on the user terminal;
updating the results of the comparison;
An image recognition method comprising: The image recognition method according to claim 1, characterized in that the position of the part of the user is adjusted in real time. The comparison between the first pattern and the portion of the user includes calculating a similarity coefficient between the first pattern and a second pattern composed of the portion of the user. The image recognition method according to claim 1. 4. The image recognition method according to claim 3, further comprising the step of displaying a special effect when the similarity coefficient is greater than or equal to a predetermined value. moreover,
Periodically acquiring the first pattern displayed on the user terminal;
Periodically calculating the similarity coefficient between the first pattern and the second pattern;
4. The image recognition method according to claim 3, wherein the first pattern, which is periodically acquired, moves on the user terminal. Further, the method further includes the step of displaying a message indicating that synchronization is acceptable on the user terminal when the periodically calculated similarity coefficient is equal to or greater than a predetermined value for a predetermined period of time. The image recognition method according to claim 5. The method further includes a step of recognizing the second pattern in gesture recognition processing, skin recognition processing, contour recognition processing, shape detection processing, object recognition processing, image comparison processing, or moving object detection processing. , The image recognition method according to claim 3. The image recognition method according to claim 1, characterized in that the parts of the user include body parts or non-body parts of the user. 3. The calculation of the similarity coefficient between the first pattern and the second pattern includes a correlation calculation process, a trajectory superimposition process, a normalization process, or a minimum distance determination process. Image recognition method described in. 2. The image recognition method according to claim 1, wherein the users are displayed in respective display areas on the user terminal, and the respective display areas are separated from each other. The image recognition method according to claim 10, further comprising the step of exchanging the display areas of any two of the users. The image recognition method according to claim 1, characterized in that the user is displayed within one display area on the user terminal. 2. The image recognition method according to claim 1, wherein the first pattern spans two mutually separated display areas displaying the user. The image recognition method according to claim 1, further comprising the step of realizing the sending of a gift based on the result of the comparison. 15. The image recognition method according to claim 14, wherein the value of the gift corresponds to a matching difficulty level of the first pattern. An image recognition system comprising one or more processors, the one or more processors executing machine-readable instructions;
obtaining a first pattern displayed on a user terminal;
comparing the first pattern with a portion of the user displayed on the user terminal;
updating the results of the comparison;
An image recognition method characterized by performing the following steps. A non-transitory computer-readable medium comprising a program for image recognition, wherein the program is stored in one or more computers;
obtaining a first pattern displayed on a user terminal;
comparing the first pattern with a portion of the user displayed on the user terminal;
updating the results of the comparison;
An image recognition method characterized by performing the following.