EP4728489A1 - Method comprising generation of an augmented-reality visual associated with a personalization object - Google Patents

Abstract

Method comprising generation of an augmented-reality visual associated with a personalization object The disclosed method (10) comprises acquisition (ET6) of an image stream; identification (ET7) of a carrier object in the image stream; identification (ET7) of a personalization object in the image stream; if the identified carrier object and personalization object correspond to a pair of associated objects, provision (ET8) of access to an image-stream transformation model associated with this personalization object, such as an image-stream transformation model generating an augmented- reality visual in the image stream

Description

DESCRIPTION

TITLE: Method comprising generation of an augmented-reality visual associated with a personalization object

Embodiments and implementations of the invention relate to data-processing methods, and in particular to a method comprising generation of an augmented-reality visual in an image stream.

There is a need to take advantage of virtual aesthetic ornamentation (conventionally such as colour filters for photographs, or such as augmented-reality objects inserted in a stream of video images) that is unique and exclusive to the virtual identities of individual users.

The expression “augmented reality” is understood to have the conventional meaning given thereto, i.e. in brief techniques that allow virtual objects, such as 2D or 3D images, to be realistically embedded in a sequence of images of the real world, typically in a way computed by a computer system in real time.

The expression “virtual identity” is understood to mean content chosen by a user to be shared with an audience or community, for example via social media, i.e. for example online applications that are based on the philosophy and technology of the Internet and allow the creation and exchange of user-generated content.

Specifically, with the continued improvement in the performance of augmented-reality techniques, the results obtained are sufficiently attractive to be used to generate virtual ornamentation, i.e. in the broad sense a virtual decorative object, such as a virtual jewel, to embellish and personalize a user’s virtual identity.

Some of the embodiments and implementations defined below provide a personalized, unique and exclusive experience of generation by augmented reality of aesthetic ornamentation for a user’s virtual identity.

According to one aspect, in this respect a method is provided comprising:

- acquisition of an image stream;

- identification of a carrier object in the image stream;

- identification of a personalization object in the image stream;

- if the identified carrier object and personalization object correspond to a pair of associated objects, provision of access to an image-stream transformation model associated with the personalization object.

According to one implementation, provision of access to the image-stream transformation model comprises: execution of the transformation model, this generating an augmented-reality visual in the image stream; and display of the image stream with the augmented-reality visual contained therein.

In other implementations, provision of access to the image-stream transformation model may comprise: loading the image-stream transformation model into a memory of the device that acquired the image stream; or unlocking use of the image-stream transformation model, for example in a way associated with a user profile.

For example, the image stream may be acquired in real time with a camera; and the image stream containing the augmented-reality visual may be displayed in real time with an LCD, LED or OLED screen (these well-known acronyms designating liquid-crystal display, light-emitting diode, and organic light-emitting diode, respectively) inter alia.

For example, the step of identifying the carrier object may be executed using a conventional object recognition technique, such as for example implemented by the object recognition module belonging to the photographic filtering engine described in publication US 2016/0203586 Al (dated 14/07/2016).

For example, the step of identification of the personalization object may be executed using: either the machine-learning technology called “SnapML”, which is an extension of the design tool “Lens Studio” distributed by Snap Inc., also known as “Snapchaf ’, the name of the social-media application; or another technique such as an image-tracking technique; or entry of a code associated with purchase of the personalization object.

For example, the augmented-reality visual may be generated using a conventional technique, such as for example the technique described in publication KR20230038604A (dated 20/03/2023), which allows virtual objects to be moved with respect to a real environment and manipulated in other ways; or such as for example the technique described in publication KR20190089091 A (dated 29/07/2019), which allows 3D models of an object of interest to be generated and manipulated based on changes in position in a field of view of an image-capturing device.

According to one implementation, the augmented-reality visual comprises at least one part superposed on the personalization object, in the image stream, so as to transform the aesthetics of the personalization object. According to one implementation, the augmented-reality visual is configured to display, dynamically in the image stream, an animation of a virtual artistic design, for example the artwork of some artistic makeup, on the carrier object, said design emanating from the personalization object.

According to one implementation, identification of the carrier object comprises identification of markers specific to a given type of carrier object, such as facial markers if the carrier object is a face; identification of the personalization object comprises detection of a location of the personalization object in a suitable region of the carrier object determined by the positions of the specific markers.

According to one implementation, a preliminary step comprises near-field communication of preliminary information by a contactless transponder contained in the personalization object, the communicated preliminary information being able to determine at least one pair of associated objects.

The expression “near-field communication” is understood to mean a technique such as defined in standard ISO 14443, usually designated by the acronym NFC (for Near-Field Communication). The expression “contactless transponder” is understood to mean a tag-type NFC chip able to communicate with a reader in a way powered by inductive coupling to an electromagnetic field generated by the reader.

According to one implementation, the communicated preliminary information is further able to command, after the preliminary step, activation of the following steps of said method, i.e. in particular acquisition of the image stream, identification of the objects, generation of the augmented-reality visual and display.

Thus, in this implementation, it is the contactless NFC communication, which typically occurs when an NFC reader approaches the personalization object, that triggers the augmented-reality experience associated with the object. This allows an immediate experience to be delivered, giving the user a pleasant surprise and a sense of exclusivity.

According to one implementation, the personalization object is a tag, a body jewel for example, designed to be adhesively bonded to human skin.

According to one implementation, the carrier object is a human face.

A computer program product is also provided, said computer program product comprising instructions that, when the program is executed by a computer such as smartphone, cause the latter to implement the method such as defined above. A computer-readable recording medium is also provided, said computer- readable recording medium comprising instructions that, when they are executed by a computer such as a smartphone, cause the latter to implement the method such as defined above.

According to another aspect, a personalization object is also defined, said personalization object being designed to implement the method such as defined above.

According to another aspect, a system is also defined, said system comprising a computer, such as a smartphone, and personalization object, for example such as defined above, configured to interact so as to implement the method defined above.

For example, the personalization object is a tag designed to be adhesively bonded to human skin and intended to be worn on a face, the personalization object thus befitting the method, which comprises provision of access to an image-stream transformation model associated with the personalization object, such as a transformation model generating an augmented-reality visual in the acquired image stream, if the identified carrier object and personalization object correspond to the pair of associated objects comprising a carrier object consisting of a human face, and a personalization object consisting of this tag.

For example, the tag is intended to be worn in proximity to an eye, thus befitting the method in which said identification of the personalization object comprises detection of a location of the personalization object in a suitable region of the carrier object, i.e. a region in proximity to an eye, as determined by facial-marker positions.

For example, the tag contains a contactless transponder designed for near- field communication of preliminary information, the preliminary information being able to determine at least one pair of associated objects, and being able to command implementation of said method, and in particular activation of acquisition of the image stream, and of said identification of the objects.

In other words, embodiments and implementations of the invention provide a body jewel with an NFC chip that allows the person wearing it to trigger a personalized experience on his smartphone, for example on a social-media application, using the front or rear camera. This unique and exclusive experience is accessible only to those who own and wear the tag on their face. By virtue of the magical sensation of augmented reality, the user may push back the limits of the physical world and enter a surprising and astonishing beauty universe that intimately corresponds to him. The experience will be reserved for owners of the tag, in particular because of the step of identification of the personalization object.

The tag may itself open the exclusive experience associated with the universe of the body jewel and for example its trademark.

Other advantages and features of the invention will become apparent on examining the detailed description of embodiments and implementations, which are in no way limiting, and the accompanying drawings, in which:

[Fig i];

[Fig.2];

[Fig-3];

[Fig.4];

[Fig.5];

[Fig. 6] illustrate embodiments and implementations of the invention.

Figure 1 illustrates an example of the lived experience of a female or male user taking advantage of this unique and exclusive augmented-reality ornamentation.

In a step ET1, the user acquires a personalization object, for example by purchasing it. The personalization object is for example a decorative accessory, such as a self-adhesive tag, intended to be worn on a carrier object.

In a first example, the personalization object may thus be a self-adhesive tag intended to decorate a carrier object such as a vehicle.

Figure 2 illustrates one preferred example, in which the personalization object BIJ is a beauty accessory intended to be worn on a part of the body VIS of the user, for example a body jewel, i.e. a tag designed to be adhesively bonded to human skin.

The body jewel BIJ may be intended in practice to be worn on the one hand on a particular part of the body, for example on the face VIS, or possibly on a wrist or shoulder, and furthermore in a suitable region REG of the body part, for example on the contour of the right eye of the face VIS.

For example, the publications FR2966704A1 (dated 04/05/2012), FR3007955A1 (dated 09/01/2015) and WO2013164521 Al (07/11/2013) describe body -jewel technologies, the body jewels being referred to as articles for body decoration, and processes for manufacturing such articles.

With reference again to Figure 1, in a step ET2 of the experience 10, the user sticks the body jewel BIJ onto the suitable region REG of his body, i.e. for example in proximity to his right eye. For an advantageously immediate user experience 10 resulting in a pleasant surprise and a sense of exclusivity, it is a contactless NFC communication, which typically occurs when an NFC reader approaches the personalization object in a readout step ET3, that triggers the augmented-reality experience associated with the object, in a preliminary step ET4.

In this respect, the tag BIJ contains a contactless transponder, suitable for near-field communication (NFC) of preliminary information.

The NFC reader may typically be a computer, such as a smartphone, configured for this functionality.

On the one hand, the preliminary information may determine at least one pair of associated objects BIJ- VIS used at the end of the identification steps ET7 described below.

On the other hand, the preliminary information may command, after preliminary step ET4, activation ET5 of the following steps of the experience ET6, ET7, ET8.

In this respect, the preliminary information may for example contain a request to open a social-media application ET5, or an invitation to download the social -media application if it is not installed in the smartphone that served as NFC reader.

The social-media application, for example the application named “Snapchaf ’, is opened in a commercial, private features mode, for example such as described in the publication US2016203586A1 (dated 14/07/2016) in regard to access to private photo filters proposed by third-party merchants, in collaboration with said social-media application Snapchat.

Next, in a step ET6, the user records, in the field of view of a photographic sensor, such as the front or rear camera of the smartphone, the person wearing (or carrier object bearing) the personalization object BIJ from the private feature of the social-media application. Thus, in this step ET6, a continuous image stream is acquired in real time with a camera.

Firstly, in a detection step ET7, the commercial private functionality of the social-media application is configured to identify, in the acquired image stream, a corresponding carrier object VIS, i.e. in this example to detect the presence of a face VIS in the field of view of the camera.

For example, the carrier object may be identified using a conventional object recognition technique, such as for example implemented by the object recognition module belonging to the photographic filtering engine described in publication US 2016/0203586 Al.

Identification of the carrier object VIS may comprise in this respect identification of markers specific to a given type of carrier object, such as facial markers (conventionally: the position of the pupils, of the cheekbones, of the nose, of the mouth, etc.) if the carrier object is a face VIS.

Secondly, in the detection step ET7, the commercial private functionality of the social-media application is configured to identify, in the acquired image stream, a corresponding personalization object BIJ, i.e. in this example to detect the presence of the body jewel BIJ in the field of view of the camera.

Identification of the personalization object BIJ may comprise detecting that the position of the personalization object is indeed located in the suitable region REG of the carrier object VIS. This position may be determined with respect to the positions of the aforementioned specific markers.

For example, step ET7 of identification of the personalization object BIJ may be executed using: either the machine-learning technology called “SnapML”, which is an extension of the design tool “Lens Studio” distributed by Snap Inc., also known as “Snapchaf ’, the name of the social-media application; or another technique such as an image-tracking technique; or entry of a code associated with purchase of the personalization object.

If the identified carrier object and personalization object correspond to a pair of associated objects, for example such as communicated in the preliminary information ET4, then the experience may advantageously comprise generation ET8 of an augmented-reality visual associated with this personalization object BIJ, in the image stream.

For example, the augmented-reality visual may be generated using a technique such as the technique of publication KR20230038604A (dated 20/03/2023), which allows virtual objects to be moved with respect to a real environment and manipulated in other ways; or such as the technique of publication KR20190089091 A (dated 29/07/2019), which allows 3D models of an object of interest to be generated and manipulated based on changes in position in a field of view of an image-capturing device.

For example, the user may choose to record an image-stream sequence including the augmented-reality visual, which is displayed in real time on his smartphone, typically on an LCD, LED or OLED screen inter alia (these well-known acronyms designating liquid-crystal display, light-emitting diode, and organic lightemitting diode, respectively).

Alternatively, step ET8 comprises provision of access to an image-stream transformation model associated with the detected personalization object, in the event of successful detection of the pair of objects in step ET7.

The expression “provision of access to the transformation model” is for example understood to mean “live” application of the augmented-reality visual such as mentioned above and such as described below with reference to Figures 3 to 6, but also other possibilities in respect of access such as loading the transformation model into a memory of the device that acquired the image stream, or unlocking use of the model for subsequent use.

Thus, for example, loading the image-stream transformation model may comprise downloading program code able to implement the model, for example from a web server via an Internet connection such as a Wi-Fi connection or 4G, 5G or LTE connection; and addition of the downloaded program code to a library of transformation models, contained in a memory of the smartphone.

Furthermore, also for example, unlocking use of the image-stream transformation model may for example be associated with a user profile, with a view to possible use on various devices with which the profile is connected. The transformation model may already be contained in the library of transformation models, or indeed be accessed online via streaming, or via one-time download, or even be loaded into a local memory, such as described above.

Moreover, if in the detection step (step ET9), the commercial private functionality of the social-media application does not identify the carrier object VIS in the image stream, or does not identify the personalization object BIJ in the image stream, or even if the identified carrier object and personalization object do not correspond to a pair of associated objects VIS-BIJ, or optionally if the identified personalization object BIJ is not positioned in the suitable region REG of the identified carrier object VIS, then the augmented-reality visual is not generated or displayed in the image stream (step ET10).

Thus, the user is able to witness an experience 10 that is exclusive, because it is reserved for owners of the body jewel (or personalization object) choosing to wear it in the suitable region of the body (or carrier object). The exclusive experience may be associated with the aesthetic universe of the trademark of the body jewel, and, by virtue of the magical sensation of augmented reality, the user may push back the limits of the physical world and enter a surprising and astonishing beauty universe that intimately corresponds to him.

Furthermore, in the step ET8 of display of the augmented-reality effect, the effect may be chosen randomly, for example from a given number (for example 5) of different themes. Since the user does not know which augmented-reality effect he will get, a sense of rarity and excitement is created.

Reference is made in these respects to Figures 3 to 6, which illustrate four examples of themes for augmented-reality visuals, generating aesthetic ornamentation designed to embellish the user’s virtual identity. These examples all correspond to the case where the personalization object is a body jewel BIJ, designed to be placed on the contour of an eye.

Figure 3 shows examples a, b, c, d, e, f of an augmented-reality visual 30 comprising at least one part superposed on the personalization object BIJ, so as to transform the aesthetics of the personalization object BIJ.

Thus, the examples of the augmented-reality visual 30 may for example develop and extend in space the shape of the body jewel, such as for example to evince a universe corresponding to a trademark, for example the look of Japanese calligraphy.

The augmented-reality visuals 30 may advantageously be animated so as to follow in real time the movements of the user's face in the image stream, in a conventional way known per se.

Figure 4 shows examples a, b, c, d, e, f of an augmented-reality visual 40 displaying dynamically, in the image stream, an animation of a virtual artistic design on the user’s face, which animation may for example spread from the body jewel.

In these examples 40, explosions of colours of various shapes and energies may erupt from the centre of the eye or from the body jewel.

Figure 5 shows examples a, b, c, d of an augmented-reality visual 50 displaying dynamically, in the image stream, an animation of a virtual artistic design on the user’s face, which animation may for example spread from the body jewel.

In these examples 50, drawings having the visual texture of a watercolour are generated on the face of the user wearing the body jewel. The drawings may for example represent an animal representative of a universe of a trademark, for example koi carp (ornamental fish symbolic of Japanese culture), which may appear from behind the jewel swimming.

Figure 6 shows steps a, b, c, d, e, f of a storyboard of an augmented-reality visual 60 displaying dynamically, in the image stream, an animation of a virtual artistic design on the user's face, which animation spreads from the body jewel.

In this storyboard 60, the scene of step a corresponds to the actual face of the user wearing the body jewel. The scene of step b corresponds to a tear generated by augmented reality in the comer of the user’s eye. In the scene of step c, the tear expands to draw a Tsubaki-tree shape, with an effect of liquid ink flowing through it. In the scene of step d, the ink and tear dry, and flowers begin to bloom. In the scene of step e, the flowers grow to a final size. In the scene of step f, the largest of the flowers may explode into falling petals, advantageously in response to a movement of the user. For example, the movement may be a wink, or a nod or something else.

It will be noted, in all the examples described with reference to Figures 3 to 6, that it is possible for the user to interact with the augmented-reality visual to animate it further through use of facial expressions, for example the eyes, eyebrows and movements of the mouth.

This interaction gives life to the body jewel on the user’s face, animating it as if by magic according to an artistic direction chosen by the brand and trend boards corresponding thereto. Possible animations may for example include swirling particles, streaks of light, butterflies taking flight, the colours of the body jewel shining randomly, coloured marks that break down into particles that draw calligraphy patterns, etc. in order to create a sense of personalization and dynamism.

Claims

1. Method (10) comprising:

- acquisition (ET6) of an image stream;

- identification (ET7) of a carrier object (VIS) in the image stream;

- identification (ET7) of a personalization object (BIJ) in the image stream;

- if the identified carrier object and personalization object correspond to a pair of associated objects, provision (ET8) of access to an image-stream transformation model associated with the personalization object.

2. Method according to Claim 1, wherein said provision (ET8) of access to the image-stream transformation model comprises: execution of the transformation model, this generating an augmented-reality (AR) visual in the image stream (ET8); and display of the image stream with the augmented-reality visual (30, 40, 50, 60) contained therein.

3. Method according to Claim 2, wherein the augmented-reality visual (AR) comprises at least one part superposed on the personalization object (BIJ), in the image stream, so as to transform the aesthetics of the personalization object (30).

4. Method according to either of Claims 2 and 3, wherein the augmented- reality visual (AR) is configured to display, dynamically in the image stream, an animation of a virtual artistic design on the carrier object, said design emanating from the personalization object (40, 50, 60).

5. Method according to one of Claims 1 to 4, wherein:

- identification of the carrier object (VIS) comprises identification of markers specific to a given type of carrier object, such as facial markers if the carrier object is a face;

- identification of the personalization object (BIJ) comprises detection of a location of the personalization object in a suitable region (REG) of the carrier object (VIS) determined by the positions of the specific markers.

6. Method according to one of Claims 1 to 5, wherein a preliminary step (ET4) comprises near-field communication of preliminary information by a transponder contained in the personalization object (BIJ), the communicated information being able to determine at least one pair of associated objects (BIJ, VIS).

7. Method according to Claim 6, wherein the communicated preliminary information is further able to command, after the preliminary step (ET4), activation (ET5) of the following steps of the method starting with acquisition of the image stream (ET6).

8. Method according to one of Claims 1 to 7, wherein the personalization object (BIJ) is a tag designed to be adhesively bonded to human skin.

9. Method according to one of Claims 1 to 8, wherein the carrier object

(VIS) is a human face.

10. Computer program product comprising instructions that, when the program is executed by a computer, cause the latter to implement the method according to one of Claims 1 to 9.

11. Computer-readable recording medium comprising instructions that, when they are executed by a computer, cause the latter to implement the method according to one of Claims 1 to 9.

12. Personalization object (BIJ) designed to implementthe method according to one of Claims 1 to 9.

13. System comprising a computer and personalization object (BIJ) configured to interact so as to implement the method according to one of Claims 1 to 9.