WO2016181200A1

WO2016181200A1 - A data processing method

Info

Publication number: WO2016181200A1
Application number: PCT/IB2015/057320
Authority: WO
Inventors: Grigory Victorovich DEMCHENKO
Original assignee: Yandex Europe Ag; Yandex Llc; Yandex Inc.
Priority date: 2015-05-14
Filing date: 2015-09-23
Publication date: 2016-11-17
Also published as: RU2015117932A; RU2632416C2

Abstract

A method for transferring an object from a first process to a second process. The first process has a first memory space and the second process has a second memory space. A computing device executing the first process comprises provides a first copy of the object in a first contiguous portion of the first memory space and provides a second copy of the object in a second contiguous portion of the first memory space. For each memory location of the first and second copy of the object, respective values stored at corresponding locations within the copies of the object are compared. An index of locations within the first and second copies of the object where respective values at corresponding locations do not match is generated. A copy of the object and the index of locations is provided for transfer to the second process so that the second process can use the index of locations to adjust the locations within the copy of the object for the second memory space.

Description

A DATA PROCESSING METHOD Cross-Referenced Applications

The present application refers to Russian Patent Application No. 2014139545 filed 30 September 2014 and to Russian Patent Application No. 2015117932 filed 14 May 2015 content of which are herein incorporated by reference in its entirety.

Field

The present technology teaches a method of providing intermediate data generated during computer program execution from a first process to a second process.

Background There are many instances where it can be desirable to transfer intermediate data encapsulated within a program object from one instance of a computer program to another. For example, it may be desirable to move data processing from one computer to another to balance load among a group or cluster of computers. Alternatively, having processed a request from a client across a network, a server, for example, a database server, might wish to transfer a program object back to the client to allow the client to continue program object processing.

Program objects can comprise a combination of executable code as well as data and so can either perform processing themselves and/or provide data for other program objects to process. For example, a program object such as a function or sub-routine might be instantiated by a parent object or even a main program to perform some processing on behalf of the calling parent object or main program. Object data can comprise a variety of types comprising, for example: primitive data types such as integers, real numbers, Boolean, characters; and structured or abstract data types such as arrays or lists and user-defined data types, each including multiple instances of data or combinations of data types. Program data can either be stored directly within a program object memory space or be indexed by pointers which contain addresses of other program objects or data. Handling the transfer of program objects including pointers can be problematic, in particular, when attempting to move a program object from a program or process running within one memory address space on a computer to another program or process potentially using another memory address space. D. M. Dhamdhere, "Operating Systems: A Concept-based Approach", ISBN: 0070611947, 2006 discloses a general approach to allocating memory at an operating system (OS) level of a computer.

There are known solutions for switching data from absolute addressing to relative addressing to avoid problems when providing data from a first computer (or process) to another computer (or process). However, such solutions can require memory management tools capable of analyzing memory content and associating it with a process. This analysis can be onerous especially if there is no a priori knowledge of the nature of the object to be handled.

US 8,566,536, discloses direct access sharing of physical memory between processes. Memory address space is mapped to each of the processes by populating a first entry in a top level virtual address table for each of the processes. As each address space is being created and mapped to a given process, a master kernel commences generation of a master list of the entry in the top level virtual address table of each address space for each process. Thus, the address space of each of the processes is cross-mapped into each of the processes by populating one or more subsequent entries of the top level virtual address table with the first entry in the top level virtual address table from other processes. This technique is however reliant on functionality being provided by a processor core.

It is an object of the present technology to provide a more readily deployed mechanism for transferring program objects between processes. Summary

In accordance with a first broad aspect of the present technology, there is provided a data processing method for transferring an object from a first process to a second process, the first process having a first memory address space and the second process having a second memory address space. The method is executable by a processor of a computing device executing the first process and comprises providing a first copy of the object in a first contiguous portion of the first memory address space; and providing a second copy of the object in a second contiguous portion of the first memory address space. For each memory location of the first and second copy of the object, respective values stored at corresponding locations within the first copy of the object and the second copy of the object are compared. An index of locations within the first and second copies of the object where respective values at corresponding locations do not match is generated. A copy of the object and the index of locations is provided for transfer to the second process so that the second process can use the index of locations to adjust the locations within the copy of the object for the second memory address space. In some embodiments, the method further comprises receiving an indicator of said object to be transferred; and substituting a custom memory allocator for a default memory allocator, the custom memory allocator being configured to allocate memory in a contiguous portion of memory. In this case, providing said first copy comprises copying said object to be transferred into said first contiguous portion of memory allocated by said custom memory allocator; and providing said second copy comprises copying said object to be transferred into said second contiguous portion of memory allocated by said custom memory allocator.

Alternatively, the method further comprises: receiving an indicator of said object to be transferred, said object occupying said first contiguous portion of memory allocated by a custom memory allocator. In this case, providing said second copy comprises copying said object to be transferred into said second contiguous portion of memory allocated by said custom memory allocator.

In some embodiments, providing a copy of said object comprises copying one of said first copy of said object or said second copy of said object to a buffer.

In some embodiments, said copied first or second copy of said object is located at a base address within said first memory address space and said method further comprises subtracting said base address from values of said copy of said object in said buffer at each location in said index of locations.

In some embodiments, the copied first or second copy of said object is located at a base address within said first memory address space and the method further comprises subtracting said base address from values of said copied first or second copy of said object at each location in said index of locations prior to copying to said buffer. Some embodiments further comprise providing a size of said object with said copy of said object and said index of locations for transfer to said second process.

Some embodiments are implemented with a serializing function performing the above method steps in response to a call by a parent object to transfer said object. In some cases, said first process is instantiated on a first computing apparatus and the second process is instantiated on a second different computing apparatus.

Alternatively, said first process is instantiated on a first computing apparatus and the second process is instantiated on the first computing apparatus at a later time.

In some embodiments, providing said copy of said object and said index of locations for transfer comprises writing said copy of said object and said index of locations to persistent memory. In this case, the persistent memory can comprise one of computer memory or non-volatile memory accessible to each of said first and second processes.

Alternatively, providing said copy of said object and said index of locations for transfer comprises transmitting said copy of said object and said index of locations to a computing apparatus across a network link.

In some implementations, the first process is a compiled C program and said custom memory allocator implemented with an overloaded malloc() function.

In other implementations, the first process is a compiled C++ program and said custom memory allocator implemented with an overloaded new() function. Each of said first and second memory address spaces can be virtual memory address spaces.

In accordance with a second broad aspect of the present technology, there is provided a data processing method for transferring an object from a first process to a second process, the first process having a first memory address space and the second process having a second memory address space. The method is executable by a processor of a computing device executing the second process and comprises obtaining a copy of the object and an index of locations within the copy of the object indicating pointer locations within the copy of the object. The object is copied to a contiguous portion of the second memory address space, the portion having a base address. The base address value is added to values of the object in the contiguous portion of the second memory address space at each location in the index of locations.

Some embodiments further comprise obtaining a base address for said object within said first memory address space and subtracting said base address for said object within said first memory address space from values of said object in said contiguous portion of said second memory address space at each location in said index of locations.

Some embodiments comprise obtaining a size of said object and allocating a contiguous portion of memory of at least said size with said second memory address space.

Some embodiments are implemented with a de-serializing function performing the above method steps in response to a call by a parent object to receive said object.

In another aspect there is provided a computer program product comprising executable instructions stored on a computer readable medium which when executed on a computing apparatus are arranged to perform the above methods.

In a still further aspect, there is provided a data processing system comprising a first computing apparatus connected to a second computing apparatus, each arranged to perform the above methods.

Brief Description of the Drawings

Various embodiments will now be described, by way of example, with reference to the accompanying drawings, in which: Figure 1 illustrates schematically a system for providing intermediate data generated during computer program execution from a first process to a second process, the system being implemented in accordance with non-limiting embodiments of the present technology;

Figure 2 illustrates a serialising method operable within the system of Figure 1 ;

Figure 3 illustrates a de-serialising method operable within the system of Figure 1 ; and Figure 4 illustrates an exemplary serialisation and deserialization of an object using the methods illustrated in Figures 2 and 3. Description of the Embodiments

Referring to Figure 1, there is shown a diagram of a system 100 including computing apparatus 20 and 20'. It is to be expressly understood that the system 100 is merely one possible implementation of the present technology. Thus, the description thereof that follows is intended to be only a description of illustrative examples of the present technology. This description is not intended to define the scope or set forth the bounds of the present technology. In some cases, what are believed to be helpful examples of modifications to system 100 may also be set forth below.

This is done merely as an aid to understanding, and, again, not to define the scope or set forth the bounds of the present technology. These modifications are not an exhaustive list, and, as a person skilled in the art would understand, other modifications are likely possible. Further, where this has not been done (i.e. where no examples of modifications have been set forth), it should not be interpreted that no modifications are possible and/or that what is described is the sole manner of implementing that element of the present technology. As a person skilled in the art would understand, this is likely not the case. In addition it is to be understood that the system 100 may provide in certain instances a simple implementation of the present technology, and that where such is the case they have been presented in this manner as an aid to understanding. As persons skilled in the art would understand, various implementations of the present technology may be of a greater complexity. In a first embodiment, the computing apparatus 20 is communicatively coupled with a data storage device 30 which stores program code 10 for a program. The data storage device 30 can be a memory device such as a hard disk integrated with the computing apparatus 20 or the data storage device 30 can be connected to the computing apparatus 20 via a network (not depicted) or indeed any suitable wired or wireless connection. In the context of the present specification, unless expressly provided otherwise, "computing apparatus" is any computer hardware that is capable of running software appropriate to the relevant task at hand. Thus, some (non-limiting) examples of electronic devices include general purpose personal computers (desktops, laptops, netbooks, etc.), mobile computing devices, smartphones, and tablets, as well as network equipment such as routers, switches, and gateways. It should be noted that a device acting as a computing apparatus in the present context is not precluded from acting as a server to other electronic devices. The use of the expression "a computing apparatus" does not preclude multiple electronic devices being used in receiving/sending, carrying out or causing to be carried out any task or request, or the consequences of any task or request, or steps of any method described herein. An instance 10' of the program 10 is loaded within an address space in memory 16 of the computing apparatus 20. The memory 16 can comprise a virtual memory system with physical storage separate from the address space, but for the purposes of simplicity, the memory is shown in Figure 1 as a simple block 16.

The instance 10' of the program is particularly arranged to in turn instantiate a number of program objects 12-1 to 12-N (together referred to as program objects 12). As described above, program objects can comprise instances of functions created during execution of the program instance 10' or simple or structured data objects some of which can comprise pointer values, with those pointers pointing to other functions or data. Program objects 12 are typically arranged hierarchically with parent objects causing the instantiation of one or more child objects. Only two layers of child objects are illustrated in Figure 1 with objects 12-1 to 12-N having been instantiated by the instance of program 10 and these being shown as having instantiated one or more child objects 12-X-Y. Thus, program object 12-1 has instantiated object 12-1-1 while program object 12-2 has instantiated objects 12-2-1...12-2-M, and program object 12-N has instantiated object 12-N-l. It will be appreciated however that in a program of even moderate complexity, the object hierarchy can comprise many layers of objects.

It can be desirable for a parent object of the program 10 such as the main program or program object 12-X to transfer a child object 12-X-Y with data at an intermediate stage of processing to another program instance where further processing of the object or by the object may be performed. In order to do so, embodiments of the present technology provide a serialising function and a corresponding de-serialising function.

The serialising function enables a parent object, either the main program 10 or child objects 12-X or further child objects within the first instance 10' of the program, to indicate a child object of theirs which is to be transferred to the serialisation function, with the serialisation function then writing the object information to a buffer 60 which can then be either stored in non-volatile storage such as the data storage device 30 or transmitted across a network connection 40 for subsequent use by another program instance 10". The indication provided to the serialisation function can either include the calling parent object passing the object to be transferred either directly i.e. by passing the entire object, or by reference i.e. using a pointer to the object. The other program instance can be a second instance 10" of the program 10 running on another computing apparatus 20'; the instance can be a later instance of the program 10 running on the same computing apparatus 20 sometime after the first instance 10' has ceased processing; or indeed the second instance can be an instance of a different program than the program 10. In Figure 1 the other program instance 10" is for simplicity shown as a separate program running on a second computing apparatus 20' connected to the first computing apparatus 20.

The de-serialising function enables a parent object in this example, an object 12' within the second program instance 10" to indicate a serialised set of values in a buffer 60 either retrieved from the data storage device 30 or obtained via a network connection 40 and to obtain from these values, an instance of program object in particular having pointer values which are correctly mapped to an address space in memory 16' in which the second program instance 10" is instantiated. In either case, the serialised information retrieved by a de-serialising function is described as being obtained from a buffer 60 - this does not imply that the information has been stored in non-volatile memory such as the data storage device 30, the buffer could just as easily comprise a transitory receiver buffer in a network receiver of the computing apparatus 20'. Referring now to Figure 2 which illustrates the steps involved in serialising an object according to the embodiments of the present technology.

In a first step 200 performed by a parent object, the object to be serialised is prepared. This involves bespoke program code within the program instance 10' processing data to the extent required before it is to be transferred to another program for further processing. So for example, where the program instance 10' is a database program, this preparation may comprise the parent object (which is likely to be a child of other objects) responding to a request from a client program, performing an SQL query on the database and then generating a database relation object including the query results with a view to transferring the database relation object back to the requesting client. In other examples, the parent object may wish to transfer an object incorporating a mixture of program code and data to another program instance for further processing.

The next step 202 can either be performed by the parent object or within a requested instance of the serialising function which is to handle the serialisation of the program object and which has been passed either the program object (directly) or a pointer to the program object (by reference) which is to be serialised. In this step, sufficient memory within the memory 16 is allocated to ensure the serialisation function can perform as required. As will be explained, instances of the serialising function need to create either one or two copies of an object to be serialised and so sufficient space needs to be available in memory to do so. As will be appreciated, program objects can either be stored in a program heap or within a program stack. The heap is a region of computer memory whose allocation is not managed automatically by the operating system hosting a program. It is a free-floating region of memory, typically larger than the stack. To allocate memory for variables on the heap in a C language program, built-in C functions malloc() or calloc() are used. In C++, equivalent functions are new() and delete(), with other programming languages using similar functions.

For variables allocated in heap memory, the C function free() or C++ function delete() can be used to de-allocate that memory once that memory is no longer needed. Failing to do this results in memory leakage where memory on the heap will still be set aside and won't be available to other processes. Because of the possibly many allocations and de-allocations of heap memory at program run time, in virtual memory systems, heap memory variables may be stored in non-contiguous portions of virtual memory as well as physical memory.

Thus, unless the calling parent object has made provision otherwise, the serialising function cannot assume that the program object it has been passed (either directly or by reference) is stored in a contiguous portion of memory.

In embodiments of the present technology, where this has not been performed by the calling program or parent object, the serialising function ensures that copies of the program object it is to serialise are stored in a contiguous portion of memory. In step 204, as disclosed in Russian Patent Application No. 2014139545 filed 30 September 2014 (Reference: 2014-0107 / 34055- 401 / B37-1479-01), this involves replacing a global default memory allocator with a custom memory allocator. The custom memory allocator ensures that any copies of the object to be serialised are allocated within respective contiguous portions of the memory which has been allocated for the function at step 202.

For programs written in C or C++, controlling the allocation of the program heap can be achieved by overloading the malloc(), calloc() and new() functions so that as new variables are declared and allocated at program execution time, they are written to a contiguous portion of memory rather than being distributed across non-contiguous memory locations - both in virtual and physical memory. Equivalent techniques can be employed for programs written in other languages; or indeed other techniques for achieving the same result can be employed according to the operating system environment of the program.

If the serialising function has had to replace the default memory allocator, then at steps 206 and 208, the serialising function makes two copies of the object to be serialised within the memory space allocated at step 202.

If a memory allocator allocating objects to contiguous portions of memory is used by default within the program instance 10', then the copy of the object passed to the serialising function (again either directly or by reference), can be used as the first copy, and so the serialising function need only copy the object once to provide two copies of the object each stored in respective contiguous portions of memory for analysis.

In any case, after steps 206 and 208, two copies of the original object occupy respective contiguous portions of memory which are accessible to the serialising function.

Because any form of object can be passed to the serialising function, the serialising function does not make any assumptions about nature of the object or its structure. The serialising function will only know the size of the object and that within the object there will be a combination of pointers and data.

For the two copies of the object, the size is the same and the data part is the same. The same in the context means the same amount of memory and that the parts are identical information-wise. However, pointers values will be different as they point to different instances of the data in respective memory locations for the first and second copies of the object.

At step 210, the serialising function compares the two copies of the object, address-by-address and creates a list of differences between the two instances of the object stored within respective contiguous portions of memory. A simple way to do this programmatically is to set a pointer to a base address of the first copy of the object and to a base address of the second copy of the object and to compare the values stored at each address before advancing each pointer in tandem to the next addresses to be compared and repeating this for each address for the size of the object. Any equivalent technique can also be used. In each case where there is a difference between the contents of the address within the first copy of the object and the corresponding address within the second copy of the object, then the amount of the difference will be the same and will be related to the difference between a base address for the first copy of the object and the base address for the second copy of the object.

Referring now to Figure 4, after step 208 of Figure 2, a first instance of an object, Copy 1 of a given size is stored in a contiguous portion of memory beginning at Base address (1). A second instance, Copy 2, of the same size is stored in a contiguous portion of memory beginning at Base address (2). For simplicity, Figure 4 shows a single address within each instance of the object comprising a pointer, with the pointer pointing to a piece of data elsewhere within the memory space occupied by the instance of the object. Within Copy 1, the pointer value located at an address Base address( 1 )+PtrOffset comprises Base address ( 1 )+DataOffset, while within Copy 2, the pointer value located at an address Base address(2)+PtrOffset comprises Base address (2)+DataOffset. Thus, the difference in value for each instance of pointer within the object to be serialised will be Base address (2)-Base address(l).

Thus in step 210, the serialising function identifies each pointer location PtrOffset within a copy of an object as well as the address Base address+DataOffset stored at the pointer location.

At step 212, a copy of the object and the information produced at step 210 are written to a buffer and either stored in buffer 60 or transmitted to another processing apparatus 20' via a network connection 40. Either the first copy Copy 1 or the second copy Copy 2 can be used as the source for providing the copy of the object for the buffer. One way of encoding the information at step 212 comprises replacing the values at locations identified as comprising pointers with relative addresses and then simply storing a list of the offsets, PtrOffset, for each pointer location within the copy of the object to be transferred. So in the example of Figure 4, where Copy 1 is used as the source, the value Base address (1)+Data0ffset initially stored at the pointer location shown would be replaced with an adjusted relative pointer value DataOffset in the buffer 60, and the address PtrOffset would be stored as an entry in a list (or array) Ptrlndex which is written to the buffer 60 along with the adjusted copy of the object including a relative pointer address at each pointer location. Finally an indicator of the size of the object is also written to the buffer 60. It will be appreciated that the adjusting of the pointer addresses can either be performed on the original copy of the object, before it is written to the buffer; or the adjusting can be performed on the corresponding values within the buffer.

In the former case, Copy 2 may be used as the source, as this copy can be discarded from memory once serialisation is complete, allowing Copy 1 to remain available for use within the program instance 10'. Whether Copy 1 is retained by the program instance 10' depends on whether the object is to be used by the program instance 10' subsequently. For example, it may be desirable to allow both the program instances 10' and 10" to process with the transferred program object in parallel and in this case the original object would need to be retained and so its pointer location values should not be adjusted. Turning now to Figure 3 and the de-serialising function which enables the reading of a serialized memory object. As discussed, de-serialising may happen on another computing apparatus 20', to which the serialised copy of the object was sent or it may be the same computing apparatus 20 with the same memory means or another memory means.

The de-serializing function is called by a parent object such as object 12' which wishes to continue processing with the serialised object. The calling parent object need not be the same object as the parent object which originally serialised the object within the buffer 60, nor as explained does the instance of program 10" need to be the same as the program instance 10'.

The de-serialising function can be passed either a copy of a buffer in which a serialised object has been written or a reference to the location of the buffer, step 300. As shown in Figure 4, the buffer 60 will contain a copy of the object where pointers have been replaced with relative addresses, an index of pointer offsets within the copy of the object, Ptrlndex, and in this example, an indication of the size of the object. Knowing the size of the object within the buffer 60 enables the de-serialising function to determine a boundary 62 between the object information stored in the buffer 60 and the Ptrlndex information stored in the buffer 60.

At step 302, the de-serialising function allocates sufficient memory for the object to be created from the buffer information using the size information obtained from the buffer 60. In the example of Figure 4, the allocated region begin at Base address (3).

At step 304, the de-serialising function then creates the object by copying the object information within the buffer into the memory region allocated for the object at step 302.

At step 306, the Ptrlndex information stored within the buffer 60 after the boundary 62 enables the deserialising function to locate each pointer address within the allocated memory region and to add Base address (3) to the relative pointer value stored at that address to convert the relative pointer value to an absolute pointer value correctly pointing at the correct data address within the memory region 16'.

At step 308, the de-serialising function can then return the de-serialised object to a calling function which can then continue processing using the object.

In another simple worked example, let us take a first copy of an object which occupies a contiguous portion of memory and where the values at each memory location for the object are as follows:

First = 4178[3]2654[5]78

The values shown in brackets indicate pointer addresses although the serialising function has no way of distinguishing these values from data values stored at other memory addresses for the object. However, in this example, when a second copy of the object is copied to a contiguous portion of memory, the values at each memory location might look as follows:

Second = 4178[4]2654[6]78 Note how the pointer values highlighted have changed from one copy of the object to the next and how the difference between the values is constant.

The serialising function, sequentially compares the two copies of the object to locate addresses exhibiting the difference in value resulting from the copy. As can be seen, beginning at index [0], such differences occur at locations [4] and [9] highlighted above. These offsets are stored and the information at these locations is adjusted.

Let us assume that the base address for the first copy of the object is 2. In this case, this value is subtracted from the values at locations [4] and [9] in the first copy to create relative addresses at these locations. The values written to the buffer 60 for this object will therefore be as follows: Buffer = 4178[1]2654[3]78,[4],[9],12.

Again the brackets and punctuation are simply provided for clarity - no such delineation is necessary within the buffer 60.

The last entry in the buffer indicates that the object information occupies the first 12 memory locations of the buffer. The values following the object information and preceding the size information indicate that the values at index [4] and [9] need to be adjusted for the receiving process memory space 16'.

It will be appreciated that the 12 addresses of object information; the 2 indices; and the size information could be written to the buffer 60 in any suitable order enabling a receiver to determine the size of the program object which has been serialised and the locations within the program object which need to be re-adjusted for the receiving process memory space 16'.

Continuing with the example, if the base address for the object in the receiving process memory space 16' were for example = 5, then this value needs to be added at locations [4] and [9] to create the de-serialised object in memory as follows:

Copy: 4178[6]2654[8]78 This de-serialised object can be passed back to any calling parent object 12' for processing as normal including for example, re-serialising the object for further processing by another computing apparatus. It will be seen that many alternative implementations of the above embodiments are possible.

For example, the pointer values within the serialised copy of the object in the buffer 60 need not be adjusted before the object is transferred. In this case, the base address for the original copy of the object within the memory 16 could be written to the buffer 60. Then, the original pointer value, the original base address from the buffer and the base address, e.g. Base address (3), for the de-serialised version of the object in the memory 16' of computing apparatus 20' can then be used when de-serialising the object to adjust for the receiving process memory space 16'. Thus, regardless of whether the object is adjusted before it is transferred to the receiving process as described in relation to Figures 2-4; or the base address for the original object is transferred with the serialised object so that it might be adjusted by the receiver, the object is transferred with sufficient information to make it independent of the base address for the original object and so allow the receiver to adjust for the receiving process memory space 16'.

Also, because Ptrlndex will vary in length according to the number of pointers within the object, an indicator of object size is written to the buffer to enable the de-serialising function to distinguish between object data and Ptrlndex. However, if Ptrlndex were allocated a fixed size, for example, the size of the object, then although likely to be less efficient, it would not be strictly necessary to write the object size to the buffer 60.

Similarly, in the examples provided, Ptrlndex positively identifies locations within the copy of the object corresponding to pointers within the object. Again, although likely to be more inefficient, because one would expect fewer pointers than other values within an object, this information could also be passed negatively, with Ptrlndex (or equivalent) including an indication of locations within the object which are not pointers. This would indicate to a deserialising function which addresses are not to be adjusted for the receiving process memory space 16' leaving the other addresses, i.e. pointer addresses, to be adjusted. In the context of the present specification, unless expressly provided otherwise, the expression "data" includes information of any nature or kind whatsoever capable of being stored, for example, in a database, or transmitted electronically, for example, in a stream. Thus data includes, but is not limited to audio-visual works (images, movies, sound recordings, presentations etc.), location data, numerical data, etc., text (opinions, comments, questions, messages, etc.), documents, spreadsheets, etc.

In the context of the present specification, unless expressly provided otherwise, a "database" is any structured collection of data, irrespective of its particular structure, the database management software, or the computer hardware on which the data is stored, implemented or otherwise rendered available for use. A database may reside on the same hardware as the process that stores or makes use of the information stored in the database or it may reside on separate hardware, such as a dedicated server or plurality of servers.

It will be appreciated that while the above method has been described for exemplary purposes with a specific sequence of steps, the various steps can be rearranged where possible to achieve the same effect.

Embodiments of the present technology find particular utility in for example, the distribution of program objects between devices; facilitating processing in virtual machines where for example processing and/or decision making can be moved between a remote server and local processors; backupYvirtualization systems; and in compilers and code executing applications.

In the context of the present specification, unless expressly provided otherwise, a "server" is a computer program that is running on appropriate hardware and is capable of receiving requests (e.g. from computing apparatus) over a network, and carrying out those requests, or causing those requests to be carried out. The hardware may be one physical computer or one physical computer system, but neither is required to be the case with respect to the present technology. In the present context, the use of the expression a "server" is not intended to mean that every task (e.g. received instructions or requests) or any particular task will have been received, carried out, or caused to be carried out, by the same server (i.e. the same software and/or hardware); it is intended to mean that any number of software elements or hardware devices may be involved in receiving/sending, carrying out or causing to be carried out any task or request, or the consequences of any task or request; and all of this software and hardware may be one server or multiple servers, both of which are included within the expression "at least one server". In the context of the present specification, unless expressly provided otherwise, the expression "computer usable information storage medium" is intended to include media of any nature and kind whatsoever, including RAM, ROM, disks (CD-ROMs, DVDs, floppy disks, hard drivers, etc.), USB keys, solid state-drives, tape drives, etc.

In the context of the present specification, unless expressly provided otherwise, the words "first", "second", "third", etc. have been used as adjectives only for the purpose of allowing for distinction between the nouns that they modify from one another, and not for the purpose of describing any particular relationship between those nouns. Thus, for example, it should be understood that, the use of the terms "first apparatus" and "third apparatus" is not intended to imply any particular order, type, chronology, hierarchy or ranking (for example) of/between the apparatus, nor is their use (by itself) intended imply that any "second apparatus" must necessarily exist in any given situation. Further, as is discussed herein in other contexts, reference to a "first" element and a "second" element does not preclude the two elements from being the same actual real-world element. Thus, for example, in some instances, a "first" apparatus and a "second" apparatus may be the same software and/or hardware, in other cases they may be different software and/or hardware.

Implementations of the present technology each have at least one of the above-mentioned object and/or aspects, but do not necessarily have all of them. It should be understood that some aspects of the present technology that have resulted from attempting to attain the above- mentioned object may not satisfy this object and/or may satisfy other objects not specifically recited herein.

As such, from one perspective, embodiments of the present technology can be summarized as follows, structured in numbered clauses.

CLAUSE 1. A method for transferring an object (12) from a first process (10') to a second process (10"), the first process having a first memory address space (16) and the second process having a second memory address space (16'), the method being executable by a processor of a computing device (20) executing said first process, the method comprising: providing (206) a first copy of said object in a first contiguous portion of said first memory address space; providing (208) a second copy of said object in a second contiguous portion of said first memory address space; for each memory location of said first and second copy of said object, comparing (210) respective values stored at corresponding locations within said first copy of said object and said second copy of said object; generating an index of locations within said first and second copies of said object where said respective values at corresponding locations do not match; providing (212) a copy of said object and said index of locations for transfer to said second process so that said second process can use said index of locations to adjust said locations within said copy of said object for said second memory address space.

CLAUSE 2. A method according to clause 1 wherein said method further comprises: receiving (200) an indicator of said object to be transferred; substituting (204) a custom memory allocator for a default memory allocator, the custom memory allocator being configured to allocate memory in a contiguous portion of memory; wherein providing said first copy comprises copying (206) said object to be transferred into said first contiguous portion of memory allocated by said custom memory allocator; and wherein providing said second copy comprises copying (208) said object to be transferred into said second contiguous portion of memory allocated by said custom memory allocator.

CLAUSE 3. A method according to clause 1 wherein said method further comprises: receiving an indicator of said object to be transferred, said object occupying said first contiguous portion of memory allocated by a custom memory allocator; and wherein providing said second copy comprises copying said object to be transferred into said second contiguous portion of memory allocated by said custom memory allocator.

CLAUSE 4. A method according to clause 1 wherein said providing a copy of said object comprises copying one of said first copy of said object or said second copy of said object to a buffer (60).

CLAUSE 5. A method according to clause 1 wherein said copied first or second copy of said object is located at a base address within said first memory address space and wherein said method further comprises subtracting said base address from values of said copy of said object in said buffer at each location in said index of locations.

CLAUSE 6. A method according to clause 1 wherein said copied first or second copy of said object is located at a base address within said first memory address space and wherein said method further comprises subtracting said base address from values of said copied first or second copy of said object at each location in said index of locations prior to copying to said buffer.

CLAUSE 7. A method according to clause 1 further comprising providing a size of said object with said copy of said object and said index of locations for transfer to said second process.

CLAUSE 8. A method according to clause 1 including a serializing function performing said steps in response to a call by a parent object to transfer said object.

CLAUSE 9. A method according to clause 1 wherein said first process is instantiated on a first computing apparatus (20) and the second process is instantiated on a second different computing apparatus (20').

CLAUSE 10. A method for transferring an object from a first process to a second process, the first process having a first memory address space and the second process having a second memory address space, the method being executable by a processor of a computing device executing said second process, the method comprising: obtaining (300) a copy of said object and an index of locations within said copy of the object indicating pointer locations within said copy of said object; copying (304) said object to a contiguous portion of said second memory address space, said portion having a base address; and adding (306) said base address value to values of said object in said contiguous portion of said second memory address space at each location in said index of locations.

CLAUSE 11. A method according to clause 10 further including obtaining a base address for said object within said first memory address space and subtracting said base address for said object within said first memory address space from values of said object in said contiguous portion of said second memory address space at each location in said index of locations.

CLAUSE 12. A method according to clause 10 further including obtaining a size of said object and allocating (302) a contiguous portion of memory of at least said size with said second memory address space. CLAUSE 13. A method according to clause 10 including a de-serializing function performing said steps in response to a call by a parent object to receive said object.

CLAUSE 14. A computer program product comprising executable instructions stored on a computer readable medium which when executed on a computing apparatus are arranged to perform the method of any one of clauses 1 to 13.

CLAUSE 15. A data processing system comprising a first computing apparatus connected to a second computing apparatus, said first computing apparatus being arranged to perform the steps of clause 1 and the second computing apparatus being arranged to perform the steps of clause 10. Additional and/or alternative features, aspects and advantages of implementations of the present technology will become apparent from the following description, the accompanying drawings and the appended claims. One skilled in the art will appreciate when the instant description refers to "receiving data" from a user that the computing apparatus executing receiving of the data from the user may receive an electronic (or other) signal from the user. One skilled in the art will further appreciate that displaying data to the user via a user-graphical interface (such as the screen of the computing apparatus and the like) may involve transmitting a signal to the user-graphical interface, the signal containing data, which data can be manipulated and at least a portion of the data can be displayed to the user using the user-graphical interface.

Some of these steps and signal sending-receiving are well known in the art and, as such, have been omitted in certain portions of this description for the sake of simplicity. The signals can be sent-received using optical means (such as an optical connection), electronic means (such as using wired or wireless connection), and mechanical means (such as pressure -based, temperature based or any other suitable physical parameter based).

Modifications and improvements to the above-described implementations of the present technology may become apparent to those skilled in the art. The foregoing description is intended to be exemplary rather than limiting. The scope of the present technology is therefore intended to be limited solely by the scope of the appended claims

Claims

1. A method for transferring an object from a first process to a second process, the first process having a first memory address space and the second process having a second memory address space, the method being executable by a processor of a computing device executing said first process, the method comprising: providing a first copy of said object in a first contiguous portion of said first memory address space; providing a second copy of said object in a second contiguous portion of said first memory address space; for each memory location of said first and second copy of said object, comparing respective values stored at corresponding locations within said first copy of said object and said second copy of said object; generating an index of locations within said first and second copies of said object where said respective values at corresponding locations do not match; providing a copy of said object and said index of locations for transfer to said second process so that said second process can use said index of locations to adjust said locations within said copy of said object for said second memory address space.

2. A method according to claim 1 wherein said method further comprises: receiving an indicator of said object to be transferred; substituting a custom memory allocator for a default memory allocator, the custom memory allocator being configured to allocate memory in a contiguous portion of memory; wherein providing said first copy comprises copying said object to be transferred into said first contiguous portion of memory allocated by said custom memory allocator; and wherein providing said second copy comprises copying said object to be transferred into said second contiguous portion of memory allocated by said custom memory allocator.

3. A method according to claim 1 wherein said method further comprises: receiving an indicator of said object to be transferred, said object occupying said first contiguous portion of memory allocated by a custom memory allocator; and wherein providing said second copy comprises copying said object to be transferred into said second contiguous portion of memory allocated by said custom memory allocator.

4. A method according to claim 1 wherein said providing a copy of said object comprises copying one of said first copy of said object or said second copy of said object to a buffer.

5. A method according to claim 1 wherein said copied first or second copy of said object is located at a base address within said first memory address space and wherein said method further comprises subtracting said base address from values of said copy of said object in said buffer at each location in said index of locations.

6. A method according to claim 1 wherein said copied first or second copy of said object is located at a base address within said first memory address space and wherein said method further comprises subtracting said base address from values of said copied first or second copy of said object at each location in said index of locations prior to copying to said buffer.

7. A method according to claim 1 further comprising providing a size of said object with said copy of said object and said index of locations for transfer to said second process.

8. A method according to claim 1 including a serializing function performing said steps in response to a call by a parent object to transfer said object.

9. A method according to claim 1 wherein said first process is instantiated on a first computing apparatus and the second process is instantiated on a second different computing apparatus.

10. A method according to claim 1 wherein said first process is instantiated on a first computing apparatus and the second process is instantiated on the first computing apparatus at a later time.

11. A method according to claim 1 wherein said providing said copy of said object and said index of locations for transfer comprises writing said copy of said object and said index of locations to persistent memory.

12. A method according to claim 11 wherein the persistent memory comprises one of computer memory or non- volatile memory accessible to each of said first and second processes.

13. A method according to claim 1 wherein said providing said copy of said object and said index of locations for transfer comprises transmitting said copy of said object and said index of locations to a computing apparatus across a network link.

14. A method according to claim 2 wherein said first process is a compiled C program and wherein said custom memory allocator implemented with an overloaded malloc() function.

15. A method according to claim 2 wherein said first process is a compiled C++ program and wherein said custom memory allocator implemented with an overloaded new() function.

16. A method according to claim 1 wherein each of said first and second memory address spaces are virtual memory address spaces.

17. A computer program product comprising executable instructions stored on a computer readable medium which when executed on a computing apparatus are arranged to perform the method of claim 1.

18. A method for transferring an object from a first process to a second process, the first process having a first memory address space and the second process having a second memory address space, the method being executable by a processor of a computing device executing said second process, the method comprising: obtaining a copy of said object and an index of locations within said copy of the object indicating pointer locations within said copy of said object; copying said object to a contiguous portion of said second memory address space, said portion having a base address; and adding said base address value to values of said object in said contiguous portion of said second memory address space at each location in said index of locations.

19. A method according to claim 18 further including obtaining a base address for said object within said first memory address space and subtracting said base address for said object within said first memory address space from values of said object in said contiguous portion of said second memory address space at each location in said index of locations.

20. A method according to claim 18 further including obtaining a size of said object and allocating a contiguous portion of memory of at least said size with said second memory address space.

21. A method according to claim 18 including a de-serializing function performing said steps in response to a call by a parent object to receive said object.

22. A computer program product comprising executable instructions stored on a computer readable medium which when executed on a computing apparatus are arranged to perform the method of claim 18.

23. A data processing system comprising a first computing apparatus connected to a second computing apparatus, said first computing apparatus being arranged to perform the steps of claim 1 and the second computing apparatus being arranged to perform the steps of claim 18.

24. A system according to claim 23 wherein said first and second computing apparatus comprise different apparatus.