[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [E-devel] Memory pool management

sorry, forget the list :S

2006/3/22, QliX=D! [aka Ezequiel Brizuela] <qlixed@gmail.com>:
I think in a rewrite of the garbage collector to get a full benefit of performance of realloc in glibc (specially in linux that support MREMAP), Let me Introduce:

The implementation of realloc is as follows:

From: http://sources.redhat.com/cgi-bin/cvsweb.cgi/~checkout~/libc/malloc/malloc.c?rev=1.158&content-type=text/plain&cvsroot=glibc
/* realloc(Void_t* p, size_t n)

  The returned pointer may or may not be the same as p. The algorithm
  prefers extending p when possible, otherwise it employs the
  equivalent of a malloc-copy-free sequence.


  if n is for fewer bytes than already held by p, the newly unused
  space is lopped off and freed if possible.  Unless the #define
  REALLOC_ZERO_BYTES_FREES is set, realloc with a size argument of
  zero (re)allocates a minimum-sized chunk.

  Large chunks that were internally obtained via mmap will always
  be reallocated using malloc-copy-free sequences unless
  the system supports MREMAP (currently only linux).

So if yot use realloc we get the benefits of fewer mem moves produced at malloc-and-copy in the GC.
I think that get really important when you get a lot of object (before 100's I think), and in the worst case, you get the same performance that malloc-and-copy but with fews lines of code :D
And now, how to rewrite it?, If U start to walk on the list forward and find a hole (a object free) U can pick up the last object allocated and put in there, and so on, until u reach the last object allocated in the list.
Then do the realloc over object, this give us 2 benefits:
  * Compact memory in a fast way.
  * Give a Last-big-chunck, specially good for unknow size allocator.

Maybe I got a patch and benchmarks for this in a couple of day, couse i need to end somethings after, but if someone can do now the changes to test, please, don't be shame :D.

QliX=D! [aka Ezequiel Brizuela]

2006/3/21, Cedric BAIL < cedric.bail@free.fr>:

        I have maid with the help of Olivier a little library that could, I hope,
improve memory management in enlightenment.

        The idea behind it was to provide a fast allocator which improve data
locality when manipulating the same object. For that purpose, the library
provide a set of function for the manipulation of memory pool dedicated to a
specific type (size in fact) of object. It also provide another memory pool
for data without the size known when the memory pool is initialised (it use
multiple memory pool for fixed size, a little bit like the kernel).

        I have also made a patch for eet_lib.c (after a little valgrind check, it
seems to be the most used part of enlightenment) and it improve a little bit
the performance. Well without the patch, I wasn't patient enough to see the
menu of enlightenment, with it I didn't need to be patient :-) This patch is
not very clean as I didn't know when the initialisation process took place, I
just check in all function that use the memory pool, if it's correctly
initialized. I am sure, we can come with a better solution.

        You will find every thing here: http://chac.le-poulpe.net/~cedric/ememoa/

        If you want a maximum speed for this library, set CFLAGS to "-O3
-finline-functions -DNDEBUG".

        I must thanks Olivier for all the autotools stuff, the reviewing of the code
and also for the name of the library.

        I don't know if you are interested in more patch for this type of memory
management (Seems ecore_list and evas_list could also benefit from this). But
some part of EFL and enlightenment couldn't benefit from it, because the
allocation is done by fonction at the end of the call tree (take for example
the data chunk used by eet in eet_read). This force the caller to free it,
and the called is forced to allocate it without knowing what usage the caller
will do with the data. I really don't know what to do about this, but I think
it certainly has a performance impact on enlightenment.

In any case, I hope this library would be usefull for this project,

This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
enlightenment-devel mailing list