A hybrid memory resources management library. It is featured with in-place non-volatile Java object programming model.
- In-place data storage on local non-volatile memory
- In-place generic Java object persistence
- Lazily data object loading
- Any map-able device could be used as a non-volatile memory resource
- Reclaim allocated memory when it is no longer used
- Hierarchical cache pool for massive data caching
- A set of persistent data structures is provided by bdmemgeneric project
- Minimize memory footprint on Java heap
- Reduce GC Overhead as following data shown (collected from Apache Spark experiments)
/** * a durable class should be abstract and implemented from Durable interface with @PersistentEntity annotation */@PersistentEntitypublicabstractclassPerson<E> implementsDurable, Comparable<Person<E>>{Eelement; // Generic Type/** * callback for brand new durable object creation */@OverridepublicvoidinitializeAfterCreate(){System.out.println("Initializing After Created")} /** * callback for durable object recovery */@OverridepublicvoidinitializeAfterRestore(){System.out.println("Initializing After Restored")} /** * setup generic info manually to avoid performance penalty */@OverridepublicvoidsetupGenericInfo(EntityFactoryProxy[] efproxies, GenericField.GType[] gftypes){} @TestpublicvoidtestOutput() throwsRetrievePersistentEntityError{System.out.printf("Person %s, Age: %d ( %s ) \n", getName(), getAge(), null == getMother()? "No Recorded Mother" : "Has Recorded Mother")} publicintcompareTo(Person<E> anotherPerson){intret = 0; if (0 == ret) ret = getAge().compareTo(anotherPerson.getAge()); if (0 == ret) ret = getName().compareTo(anotherPerson.getName()); returnret} /** * Getters and Setters for persistent fields with @persistentGetter and @PersistentSetter */@PersistentGetterabstractpublicShortgetAge(); @PersistentSetterabstractpublicvoidsetAge(Shortage); @PersistentGetterabstractpublicStringgetName() throwsRetrievePersistentEntityError; @PersistentSetterabstractpublicvoidsetName(Stringname, booleandestroy) throwsOutOfPersistentMemory, RetrievePersistentEntityError; @PersistentGetterabstractpublicPerson<E> getMother() throwsRetrievePersistentEntityError; @PersistentSetterabstractpublicvoidsetMother(Person<E> mother, booleandestroy) throwsRetrievePersistentEntityError; @PersistentGetterabstractpublicPerson<E> getFather() throwsRetrievePersistentEntityError; @PersistentSetterabstractpublicvoidsetFather(Person<E> mother, booleandestroy) throwsRetrievePersistentEntityError}// create an allocator object with parameters ie. capacity and uriBigDataPMemAllocatoract = newBigDataPMemAllocator(1024 * 1024 * 8, "./pobj_person.dat", true); // fetch underlying capacity of key-value pair store for persistent handler storageKEYCAPACITY = act.persistKeyCapacity(); .... // close it after useact.close();// create a new durable person object from specific allocatorperson = PersonFactory.create(act); // set attributesperson.setAge((short)rand.nextInt(50)); person.setName(String.format("Name: [%s]", UUID.randomUUID().toString()), true); // keep this person on persistent key-value pair storeact.setPersistKey(keyidx, person.getPersistentHandler()); for (intdeep = 0; deep < rand.nextInt(100); ++deep){// create another person as mothermother = PersonFactory.create(act); mother.setAge((short)(50 + rand.nextInt(50))); mother.setName(String.format("Name: [%s]", UUID.randomUUID().toString()), true); // set the person's motherperson.setMother(mother, true); person = mother}for (longi = 0; i < KEYCAPACITY; ++i){System.out.printf("----------Key %d--------------\n", i); // iterate persistent handlers from key-value store of specific allocatorval = act.getPersistKey(i); if (0L == val){break} // restore person objects from specific allocatorPerson<Integer> person = PersonFactory.restore(act, val, true); while (null != person){person.testOutput(); // iterate all mother's ancestorsperson = person.getMother()} }Please see the file LICENSE for information on how this library is licensed.
- src -- the source for the library
- src/main/java -- the Java source for the library
- examples -- Brief examples for this library
- src/main/native -- the native source for the library
- src/test/java -- the Java test & example source for the library
- uml -- modeling documents for the library
- target -- the generated packages for the library
- target/apidocs -- the generated API documents for the library
To build this library, you may need to install some required packages on the build system:
- NVML -- the NVM library (Please compile this library that is tagged with 0.1+b16) (http://pmem.io) [Required]
- JDK -- the Java Develop Kit 1.6 or above (please properly configure JAVA_HOME) [Required]
- PMFS -- the PMFS should be properly installed and configured on Linux system if you want to simulate read latency [Optional]
- PMalloc -- the supported durable memory native library at https://github.com/bigdata-memory/pmalloc.git [Required]
- Javapoet -- the 1.3.1-SNAPSHOT revised for bdmem at https://github.com/wewela/javapoet.git [Required]
Once the build system is setup, the Big Memory Library is built using this command at the top level:
$ mvn clean package -DskipTests -Dmaven.javadoc.skip=true -Dmaven.test.skip=trueTo build and run the unit tests:
$ mvn clean packageTo install this package to local repository:
$ mvn clean installTo build examples:
(Note that the Big Data Memory Library should be installed to local repository at first):
$ cd examples $ mvn clean packageTo run an example:
$ cd examples $ java -jar target/examples-X.X.X(-SSSSS).jar