Week 4

This document summarizes work done/being done in week 4.

1 Implementation + Issues

This week was also spent in trying to implement instrumentation and profiling, with more success than last time.

2 Libraries/Tools

I decided to push forward with a javaagent based implementation which I talked about last week. However, I decided to use a library other than ASM, which I was using last time. ASM provides low-level utilities for rewriting the bytecode directly at class loading time. This time, I went for libraries using ASM as their base but providing a higher level of abstraction (which would make my life easier). Even though I did this, I still faced some rather surprising issues, which I'll be talking about here.

3 Organization

fig1.png

The time is in terms of total bytes allocated since inception. The basic idea is to track all objects that have been every allocated by maintaining a WeakRef to them, and then checking all WeakRef s after a GC operation to see which ones of them still point to an object, and which one of them are null pointers. We can then remove the size of those objects from the heapSize we maintain.

Another field, lastUse is maintained inside an ObjData. This keeps track of the last time when we needed this object (in terms of total bytes allocated since inception).

4 Allocation/Deallocation

I found a fairly nice library which helped me track allocation: https://github.com/google/allocation-instrumenter.

Using this, and the method of keeping weak references, I was mostly able to keep track of objects. My code now runs with two javaagent s, one for this, and one of my own.

There were a few issues I faced while doing this (issues which I don't think are a part of the picture when not writing a javaagent). For instance, at some points, when a class was loaded and an object corresponding to that class was created, the hashCode method was unable to work - and this was a problem for me because the set I was using was a java.util.HashSet, which needed that method. This threw a rather strange NullPointerException during the running of the code, since a hash code is something all Object s do have. The simple workaround for this was

try {
  objects.add(o);
  heapSize += o.getSize();
  heapNumber += 1;
} catch (NullPointerException nex) {
  // Gracefully accept it
} catch (Exception ex) {
  throw ex;
}

5 Lastuse Tracking

I've not been able to get this working satisfactorily yet, but I have a fair idea in mind. I'll be using a library called Byte Buddy: https://bytebuddy.net.

I chose this library because it is documented fairly well, is not as low-level as ASM, and the developer of this library is quite active on StackOverFlow, so I will be able to ask for help in case anything goes terribly wrong.

I will be adding a third java agent which is based on this, which will also transform class files at load time. I will intercept all method calls (and field accesses are modeled as method calls), and find the object corresponding to that method, and update lastUse for that in the ObjData. The main problem I had last time, that of accessing this during operations, should be solved, since the library provides a @This decorator which is supposed to be available for intercepting methods.

This might be a bit slow, because I will need to iterate through the whole Set of weak references to compare it with the object reference this. I can probably solve this by making ObjStore.objects a java.util.Map from hash codes of objects to their data.

I started working on intercepting methods and faced certain issues which I'm working on solving. The first one seems to arise from the fact that I need to add a javaagent for ByteBuddy as well, which doesn't seem to play well with the last library's javaagent - I'm able to get basic method interception working, but if I enable both the agents, I get a java.util.ConcurrentModificationException. Usually, you get this if you're modifying a structure while iterating over it - but that does not seem to be the case here.

Author: Milind Luthra

Emacs 25.3.1 (Org mode 8.2.10)

Validate