Paging with Hibernate

Hibernate query api has setFirstResults and setMaxResults methods that restricts the result set for ceratain amaount. Typical usage inside a generic method would be like :

public List find(String queryString, Object[] queryArgs, int first, int max) {
 Query q = getEntityManager().createQuery(queryString);
 if (queryArgs != null) {
    setParameters(q, queryArgs);
 }
 q.setFirstResult(first);
 q.setMaxResults(max);
 return q.getResultList();
}

Here If you are using Hibernate with Oracle it will wrap the original query with a select query which will filter the result based on rowid's.

One rule of thumb while using paging is that your query shouldn't have any joins on collections. Than Hibernate will just execute the original query claiming it can't exactly determine the number of results returned. The warning "firstResult/maxResults specified with collection fetch; applying in memory!" is logged and paging only occurs on the ResultSet object.

Luckily the queries I came across mostly could be refactored so that they don't use joins on collections. Like :

   EJBQuery query = new EJBQueryImpl(Foo.class, "f");
 query.join(Joins.joinFetch("f.fooBooSet", "fbs"));
 if(sorguKriteri.getBoo() != null){
    query.addRestriction(Restrictions.addEquals("fbs.boo", sorguKriteri.getBoo()));
 }

Could be rewritten as :

  EJBQuery query = new EJBQueryImpl(Foo.class, "f");
if (sorguKriteri.getBoo() != null) {
query.addIn("f", new EJBQueryImpl(FooBoo.class,"fb", (EJBQueryImpl) query)
   .addProjections(Projections.add("fb.foo"))
   .addRestriction(Restrictions.addEquals("fb.boo",sorguKriteri.getBoo())));
}

A sub select is used to filter the result instead of join.

Another common pattern used with hibernate is using a lazy list which loads results, page by page basis which looks like :

 public class PagingList<E> extends AbstractList<E> implements Serializable {
private Query query;
private int numOfResults = 0;
private int pageSize = 10;
private List<E> currentResults = new ArrayList<E>();
private int currentPageIndex = -1;
public E get(int index) {
  int pageIndex = index / pageSize;
  if (currentPageIndex != pageIndex) {
    currentPageIndex = pageIndex;
    currentResults = getService().find(query, currentPageIndex * pageSize, pageSize);
  }
  return currentResults.get(index % pageSize);
}
public int size() {
  return numOfResults;
}
public PagingList(Query query, long numOfResults, int pageSize) {
  this.query = query;
  this.numOfResults = (int) numOfResults;
  this.pageSize = pageSize;
}
public Service getService() {
  return (Service) SpringUtils.getBean("service");
}
...
}

Here the query is executed when the get method is called on the list object. Down side of this approach is that query size must be determined with a separate select.

Hibernate Id Generation and Oracle Sequences

I thought that @SequenceGenerator allocationSize attribute defined the number to increment the sequence. The API says that anyway... If you forget the fact that, you can't increment the Oracle Sequences more or less than the number you specified while you created them, you may find your self in a trap.

By default you create your sequences with an increment of 1. This becomes a problem if your app. does bulk inserts. Every insert will need a select to the sequence to generate an id which in turn will hinder the performance of app.

One of the ways Hibernate tries to overcome this problem is using a simple strategy called HiLo. Basically Hibernate uses the allocationSize attribute as a multiplier to sequence value. If the sequence is 'x' Hibernate uses id's from '(x-1)*allocationSize' to 'x*allocationSize' to following inserts after a single select and an increment to the sequence.

While I think this is a feasible approach, down side is all the apps. using this sequence will have to use this algorithm to generate the id's, dealing with the fixed allocationSize, or the generated id's may collide with each other.

Instead of using sequences, using a table which stores id's is much better since you can define the increment size to the amount that you actually require. Using the following annotation and the optimizer, causes a simpler and similar behavior that API actually states. Here is the annotation;

1:    @GeneratedValue(strategy = GenerationType.TABLE, generator = "genFooTable")
2:    @GenericGenerator(name = "genFooTable",
3:        strategy = "org.hibernate.id.enhanced.TableGenerator",
4:        parameters = {
5:            @Parameter(name = "table_name", value = "SEQUENCES"),
6:            @Parameter(name = "value_column_name", value = "NEXTVALUE"),
7:            @Parameter(name = "segment_column_name", value = "NAME"),
8:            @Parameter(name = "segment_value", value = "genFooTable"),
9:            @Parameter(name = "increment_size", value = "10"),
10:            @Parameter(name = "optimizer", value = "org.mca.PooledIdOptimizer")
11:        })

The optimizer :

1:  public class PooledIdOptimizer extends org.hibernate.id.enhanced.OptimizerFactory.OptimizerSupport {
2:    private long value;
3:    private long hiValue = -1;
4:    public synchronized Serializable generate(AccessCallback callback) {
5:      if ((hiValue < 0) || (value >= hiValue)) {
6:        value = callback.getNextValue();
7:        hiValue = value + incrementSize;
8:      }
9:      return make(value++);
10:    }

I made a little editing with the Hibernate's original pooled optimizer since I didn't like the way it work.

Cheers!

Hibernate Usage Strategy

Lately I got to refresh my hibernate knowledge. On previous posts I wrote about my experience with hibernate here and how things could get tangled up trying to load objects.

When using hibernate most important decision you have to make is how you are going to manage sessions. Basically you have to choose from a method scoped (service) session or conversation scoped long running sessions. My choice is here is to use the first one. While using a conversation scoped session might be easier to programmer downside is that database access could happen anywhere on your application uncontrolled. I think that could end you up, having to track down some performance bottlenecks, plus it means having a big object (session) in memory for some time.

Strategy

Not using a conversation scoped session forces you to determine an access strategy for your lazy associations. Note that you should still use these strategies even if you are using conversation scoped session, difference is it doesn't force you to.

Using Join Fetches

Joining usually gets omitted but it's the first thing to do. If you need lazy association, of a list of results, you have to join fetch them. Query below loads the cities with its parks:

 select c from City c join fetch c.parks

Of course if you need to list cities which don't have parks you have to use "left join fetch".

Caching

I remember reading, probably, on hibernate reference documentation something like caching is the last thing to do for performance. Now I think that I have underestimated using second level cache. Tables that hold rarely changing data should be loaded eagerly and cached. I also use a separate select to load them. This is how I mark such fields;

1:  @ManyToOne
2:  @Fetch(FetchMode.SELECT)
3:  CityType type;

This will force hibernate to cache the data upon first access.

Using a cache will generate much smaller queries and you wont have to write huge join statements to load lazy stuff.

I'll probably do more work using cache on an clustered environment.

Hibernate Journey

Now

So we have started our new project and by default (not my decision, not saying its a bad decision either, just saying that there could be a better decision) we are going to be using Hibernate.
For those who don't know Hibernate is 'the' ORM solution. ORM is for 'Object Relation Mapping' which is pretty much self explanatory. It maps your relational data model, such as a database schema, to the object model.

The Begining

Back in 2005 we started our project with Hibernate2. There were no annotations yet so we had to use XML files to map the object domain. So here is how the object domain looks like ;

 public class City {
private Long id;
private String name;
... more fields and getter / setters

Our object-domain is made of classes with complex (like other classes or lists...) or primitive (String, Long ...) fields and their getter / setter methods, which is called the 'POJO' (Plain Old Java Object). These POJOs will be representing the rows of a table. String's will be mapped to VARCHAR's on Oracle, complex types will be foreign keys to other tables. Of course we need to specify which POJO is related with which table or which field is mapped to what column. This is done in the mapping XML and here is how it might look like ;

 <class name="foo.City" table="CITY">
    <id column="ID" name="id" type="java.lang.Long">
... some mechanism to generate id's
    </id>
    <property column="NAME" name="name" type="java.lang.String"/>
...

With this model we could save ( 'persist' ) our POJOs to databases with out us explicitly writing insert statements and query our object-domain with a special query language (similar to sql) HQL and get POJOs as result.

The Plugin

We were creating our POJOs through a class modeler (RSAs class diagram plugin) and we realised that we could write a plugin that could also generate the mapping XMLs. All we had to do was mark the class diagrams with some 'StereoType's that we have created;

The things with '<<...>>' are the stereotypes where we could add the table names and such through attributes.

The Evil

I was happily programming until I was told that we were to update to a newer version of Hibernate. Well after the update nothing worked :) . The main reson was that with this version all the relations were 'Lazy' by default (which is the correct way to work with hibernate by the way). To better understand the problem suppose that we are querying the City object. Now we might not be needing the Park's of the City so loading them since it will cause more selects or joins would not be desirable. This is what is trying to be achived with the 'Lazy'. Loading the partial object graph and loading the specific lazy relations only when required.

When we have a City with lazy parks we could load the parks at the time when its getter is called however the city object must have an open session (session is hibernate's transaction unit). If the hibernate session where the city object has been loaded was closed a LazyInitializationException is thrown. Here is a code piece which shows whats ok and whats not :

 session.open();
City aCity = session.load(City.class,1);
City anotherCity = session.load(City.class,2);
aCity().getParks();                      <= OK to call parks here
session.commit();
session.close();
aCity().getParks();                      <= Still OK to call parks here because its already initialized
anotherCity.getParks();                  <= This will throw an exception !

The thing to note here is that Hibernate POJOs are not so 'Plain'. A POJO is actually wrapped with hibernate generated proxies to do the initializing when required. You should always think POJOs with the underlying session object. There are numerous patterns to manage the sessions like 'Session Per View', 'Session Per Conversation' pattern ... You should always think of the object graph you will be using and join the objects on your queries according to that.

5

With the Java 5 came the annotations and it replaced the need to have XMLs. XML is still supported but I don't see any reason why someone choose that to annotations. Here is how our POJO might look like with annotations:

 @Table(name="DIL")
@Entity
class City {
 @Id
 private Long id;
 @Column(name="NAME")
 private String name;
...

Back

Hibernate has a indexing integration (lucene) , a validation framework and caching frameworks. It should be used with care and respect :)