Feb 21 2008

Duplicate Madness

Tag: NHibernate, ORMSymon Rottem @ 4:08 pm

One of the things that really kills me in NHibernate is the way that when using joins there are often duplicated entities in the result set. The thing that set me off today is a thread I got involved in on the NHiberate forums where the poster was getting what appeared to be bizarre entity duplication in a list property of an object retreived using ISession.Get.

In this particular case the model was…

Person -< Order -< OrderItem

…where each Person had multiple Orders and each Order had multiple OrderItems. Each of the collections (Person.Orders and Order.OrderItems) had been mapped as bags and for performance reasons (presumably appropriate to their context) the chosen fetch strategy for the collections was set to fetch=”join”.

Everyone with me so far?

The problem here is that if you perform an ISession.Get on a Person who has 1 Order and that Order has 3 OrderItems the Person.Orders collection will contain 3 of the same Order. Why? Well this stems from the way the underlying result set returned by the database works - NHibernate formulates a query with joins in it to get all the data in one go (since that’s what we asked for).

The SQL query for ISession.Get(typeof(Person), 300) would look something like this (select clause omitted for brevity):

...FROM Person p INNER JOIN Orders o ON o.PersonId = p.Id INNER JOIN OrderItems i ON i.OrderId = o.Id WHERE p.Id =300;

(Note that this is hand rolled. And they might not be inner joins, but you get the idea.)

Using our scenario of a Person with 1 Order which has 3 OrderItems from above the result set might look like this:

p.Id p.Name o.Id o.PersonId o.Date i.Id i.OrderId i.Description
300 Symon 595 300 21/02/2008 9876 595 Fishing Rod
300 Symon 595 300 21/02/2008 9877 595 Hat
300 Symon 595 300 21/02/2008 9878 595 Boat

Now, you see the Get operation only returns one Person object but the Orders collection is populated from the result set and there are 3 rows there that contain order information so 3 order items are added to the collection.

Now it’s not quite as bad as it looks. The database operation was efficient because we didn’t have to go to the database multiple times and if you’re thinking we’re using up a chunk of extra memory with the unwanted extra Order items you’d be wrong - the list contains 3 references to the same object in memory, so there’s really only one Order there.

In this particular example the duplicates would not be there if the collections were mapped as Sets rather than Bags. This is because a Set does not allow duplicates so they are filtered out when they’re added.

A more likely situation where you’re going to see this behavior is where you use the default fetch strategy in your mapping but you execute an HQL or ICriteria query with a join:

.SELECT k FROM Cat c WHERE INNER JOIN c.Kittens k WHERE c.Colour = :ParentColour AND k.Colour = :KittenColour

When you call the IQuery.List() method this will return a list of Kitten entities, but since there can be more than one Cat that has the same Kitten in their Kittens collection (Mummy and Daddy, right?) the same Kitten may show up more than once in the result set.

This is, apparently, behavior by design. When the original Hibernate (Java version) developers were making decisions they felt that it made more sense for the returned objects to match the result set - I can’t remember the post I read it from since it was more than a year ago now, so I might not have this exactly right and I’d be happy for someone to correct me.

Again, the solution is to feed the results of the returned IList into a set which will remove the duplicates for you. There is also a feature on the ICriteria API that allows you to provide a result set transformer too - if you don’t believe me you can re-read the documentation…


Feb 16 2008

Do I Need Message Prioritisation?

Tag: SOASymon Rottem @ 11:56 am

For a while now I’ve been working on modelling a service based application that accepts messages and sends them to a set of processing services that are spread across several hosts. One of the requirements is that the application should be able to provide different service level agreements for some message originators.

Since this looked like an Service Oriented Architecture (SOA) I started looking at an Enterprise Service Bus (ESB) to standardize the model for communicating between the applications, provide publish-subscribe semantics and allow me to abstract away the underlying message transport system.

At just the right moment Udi Dahan announced that he was releasing NServiceBus, his ESB implementation for .NET, so I started playing around with it. It’s OSS and written in C# but it was pretty new at the time I started looking at it so there wasn’t much documentation available. Regardless, it looked like a good fit so I rolled up my sleeves and started rummaging around in the source and playing with it…and it still looks like a good fit.

Everything seemed to be going well with the solution design until I looked at the SLA requirement and discovered that NServiceBus doesn’t have any notion of message priority. Since my initial approach had taken me toward using MSMQ as the underlying transport and MSMQ has some basic message priority support I thought I’d be able to use it from NServiceBus, but apparently not. A possible workaround was to implement separate queues for separate priorities but since MSMQ for the versions of Windows I’m targeting doesn’t provide remote transactional read it would be necessary to implement dispatchers and the proliferation of queues was starting to look pretty daunting.

So, next stop was to ask Udi so I dropped him an email. After a reasonably lengthy thread where he essentially suggested that trying to use priority in a queuing environment is probably not a great idea for several reasons he said he’d like to think further about the problem. Imagine my surprise and pleasure when he responded with a podcast entitled “Message Priority - You Aren’t Gonna Need It” in which he dedicated 20 minutes to provide further guidance.

His response was well worth listening to and answered a lot of the questions that had been banging around in my head but, as with most of Udi’s podcasts, it raised a whole new set of questions. The net result is that NServiceBus may still work for me but I think I need to start examining other message transports to find an easier path to my solution.

Now I find myself trying to wrap my head around space-based architectures and am looking for a .NET based implementation - if anyone knows of one please post me a comment.


Feb 13 2008

Being a Better Developer

Tag: UncategorizedSymon Rottem @ 5:56 pm

The other day Ben Schierman had the following to say on his blog post about interviewing developers:

The technical responsibility we have is too great to leave in the hands of those who don’t know what they are doing and refuse to learn.

I couldn’t have said it better. As individuals and a community we have need to keep moving forward and to try be better developers all the time.

Sure, some days we don’t have the get-up-and-go to chow down on new knowledge, but if your get-up-and-go got up and went and won’t come back then maybe you need to find something you can be more passionate about.


Feb 12 2008

O/R Mapper Choices

Tag: NHibernateSymon Rottem @ 3:11 pm

I saw this article by Mario Van Damme the other day about some of the decisions you should probably take into account when choosing an Object Relational mapping tool.

Best Practices for Object/Relational Mapping and Persistence APIs

Although published in 2006 it’s a good article on some of the considerations that I didn’t take into account when choosing an ORM and probably should have. Luckily, NHibernate has worked out as a good choice overall

Mind you, when I started with NHibernate I did struggle against transparent persistence - to my mind I should have full control over when all persistence actions take place, so I did everything I could to disable transparent persistence - to my detriment. That said, I’ve learned from my mistakes and have fully embraced NHibernate’s transparent persistence and transactions and all the performance advantages it provides, however there have certainly been some projects where an persistence solution that allows me full control would have been a better choice.

Had I read the Mario’s article I might have had some idea of where I was going before I started.


Feb 10 2008

DDD and NHibernate

Tag: DDD, NHibernate, ORMSymon Rottem @ 12:35 am

I was trawling around the net today and stumbled over a series of posts (some of which are now quite old) by Ben Scheirman about developing applications using DDD with NHibernate. I don’t recall having seen this series when I was starting to get my head around both of these concepts and they would have been invaluable at the time so I thought it would be worth re-posting the links to the posts for posterity’s sake and for those developers out there who are just getting started.

The series was (after having the name refactored a couple of times) called A Journey with Domain Driven Design (and NHibernate). In the series Ben manages manages to cover a few design patterns, some TDD concepts and examples, separation of concerns, configuring NHibernate, writing mapping files and pulling it all together.

Following are the links to the articles and a quick overview of what each part covers:

  • Part 1 - Describes the reasoning behind his approach and the project structure used in the series.
  • Part 2 - Covers the application and its requirements and begins designing the domain model.
  • Part 3 - Demonstrates writing basic unit tests for the domain model.
  • Part 4 - Demonstrates more sophisticated tests that include interactions between domain entities.
  • Part 5 - Database design strategies, configuring NHibernate and starts unit testing the persistence assembly .
  • Part 6 - NHibernate class and component mappings and some tests to check they’re working as expected.
  • Part 7 - NHibernate collection and association mapping.
  • Part 8 - Some refactoring, the introduction to use of the Repository pattern and use of the Template Method design pattern.
  • Part 9 - Wrapping up the feature list and a link to download the latest version of the code.

The series is excellent and recommended reading if you are trying to get off the ground with DDD and NHibernate.


Feb 09 2008

Configuring NHibernate Caches

Tag: NHibernate, ORMSymon Rottem @ 3:05 pm

A little while ago I decided it would be prudent to start using the second level cache in NHibernate to improve performance in a couple of web applications my team and I had written. The database was taking quite a bit of punishment and performance was degrading while the data being queried was often fairly static in nature so caching seemed like the right choice.

I’d seen some samples out there where people had added the cache configuration directives in their .hbm files so off I went and started adding the <cache> element to my class and collection mappings, configured the SysCache provider in my configuration file and voila, we were off and running with the first application. Then I sat down and started thinking…

Now you are probably asking yourself the same question I was - why didn’t I sit down and think first? An excellent question - one I’ll sit down and think about. Hmmm.

The problem was that the applications in question share the same domain model assembly, which in turn share the same NHibernate mapping files but each application has it’s own caching requirements so I couldn’t reuse the new cache settings between applications. For one application we don’t intend to ever write values to a particular class so they could be mapped to the cache as read-only, while in another application the cached value for the same class should be read-write. Since we were looking at using caching for performance reasons it certainly seemed to make sense that we use the cache in the most performant way possible.

After doing a bit of extra reading of the NHibernate docs I realized that you don’t have to add you cache directives to your mapping classes; there’s a single line in the documentation I had missed:

Alternatively (preferrably?), you may specify <class-cache> and <collection-cache> elements in hibernate.cfg.xml.

I guess the lesson here is read the documentation more carefully…

So, after adding the class and collection cache mappings to our NHibernate configuration we also added some cache regions with appropriate expiry periods for each application and turned on the query cache and started up the applications again. The performance difference is quite something - the database load, which was moderately heavy, dropped significantly and the response times for the applications in question suddenly jumped. I never did do any real comparison measurements but the difference was significant enough.

I have noticed, however, that when using the SysCache provider and setting expiration that sometimes the objects aren’t expired from the cache as promptly as I would have expected. I haven’t yet investigated this issue in depth, but it’s nagging at me. I’ll report back when I have more concrete data, but if there’s anyone out there who can shed some light I’d be glad for some feedback.


Feb 04 2008

ALT.NET.UK Conference

Tag: EventsSymon Rottem @ 11:39 pm

This weekend I got the chance to attend the ALT.NET.UK conference in London and it was a blast. There were people present from France (me), Germany, Belgium, Israel, Scotland and all over England and we made a good size group at about 50ish individuals.

For those who haven’t really had a chance to grok what ALT.NET is all about here’s the description from the altnetpedia site:

We are a group of people who are passionate about improving the way we develop software. We recognise there is no single solution, but instead there are multitudes of alternatives that can be applied to different situations. Our community is a place for sharing these alternatives, so that together we can learn, teach, and encourage new ideas.

The Friday consisted of an initial discussion of the rules, introductions and proposals for topics for the next day. Once this arduous chore was complete (in fact, not at all arduous - a pleasure, in fact) we adjourned to the pub to socialize and start talking shop over a couple of beers. The next day the sessions started in earnest.

The first cab off the rank for me was a topic surrounding Inversion of Control and IoC containers. Mike Hadlow kicked this one off and we almost immediately slid more to IoC in general, or more specifically to TDD and it’s relashionship with Dependency Injection (DI). I think there were a couple of people who were disappointed by this turn of events, however as I suspect some people wanted a chance to understand more about the container tools, how they worked and how to use them rather than talk about theory and practice. In the end this didn’t turn out to be a problem as a bunch of us got together and talked about the tools afterward. :)

Roy Osherove and Ken Egozi both took strong and semi-opposing stances here but I’m not sure they really disagreed so much as were talking at the same point from different practical angles. Roy asked the question of whether the use of DI in TDD was often only there because TDD requires it in the absence of mocking to allow units of code to be tested in isolation. Roy does work for TypeMock and was entirely up front with full disclosure, so in some ways it’s not surprising, but from experience I’m inclined to agree.

I was involved in another conversation on the topic with Roy and some others later on over lunch and it does occur to me that there are some benefits to enforcing the use of DI when practicing TDD as it helps enforce separation of concerns, which can aid some developers in maintaining good development practices. There is, however a point of diminishing returns when you get a proliferation of interfaces for things you’re probably never going to want to swap implementations for except during testing. I guess in the end this comes down to a developer having a reasonable idea of when they really need to provide an interface and when it’s appropriate to mock instead. It would certainly be valuable to the community at large if there were a repository of information that helped newer TDD practitioners to make that choice appropriately.

Well, having said more about that than I intended, I’ll save the other sessions I was in for another post but to give you the teaser of their subjects to whet your appetite (or make you feel slightly ill, if we’re on a different wavelength):

  • MVC/Castle
  • Source Control & Build Tools
  • Database Testing
  • NHibernate/LINQ

As a final note, for more information on ALT.NET and the UK event in particular here are a couple of links lifted wholesale from Ben’s blog:

News River - http://newsriver.altnetuk.com/
Google Group - http://groups.google.com/group/altnetuk-discuss?hl=en
Conference Wiki - http://www.altnetpedia.com/London%20Alt.Net.UK%202nd%20Feb%202008.ashx


Feb 04 2008

Who am I?

Tag: UncategorizedSymon Rottem @ 10:36 pm

Is that an existential question or what?

Ok, so what am I? I guess I’m…

  • The manager of a small development team for a multinational messaging company.
  • A C# & VB.NET developer of about 5 years working on a high volume transactional messaging solution at the moment.
  • Providing consultancy services in Australia an Europe.
  • The husband of a talented web developer.
  • Based in Paris (yes, the one in France) .
  • A proponent of Agile practices.
  • Working on my skills in the field of a number of acronyms (eg. TDD, DDD, ORM, SQL, IoC, SoC, SOA and ESBs, to name but a few).
  • An Aus-pom-erican (born in New York, have British citizenship through my mother and grew up in Australia).
  • An avid blog reader - some of my favorites are in the sidebar.
  • Running out of things to add to this list.

 
So, as for who I am - is it really that important? If you really must know you can find me on the NHibernate forums on a regular basis as merge_s.rottem as the result of some issues that occurred during the merging of the old NHibernate forums with the Hibernate forums as I had an account on each with the same name. I try to lead a moderately private life, so I’ll leave it to you to pry if you feel you need to know more. At least for now… :)


Feb 03 2008

And I christen this blog…

Tag: UncategorizedSymon Rottem @ 9:29 pm

Well it had to happen eventually…

After several years of sponging up the information spilling forth from other bloggers and using that information to become a better developer I have finally succumbed to temptation to try to add my own voice to the crowd in an effort to give something back to the community that has been supporting me. And really - if developers are going to try to develop their skills shouldn’t it be a two-way street rather than just take, take take?

Sure, there’s a healthy level of competition in the developer community but it’s the symbiotic relationships that really seem to make us thrive so in that spirit I’m launching the Symbiotic Development blog.

The catalyst that finally got me blogging was my attendance at the ALT.NET.UK conference in London this weekend where I met several very interesting people and had the chance to participate in a variety of collaborative discussions covering a wide range of subjects. During the last several months I have been hearing about how great the ALT.NET conference in Austin was from some of the other players out there and the chance to attend and see what all the fuss was about was an opportunity that was simply too good to pass up. Upon seeing an annoucement on Jeremy Miller’s blog a codebetter.com I started watching the website and as luck would have it I managed to snag myself a position there.

It was an excellent event and I’ll be posting with more information about the sessions I attended once I’ve had a chance to sort my thoughts into a digestible format. Until then…keep on sharing.


« Previous Page