(Version 1.1 – March 20th, 2009)

Developing with today’s platforms, a developer can utilize great tools and libraries to increase development speed and software quality. Two very common in the .NET space are WCF and LINQ to SQL, besides various other O/RM tools like NHibernate and the ADO.NET Entity Framework.

Unfortunately, using a messaging framework like WCF that has to perform quite a lot serialization work in combination with a tool to automatically retrieve data from data sources and create deeply nested object graphs like LINQ to SQL creates some pretty hard challenges. The reason for this is that WCF has to transform the graph into a “flat” message representation. If the graph contains cyclic references, WCF has two options to avoid running in circles (and kill your App) when it encounters an object it already has processed:

1. Abort the whole serialization. This is the default behavior and it is very reasonable to throw an exception here.
2. “Mark” that part of the message as “reference” to an earlier serialized part. This behavior can be introduced using your own WCF components (see the links at the bottom).

For service developers that want to use object-relational mapping tools to automate database queries, neither of one is really sufficient. Although the latter part may be an option in a strict .NET to .NET scenario, it does not implement any standard (because there is none) and thereby makes it nearly impossible for other platforms to use the serialized message with reasonable effort.

Even worse, in the Web 2.0 area more and more WCF services create JSON and XML output for client-side components like ASP.NET AJAX, JQuery, Silverlight and Flex. Therefore it is essential to provide client-side developers with all necessary information in a strctured way without the need to blow up the server side development.

Taking a look at the .NET framework, Microsoft currently does not offer any solution for this in its primary frameworks WCF and LINQ.

After re-designing and overworking our Reference Architecture over the last week, we finally managed to develop an approach here at netzkern to integrate both technologies with each other in a pragmatic and pretty “clean” way. We created a utility called the DataTransferObjectManager that is able to dynamically fill DTOs with the values and copies of all requested references. The trick here is that it does only selectively chooses the right references by letting the client-side developer provide a configuration parameter that indicates which properties he really needs to do his work. All other references are ignored and not returned to the client, they are usually not even loaded from the database.

Let’s have a look. First, we create two entity classes as part of our model. They don’t need to be public as they are never used outside of the core assembly.

namespace App.Core
{
    class House
    {
        public string Name { get; set; }
        public Person Owner { get; set; }
    }

    class Person
    {
        public string Name { get; set; }
        public House Residence { get; set; }
    }
}

Let’s pretend that instances of these classes are being tracked by an O/RM manager component that automatically detects changes and performs updates. Now, let’s create the corresponding DTO’s:

namespace App.DTO
{
    public class HouseDTO
    {
        public string Name { get; set; }
        public PersonDTO Owner { get; set; }
    }

    public class PersonDTO
    {
        public string Name { get; set; }
        public HouseDTO Residence { get; set; }
    }
}

These classes are basically message definitions. In WCF you would use these as DataContracts and annotate them with the appropriate attributes. Instances of these classes are somewhat “dumb”, they only contain data.

Now let’s consider, we have the following object graph:

House h1 = new House();
h1.Name = “House No. 1”;
Person p1 = new Person();
p1.Name = “Person A”;
h1.Owner = p1;
p1.Residence = h1;

This creates a graph with just two objects and a cyclic reference. When you try to serialize “h1” using WCF’s default DataContractSerializer, you get the above mentioned exception. Now let’s use the netzkern.DataTransferObjectManager (or nk.DTOM for short) and see what it can do for us:

// Create an empty base DTO.
HouseDTO h2 = new HouseDTO();

// Call the nk.DTOM and copy the contents of h1 to h2.
DataTransferObjectManager.CreateDataTransferObject(h1, h2);

What we get from this is the following “filled” DTO message object:
h2.Name: “House No. 1”;
h2.Owner: null;

So far we created a flat copy of “h1” with all references excluded. Now let’s include a copy of “h1.Owner” as part of “h2”:

// Create an empty base DTO.
HouseDTO h2 = new HouseDTO();

// Tell the nk.DTOM to include the following properties.
PropertyPathCollection includePaths = new PropertyPathCollection();
includePaths.Add(“Owner”);

// Call the nk.DTOM and copy the contents of h1 to h2.
DataTransferObjectManager.CreateDataTransferObject(h1, h2, includePaths);

The result now includes a copy of “p1” that has been mapped to a PersonDTO type:
h2.Name: “House No. 1”;
h2.Owner: [PersonDTO];
h2.Owner.Name: “Person A”;
h2.Owner.Residence: null;

We can now even include “deeper” properties, that are being copied equally by the nk.DTOM:

// Create an empty base DTO.
HouseDTO h2 = new HouseDTO();

// Tell the nk.DTOM to include the following properties.
PropertyPathCollection includePaths = new PropertyPathCollection();
includePaths.Add(“Owner.Residence”);

// Call the nk.DTOM and copy the contents of h1 to h2.
DataTransferObjectManager.CreateDataTransferObject(h1, h2, includePaths);

That creates the following result:
h2.Name: “House No. 1”;
h2.Owner: [PersonDTO];
h2.Owner.Name: “Person A”;
h2.Owner.Residence: [HouseDTO];
h2.Owner.Residence.Name: “House No. 1”;
h2.Owner.Residence.Owner: null;

So, how do we use this in WCF? After you have successfully loaded entities in your service operation using an O/RM, you just call the nk.DTOM and return the DataTransferObject as result. The trick is to let the client(!) choose which parts of your internal object graph are necessary to do its work. A simple service call might look like this:

IHouseService service = new HouseServiceClient();
HouseRequest req = new HouseRequest();
req.HouseID = houseID;
req.IncludeProperties = “Owner.Residence”;
HouseResponse resp = service.GetHouse(req);

That’s basically it. By using the netzkern.DataTransferObjectManager, you can define your data transfer objects independently of your internal entity objects. You can even omit certain properties and nk.DTOM will ignore them. We will now test this component internally and make it available on CodePlex as soon as it reaches BETA status. We also would love to hear your thoughts, just send a mail to blog@juliusganns.com.

Update
I’m currently writing an article for a leading German .NET magazine about the above mentioned topic. The article will contain more details about the architecture and more samples how to use the library. I will post an update here when it has been published in a couple of weeks.

Update
Microsoft has just announced the .NET RIA Services framework which also targets parts of the above mentioned problems, but solves them in a different way. As far as I understand the library, it may be possible to combine both approaches. When more bits and bytes come available, I will definitely try to integrate those with our framework.

I’ve read about 30 books about Time Management and Productivity over the last few years and there are a couple of things all the techniques in these books have in common. One of them is: Do one thing at a time. This is one of the main reasons why I stay disconnected most the time, and it is also the basic idea behind this post.

This post is about all the little applications and programs you can install on your notebooks and desktops as companion to various online services and websites. Although they come with a lot promises and large feature lists, I have a pretty strict opinion regarding all these little tools: If you can do it on the web, don’t use software.

Not only do they slow down your PC, they also distract you constantly from the one really important thing you should be doing right now. Using your browser to access those services gives you a lot of advantages, that may not be that obvious. Here are my top three reasons not to use desktop counterparts for web services:

1. You are in control of your time and you avoid distractions. The reason for this is pretty simple: If you need to access a website, it is totally up to you not to do so. So there is no chance for any of these services to make that decision for you. This means, you don’t get interrupted (or maybe even disturbed) by all these “tiny little helpers” hiding in your system tray (that is the area right next to your clock in the lower right corner on your desktop).
2. You focus on one thing at a time. As you probably use one browser window in fullscreen mode to access different websites and services, you are automatically focused on one tool at a time. This makes your work much more efficient and effective.
3. You don’t waste time for installing, configuring and maintaining an application that doesn’t work offline anyway, so why not accessing the native website instead? Just put a bookmark on your browser for fast access and enjoy the web.

This doesn’t mean that desktop versions of popular web services are inherently a bad thing. They’re not. But if you use them, you should consider the basic idea this post is based on: Do one thing at a time.

The following list contains the most important applications I use on my desktop although there are web-based counterparts: Windows Live Messenger (because there is no support for advanced features like File Sharing or Video Conferencing in web-based clients like Meebo), Skype (because there is no real web equivalent), EverNote and MindManager (because I need to be able to offline access my stuff and because of the great integration with other applications).

Finally, after 10 years of HTML 4.01, a W3C working group is going to publish a new HTML standard, hopefully cleaned up and bringing more "native" support for the Web Applications without the use of JavaScript for standard behavior (like editing, drag'n'drop and so on). The following Working Draft summarizes the differences between HTML 4.01 and HTML 5.0.

When I started reading Getting Things Done back in 2007, I would have never expected it to change the way I work that fundamentally. There are those things that immediately kick in and help you to structure your day like the Mastering Work Flow Model, but there is even more to GTD than that basic process. This post is about the things I learned in the last 1.5 years and that are easily forgotten.

When you start to implement GTD, especially if you're a techie like me, you will immediately create all these fancy lists, install all these feature-rich tools and register for every web 2.0 productivity application you can find. This "learning by playing around" is an important part of getting familiar with the methodology.
In the beginning, your new tools and your new process will definitely slow you down. And that's okay, because you have to find your way of doing things without violating the core GTD principles. Just remember yourself from time to time, why you're doing the whole thing. GTD is not your new hobby. It's not a new reason to procrastinate. And it's definitely not your new purpose of life.
You're implementing this productivity process to get more done with less thinking and with less effort. Don't stumble around and rework your GTD infrastructure every week by setting up new tools, creating more rules or cramming more fancy "appointments with myself" in your calendar. At some point you should be done with your system (at least for a certain amount of time) and return to the really important things.

Tip #1: Allow yourself to have fun with GTD. Cut yourself some slack and experience with your new process and your new tools. But make sure that you don't get lost in the fancy world of GTD tools and practices. Don't forget the reason you're doing all this stuff.

Playing around is also important because it helps you with another important thing: Do one step at a time. In the very beginning implementing GTD "to the fullest" seems to be the greatest thing on earth. Trust me, doing it all at once is not possible. Give yourself time to learn, to adjust and to integrate the principles into your life. GTD (and every other "habit") only sticks if it fits naturally in your day, without distracting or hindering your way of doing things.

Tip #2: Do one step at a time and keep it simple. GTD is a simple thing, but reading about it and implementing it is not the same. Give yourself the time to learn and don't overwhelm yourself.

Another common problem is the thing with the weekly review. I am aware of that because I had it myself. I put in on my calendar, I wrote reminders and I read numerous blogs about the importance of the review process, but I just never really did it. I talked to my friends who are also familiar with GTD, but their response was even more frustrating: No, I don't do it and I don't think it's really that important.

It was about that time I also discovered my enthusiasm for Steven Covey, especially The 7 Habits of Highly Effective People and First Things First. The latter one also contains great productivity and time management principles, but instead of starting at the profound organizing level, it helps you to get a new perspective of your life and focus on those things that are really important for you. One of these principles is "Plan Weekly - Adjust Daily". While reading that particular chapter, it suddenly hit me: Why do you always try to do your "Weekly Review" Friday afternoon? Why perform such an important control task when you are exhausted from the week, tired, looking forward to the weekend, hungry and already mentally out of the office?
So I decided to merge my "Weekly Review" with my "Weekly Planning" appointment, Monday Mornings at 10:00 AM. It worked great. I was able to review all the stuff I had and hadn't done in the last week with maximum energy and because of that, it usually didn't take longer than 10 minutes. It also helped me to plan my upcoming week more accurately by defining new projects and actions.

Driven by the success of my idea, I even decided to create a recurring breakfast meeting with all executives at netzkern before my personal Weekly Review and Planning session, every Monday at 9:15 AM. This so called Central Operations meeting still exists and it is a great establishment. So this brings us to our first tip...

Tip #3: Do the Weekly Review. It's not only important, it's critical for the successful implementation of GTD, especially in corporate environments. In the beginning, you may be able to manage a smaller amount of projects without it, but the earlier you start to review your stuff, the better. Set aside 15 to 30 minutes per week and make sure that you're neither tired nor have something "more important" to do at that moment. Use the Weekly Review to plan your upcoming week as well and start right away afterwards with the most important thing on your updated list, ideally one that helps you to conquer one of your "Big Rocks".

As I mentioned earlier, there is more to GTD than just the Mastering Work Flow Model. It is more than just projects and actions, although most people might perceive it that way. GTD also includes two higher level models that are easily forgotten, yet they are even much more powerful.

The first one is called the Natural Planning Model and is intended to help you with the management of all kind of projects. David often refers to projects as "everything that needs more than one action to complete". The problem with that sentence is that because of that statement, a lot of people tend to think of projects primarily as "containers" for actions. But in GTD, projects are much more than that, especially when they do not have a clear outcome or already defined next actions.

I will not repeat the Natural Planning Model here, but I want to make sure that everybody understands how important this part of GTD really is. Following its five step process, Purpose and Principles - Vision - Brainstorming - Organizing - Next Actions, is a great way to conquer your own thoughts and I do it every day.

Tip #4: Don't stop halfway between, there is much more to GTD than the Mastering Work Flow Model. Study and implement the Natural Planning Model and the Horizons of Focus Model for maximum effectiveness and efficiency in your day-to-day productivity work.

Alright, these are my most important tips so far. I could go on a while longer, but instead of just telling you about them, I find it more important that everyone experiences certain parts of the learning process himself. In the end, I really recommend that you read GTD at least twice and maybe revisit it every couple of years. It gets better every time...

Everyone knows, I’m a pretty “connected guy”. I have several Notebooks, Netbooks, PDA’s, my iPhone, my HTC smartphone and plenty of other gadgets that are usually constantly connected to the internet. There is practically no IM service and no community I’m not registered at. Although this perception is largely correct, several years ago I discovered that while these communication services can make your life a lot easier and they help you to stay in touch with your people, they are usually a real productivity killer and a constant distraction.

Let’s be honest, 95% of all IM conversations, community messages and even most of your emails fall in one of two categories: They are pretty meaningless, if not stupid -or- they contain important information that really can wait. But yet every single “connection attempt” pops up, “plings” and demands your immediate attention. That’s true for Outlook, Windows Live Messenger, ICQ and even your browser showing Google Talk or Facebook. Everything screams “here, here, read me” and implies “quick, write something back, it doesn’t matter what it is”. So most people just “take a short look” again and again and they wonder why they don’t get anything done…

So how should we deal with that? Although communication is a great thing, the first step is to disable everything… Yep, everything, just shut it off. Don’t let your IM applications connect to the internet automatically on windows startup, do not open Facebook first thing in the morning and leave it running in the back and don’t reply to every SMS right after it has been delivered to your phone. Turn off any annoying “you have new mail” notification on your desktop and for god’s sake, do not update constantly all twenty-something services with meaningless status messages like “Having fun with…”, "Just decided to..." and "Thinking about...". Trust me, nobody really cares, except those people that have the same addiction to those things you might currently experience.

The second step is to define a communication pattern that does not get in the way of your work and your life. The truth is, having all your communication service enabled all day long is like having 10 people in your office and being constantly interrupted because every 10 minutes someone asks you something. It is really the same.

My personal communication pattern is pretty strict. I have three regular times during the day on which I enable all my communication services: When I start my day at the office, right after lunch and when I leave the office. During those periods which usually last about 30 minutes, I open all “channels” to the world, check my email and chat a little with colleagues. I read through my SMS, reply to my friends and return calls. I update my twitter status, take a look at Facebook and check for new messages at studiVZ. After 30 minutes my timer interrupts me, I finish my conversations and disable all tools, sites and features completely.

Of course, this is my way and it’s only working pretty great for me. You have to find your own way, but I strongly encourage you to work more disconnected and don’t get trapped in unimportant, apparently urgent but surely mindless things. Turn off the noise in your life.

I followed the ASP.NET MVC development since it first appearance on the ALT.NET conference in the end of 2007 and as you can see from various posts, I'm a big fan of it (ASP.NET Web Development - some thoughts... -and- A Conceptual Alignment of Sitecore, ASP.NET WebForms, ASP.NET MVC and the Microsoft AJAX Client-side Library). From an architectural point of view as well as from a lead developer point of view, MVC is and has always been the best server-side pattern for web applications (and as long-standing Java EE and Rails developer I'm allowed to say that). In combination with event-driven GUI frameworks on the client side like ASP.NET AJAX, JQuery and Silverlight it should always be the first choice for every serious web developer in 2009. It is also an important building block of our Reference Application Architecture and Microsofts Application Architecture Guidance 2.0, especially in combination with WCF.

The question is now: When will Sitecore finally adopt ASP.NET MVC as supported framework? Clearly there is no doubt that the development model of ASP.NET MVC and Sitecore are even closer aligned than the development model of ASP.NET WebForms and Sitecore and that there are enormous synergistic effects by integrating both frameworks. This is true for various parts of the system, especially routing and rendering (where XSLT was - beside other reasons - chosen because "inline code" was a bad thing back than). I personally expect not only basic support for ASP.NET MVC but even a complete switch to use it as primary base framework instead of ASP.NET WebForms over the next couple of months, maybe a year.