Thursday, November 18, 2010

Development Database Setup

Hello Everyone,

If you're working on an application of any significant complexity, one of the tasks developers must deal with are database set up scripts.  One trend that I've noticed over the years is that projects where developers maintain a set of scripts that setup the databases from scratch tend to go more smoothly.  To be clear, this process includes tearing down any existing database, rebuilding the database to the most recent release version, loading good test data, and finally running all change scripts for the next release.  Though those last two steps can be reversible depending on the project.

Example the first.  I have a system.  Yaay!  I need to make a change to the database.  It's a relatively complex change involving creating a new table including keys and indexes, copying data from an existing table to the new one, and removing columns on the old table. Booo!  After I make all these changes and have the scripts in place, I need to test that I did everything correctly.  On the plus side, I have a thorough rebuild process in place.  I can click a button and the whole system is rebuilt, I find problems and fix them myself and I know without bothering any other developers that the scripts are good.  Yaay!

Example the second.  I have a system.  Yaay!  I need to make a change to the database.  It's a relatively complex... yada yada yada... I need to test that I did everything correctly  Booo! Unfortunately, databases are not torn down and rebuilt as part of the setup process, they are only modified.  What do I do to easily test my script.  I can undo the changes by hand and rerun the scripts to make sure, or I can check in the changes, hope for the best and let the next developer find any problems and repeat the cycle with yet another developer.  Which one do you think ends up happening more often?  Booo!

This may not be a huge issue that comes up a lot, but when I see emails going out a few times a month with issues lasting a couple of days because things are done the latter, it reminds me of how many things that developers already need to keep track of and this is just another disturbance.  In the end the amount of work to make the changes is the same, but having a complete rebuild is a way to minimize disturbances and keep developers developing.  

If starting a new project, I highly recommend the complete rebuild approach because the hardest possible task would be overcoming momentum and switching from one process to the other.

Thursday, June 17, 2010

IQueryable and Repositories

One approach for coding repositories that I have seen a couple of times recently is that of exposing IQueryable collections through the repository interfaces. On the one hand this allows an extremely convenient way of exposing a flexible data source to your business layer or controllers. However, it also breaks the Single Responsibility Principle thus creating less reusable and testable solutions.

By allowing your business layer to define queries within its own methods, those methods are taking on two responsibilities, performing data access and performing logic on the results of that data operation. It's all well and good to complain about not hitting some high ideal, but what affect does this have in the real world?

First, if you happen to reuse the same data query in multiple parts of your application, you would have multiple places that you would then need to make changes and test. DRY this isn't.

Second is the testability of this code. In the terms of this afore mentioned white paper, you would need to write your tests for your business layer or controller in such a way that the state is correct as well as the interactions, for each method. As the paper says, testing the state aspect requires a larger set of data. Why would you want to write that extra code for every method in your business layer / controller that uses the same query? If you don't test each one, how do you know they work? How do you know when they break?

The simple solution is to keep your queries in the repositories behind the repository interfaces. Test the state of each one thoroughly and then you only need to test the interactions in your business layer. Yes it requires an extra function call in your business layer, another method in your repository interface and another function declaration in the repository. And while that might be a lot of words, it's only four or five more lines of code for each query (including brackets on their own line in C#.) And if you reuse your queries, you get those extra lines back anyway.

I will certainly concede that no one paradigm fits all of the software problems out there. Maybe what I suggest is overkill. Maybe it's another example of a programmer too set in his ways. But by exposing IQueryable collections through the repository interface, we're ignoring a number of the principles of coding that have been making our lives easier and our code better for a while now.

After a second read through of the article, K. specifically states that enumerables could be returned instead of queryables. Not that this is the only publication I have read where queryables were suggested. Let's just say that now all four of my regular readers know where I stand on the subject.


This white paper by K. Scott Allen is a great read and was my formal introduction to the Unit-Of-Work concept.

For a while, the recommended approach for providing data contexts was to create the context and store it in a session of some sort. Each call to your server would provide a single data context for all of the logic needed to render a new page or perform some sort of data operation.

UnitOfWork might not be a name that explains exactly what the object is (a data context container). But, it does describe what the object does. It frames a the data context within a single logical unit of work. The distinction being that it may take many logical routines to render a single web page.

One advantage to the technique is that you are committing data changes at the end of each logical routine. This means that many of the data changes for that routine can all be committed at once using fewer network and database resources over a shorter period of time.

Another advantage is that you are holding the data necessary for each operation in memory for only as long as it is needed. When the operation is done and the data is likely no longer needed, the memory is flushed and reused for other operations.

Lastly (that I'll mention anyway), it separates your data context scope from session scope. The session will likely be different in web, windows and service applications. This means that your session across different applications may comprise a different number of business methods which can have unforeseen affects on performance and functionality. Instead, you can tie the scope of your data context to each business method as those make good candidates for a unit of work. This in turn means that your routines will perform more predictably across different types of applications.

Thursday, June 10, 2010

Backgrounds, Google? Really?

If you got to this post, you're probably wondering how to turn off Google backgrounds. I for one am pretty sure that if there isn't already a setting, there will be one shortly. I can't be the only person that want's my plain white, pleasant and unobtrusive background back. I'm also far too lazy to go look for that setting given that a link to it doesn't exist on the front page.

That being the case, I created some 800 x 600 solid color backgrounds and uploaded them to Picasa. 800 x 600 by the way is the minimum resolution that can be used. Yes, I do like the fact that Google is hosting my solution to my Google problem.
Download early, download often. Here's the links:
I recommend the dark gray though the white does give a good 'White Album' feel. Obviously, you can also easily create your own image using MS Paint or any simple image editor if you prefer, say, color. Click on the 'Change background image' link in the lower left hand of the Google front page and upload the file from your computer.

Fine, this didn't take much imagination or skill. But I was so dumbstruck when I saw the page, it was all I could do to listen to various suggestions from coworkers.

In retrospect, this may have just been really good marketing. Google did seem to have a fix in place fairly quickly. I'm sure ours wasn't the only office where the new background had everyone talking for a few minutes. And who knows how many bloggers out there were duped into creating buzz about it.


Wednesday, April 14, 2010

Milyli Update

They say that in order for a blog to be successful, you need to write about what your audience cares about. Since I've started writing, I've only had one person ask me to give updates on how our little company / experiment is going. Then again, I've only had one person leave comments about the coding tips I post here so I guess either topic is as valid as the other at the moment. Which one will I talk about today?

That's right, it's company news. Our little company has grown to 6 whole, full time employees. That's right pretty soon, they'll out number us founders and then who knows what variants of heaven or hell they'll unleash on us. Whichever it is, I'm sure things are just going to keep being interesting for a while.

We're also considering leasing out the rest of the floor in the building we're currently in. It's a big step, but we've realized that there really is no substitute for closed off separated meeting places and work areas. Keep the loud side loud and the quiet side quite. We currently lack that separation and every new familyli member brings that fact into further relief.

We're in the home stretch of a nice long project as well. I can't talk much about it, which is probably why I haven't posted here much. Oh the mystery is killing you. I know. Hopefully though, I'll soon be able to take up posting on all the various interests I have, thus ensuring that my blog continues to wallow in obscurity.

Until then...

Wednesday, February 10, 2010

Google Applications

I logged into GMail today and got an invitation to activate Buzz, so I did. I even posted a quick little message about my immediate impressions. And a fine bit of useless drivel it turned out to be if I do say so myself.

There was one thought that stuck me as interesting though. The realization cemented in my mind that there is bit of a pattern Google has to creating their solutions. They build or sometimes buy software that solves specific and difficult problems in easy to use ways that they provide for free. And with advertising much less obtrusive than other applications. They have many such pieces of software at this point. Now, they are finding the best possible ways to integrate them as elegantly as thy can. Networking applications to solve social networking, if you will.

I know, I'm late in realizing this. But I never accused myself of being overly watchful of other companies' strategies. Many people will argue that the one point of entry provided by Facebook makes social networking more convenient. My problem is that the utilities provided there just don't provide the functionality I want in photo management, calendars, email and event planning.

Thursday, February 4, 2010

MVVC, Silverlight and Staying Grounded

I was reading some Visual Studio Magazine today and there was an article on MVVC gushing over how great it is. I know it's not exactly new. I also know that the pattern, when combined with the trend toward data binding is a powerful combination. But it's surprising that many 'architects' apparently swear by this solution and insist on writing whole applications of every type without deviation.

There are many in-the-trenches sorts that believe that while it is useful in certain situations, it actually causes more work and creates less maintainable code much of the time as well. I ran across many differing opinions all in the same day. Probably the most sane of which were saying that it's a good architecture for mid size, data processing applications; too much for small tools and not flexible enough for large, interaction rich utilities.

As with all layers of an application, there appears to be different problems to solve in the UI. The different paradigms are good at specific problems of certain sizes, but no one is best suited for all tasks of every caliber. From messaging between disparate parts of a large application, to showing data in a small application, to interacting with complex UI's and business logic, it seems like you need to choose the solution(s) that will best fit the problem(s) at hand. This might not help you out much in choosing the best model to use on your current project, but it may be ammunition against those of the one-solution-fits-all mind set.

Maybe it all comes back to the fact that a text box and button with some in line SQL in the code behind is a perfectly acceptable way to modify some simple data under certain circumstances...

I also read up a bunch on Silverlight 4 today. I guess there's a slew of new technology being released along with .NET 4 and VS 2010. The general summary is that it makes Silverlight a viable line-of-business application platform. But, there seems to be a lot of bells and whistles that are more web 2.0'ish rather than business requirements. Maybe there's some overlap there, or maybe there will be when businesses are ready for it. That being said, it's probably worth looking into the specifics on the WCF changes as those were the hardest challenges that I remember having to overcome with the platform in a business application. The whole printing thing is probably pretty nice too.

Friday, January 22, 2010

Milyli Update

It would seem it's been a while since my last post. As Google appears to be the largest source of my traffic, I don't think there's all that many loyal followers on RSS that need apologizin' to. And I wouldn't want to assume that my words are so engrossing that the few regular readers I do have even realized I hadn't written.

So just read this thing.

The past few months Milyli has been growing. In addition to the three founders we have hired our first full time employee, we are on the look out for others, we have two part time employees and we just moved into a real office. Fine, it's a big loft space that we use as an office, but it has desks, meeting tables, shelves and colorful network wire running every which way. It's more of an office than the house that we moved out of.

Probably one of the best articles I have read that sums up the experience is this one by Paul Graham. I direct you in particular to the section at the end sub titled The Super Pattern. It's worth a read, but the gist of the article is that so many people have written about the process of starting up a business that there should really be no surprises about what experiences you're in for.

What you probably will be surprised about however is the intensity of those experiences. Yes, you will have to do all of the things that everybody always says you need to in order to start up a business. But the peaks and valleys of this here roller coaster are far more extreme than I had anticipated.