As I noted earlier, I dropped my attempts to use a SoHo NAS product in favor of building a NAS from spare parts and a couple of extra items (an SFF case, SATA card and extra drives). The total cost of the DIY NAS was about $700, and it did involve some time for configuration. Cost-wise, if you are billing your time, it probably comes out to a wash to go DIY or buy the Netgear ReadyNAS NV. The ReadyNAS NV is a step above the products I tested in cost, features, and functionality.

However, I have found other uses for the new NAS that make the configuration effort worthwhile, at least for me. We have been running an installation of SugarCRM, but a little while ago we had to re-purpose the machine it was running on, and we had not taken the time to re-install SugarCRM anywhere else. Enter the DIY NAS, a full-featured Ubuntu 7.1 server. In a couple of hours, we were able to install and configure a base deployment of SugarCRM Community Edition. (If you have worked with a CRM system, you know that installation and configuration is just the beginning of the effort for building and maintaining an effective  system). I expect theSugarCRM system to be low traffic in the short term, so pulling double-duty as a file server and CRM application should be fine. Long term, we will need to port the CRM system to more powerful hardware, but for now we have added significant capability to our business at minimal cost. We may also set up the NAS as our local LDAP server in the office, though as a small business we are not yet at the point where domain services are a real necessity. 

For small business, the combination of ever-faster computer hardware and open source software provides a low-cost avenue to business automation that was once the province of huge corporations. Today, any mom-and-pop shop with some technical skills or a few thousand dollars for consulting services can deploy enterprise-class business automation solutions. In one sense, it signals a loss of competitive advantage for big enterprise, and it knocks down an argument for scale in business, which is the cost and complexity of such solutions. There are still lots of other reasons why scale matters, but it is interesting to see how IT evolution has become an equalizer for small businesses.

I have had several r eaders ask me for a copy of the compiled 64-bit Java Service Wrapper for Windows x86-64, so I am including a link to it. If you plan to download and use this code, please be warned, I did not compile the code, it was sent to me by a guy named Sandeep whose info I found on the Java Service Wrapper mailing list. Having said that, I have used his compile of the wrapper and it seems to work fine. It comes with no warranty and no guarantees that it will work for you. If you want it, here is the link:

64-bit Java Service Wrapper for Windows x86-64

If you need help with configuration, take a look at the Java Service Wrapper web site.

Apple just released a big patch bundle for OSX, 88 patches in all, 105 MB You can read about the full details in an article on the Register   I wonder if the "big bang" patch release system set up by Apple is related to the release cycles of open source projects like Apache, OpenSSH, and ClamAV? 

 I like the idea of releasing fewer patches with more frequency, but I don't know if Apple has that luxury. Ubuntu has a six month release cycle, and I see some packages missing a release on occasion if their project teams are not on the same schedule. I wonder if it would be possible to get major open source vendors on a common release schedule? Maybe they try, I am going to ask someone from an Ubuntu team and see what they say. If you know the answer, drop me a comment, I would love to know.

One thing I know for sure is that Ubuntu is big, REALLY big, 17015 packages and counting at the moment. Even if only ten percent of those packages are being actively worked, that represents an enormous logistical challenge to get all the project team to hit specific targets. Ubuntu contributors are using collaboration software, especially emergent systems like wikis and blogs, which are great for software projects. I'd love to see someone attempt to build metrics around the use of these collaborative tools in the development of Ubuntu, because it is such a big open source project and it provides both transparency to make it easy to study and enough size to provide both a really good sample of data and scaling evidence from projects large and small. 

My first week using Ubuntu on the desktop has, overall, been a very good experience. Over the last ten years, I have tried several times to switch from Windows to Linux using a variety of distros and strategies. In the past, there have always been deal-breakers - critical applications that would not work on Linux, or that were sufficiently hard enough to deal with that I felt the tradeoff was just not worth the trouble.

This time around, I am happy to report, much has changed. Ubuntu is, in my estimation, now a perfectly acceptable alternative desktop OS for certain situations.  It is not (nor do I expect it to be) a 100% replacement for Windows or Mac OS X. As a friend in the Linux world once said, it is really hard to compete with a company that spends billions of dollars a year on R & D. Still, for all its limitations (and there are limitations), desktop Linux is now a workable everyday solution.

Hardware Support

The major challenge for Linux on x86 hardware is, as it has been for years, providing good hardware support for the vast universe of x86 hardware available in the market. On this front, Linux has made huge gains in the last five years, both from a distro standpoint (see Ubuntu's restricted driver manager) and from a vendor standpoint. Hardware vendors are no longer totally blind to the Linux phenomenon. In fact, I have been able to find usable drivers for all of the hardware I have in current use, including the printing and scanning functions of my Brother MFC-9700. 

Nevertheless, driver support still lags behind Windows. ATI display drivers have become notorious for their spotty Linux compatibility (are you listening, AMD?) . Linux does not support motherboard-base RAID systems (called fake-RAID by the Liunx crowd because the on-board RAID chipsets use motherboard resources for actual operation). 

Even in situations where drivers are available, downloading, installing, and configuring them is not the job of a novice. It took me several hours to properly configure my MFC-9700 so I could both print and scan with it. Let's just say that unless you are comfortable in a Linux command shell, adding this kind of functionality to your system is not in the cards right now.

If you are looking to migrate away from Windows, either individually or as a strategic move for your business, take the time to investigate vendors who supply hardware that is certified (and supported) on Linux. Dell, with its certified Ubuntu offering, is now the big dog  on the block in the desktop Linux market. Dell also sells systems with Red Hat Enterprise Linux, so they have experience in this market, and realistically, Dell is only going to offer products that they are 100% sure they can support. 

If you are more adventerous, you can install Ubuntu on almost anything, and hardware support for add-on cards and peripherals is very good, but be prepared to spend a lot of time browsing the Ubuntu forums, blog sites, and vendor sites, and keep a Terminal window open, because you are going to need it.

Functionality Gap

I have, in the last week, added support for almost everything in my system, although there is a qualification to that statement. I switched from an Intel-based system powered by an Asus P5W DH Deluxe tweaker motherboard and an ATI x1900 PCI-E video card because of reported (and experienced) compatibility issues with Linux. My new hardware profile is an Asus MN2-MX motherboard with an AMD CPU and a Geforce 7200GS series video card.

So what is working now? First, I added support for my Bose Companion 3 USB speaker system. If you haven't heard these Bose sound systems, get to your local electronics store (Frys has them on display) and take a listen. The sound quality beats every other PC-based sound system out there, hands down. The system recognized the presence of USB audio right away, but I couldn't get any sound. After digging through the Ubuntu forums and some other sites, I figured out that I had to disable the on-board sound support from the Ubuntu config files and set the USB audio as the default audio system. Once I did that (and it sounds easier than it was), everything started working immediately.

 I also managed to get my 160GB iPod Classic mounted  (listening to The Clash - London Calling right now), although it was little trickier than I thought from my first look at the blogs. Apple changed the iPod database recently enough that older software meant for older iPods was not working for my Classic. A couple of hours later, I got the necessary software installed, but I still could not see my iPod. A quick check of hardware using the dmesg command revealed that my iPod was attached to the system as /dev/sdc1, but it was not mounted. Mounting a drive is easy, though:

#sudo mount /dev/sdc1 /media/ipod

After mounting the volume, my iPod appeared and Amarok (my music player of choice because it can use MySQL/Postgres to store its database) was able to import the database from my iPod and play all of my music.

I have not re-visited the problem of synching with Windows Mobile. I might just wait a little while and see how Windows Mobile support shapes up in Hardy Heron (Ubuntu 8.04). 

Somewhat unexpectedly, I was able to add full support for my dual monitor setup, including installing and configuring the awesome Compiz Fusion UI for Gnome.  I probably spent an entire day on this solution, though, so consider your time factor before you make the move to install it. Compiz Fusion is an Aero-like UI for Gnome that uses your system's 3D rendering engine to provide cool visual effects. Take a look on YouTube for some cool examples of what you can do with Compiz Fusion. 

Limitations

There are, alas, still applications that are not available for Linux. The most important missing application, purely for compatibilit issues with web sites, is Internet Explorer. My wife's primary business web site is an IE-only affair at the moment, so for her, Linux just isn't a realistic option right now.

Some network services may be complicated by the use of Linux in heterogenious environments. If you are looking at replacing a set of Windows desktops with Linux, do your homework first. Get a couple of machines set up and see how well they play with domain services, network shares, network printers, and the like.  Providing sufficient support for Linux desktops in an enterprise environment is going to be dependent on having certified hardware and software solutions (check out Dell) and a trained IT support staff that can solve issues like getting network printers and shares to work with these new systems. All in all, the support costs are probably still going to outweigh the benefits of throwing off the Windows licensing model for desktops, but that's a judgement that IT execs will need to make for themselves.

Last month I decided to install the Vista Transformation Pack rather than do anything drastic like re-imaging my desktop. Being a techie at heart, I couldn't resist the urge to continue installing and experimenting with new stuff. As a result, I finished the month of March with my desktop in a dangerously unstable state. 

Last weekend, I finally decided that the pain of re-imaging was going to be lower than the pain of having an unstable system. Rather than re-image on Windows, I decided to wipe the system and install Ubuntu 7.10 Gutsy Gibbon.  (I still have Windows on my laptop, but I really wanted to give Ubuntu a try on the desktop, especially since the hosting environments at Emergent Path are primarily using Ubuntu today). 

I stated with a bold (and ultimately fruitless) decision to install Ubuntu 64-bit. The install itself went fine. I am amazed at how quickly Ubuntu installs compared to Windows XP.  I got a few basic apps set up, and then I started hitting roadblocks to 64-bit at every turn. There is no Flash Player for 64-bit Linux. There is no Flex Builder for 64-bit Linux. There are 32-bit versions, but installing them is hit or miss (more miss than hit from the blogs and forums that I read on the subject). And that's just Adobe, which actually has good support for Linux. Most desktop application vendors that I have experience with have no Linux support at all.

After a day of banging my head on the 64-bit wall, I decided to dump the install and go to 32-bit. Re-imaging with Ubuntu is a breeze- again I am amazed at how fast the OS installs. This time around, almost everything worked right out of the box. Firefox works, the Flash Player works, there is even a FlexBuilder 3 Alpha for 32-bit Linux that I was able to install successfully.  Here is a list of the main programs I am using:

• Firefox
• Flash
• FlexBuilder
• ColdFusion Server 8
• Eclipse
• Pidgin IM client
• Putty terminal client
• OpenOffice
•  Evolution mail client
• PDF Reader

I use lots of other programs, but those few are the programs that I use day in and day out. The functionality on Ubuntu is every bit as good as Windows. So what about the downside?

Well, there are a few potholes on the road to Ubuntu heaven, it seems. First, I have yet to successfully sync my Verizon vx6800 (Windows Mobile 6) phone with Evolution. There are lots and lots of blogs and Ubuntu forum posts on the subject, and the only common thread in all those posts is what a complicated PITA it is to try and sync Windows Mobile with Linux. I managed to get a Bluetooth connection working between my desktop and phone, but only insofar as the devices could see each other, they could not actually share data or do anything else. I'll re-visit that subject next month when the next release of Ubuntu, Hardy Heron, debuts.

I have also yet to do anything with my iPod, although the posts that I have read on the subject are far more encouraging than the posts about Window Mobile.

Overall, I am very happy with the switch, primarily because it is forcing me to expand my knowledge of Linux to include the latest in the Ubuntu world. Though some people may be loathe to admit it, Windows XP is the gold standard in the OS world. Sure, OS X is easier to use in some respects, but the differences are minor. Sure, Windows leaks like a sieve in terms of security, but that's a result of being the biggest target of opportunity more than anything else.  The acid test is whether you can switch from one OS to another without losing any applications or major functionality. I am at about 95% with Linux right now- not perfect, but definitely acceptable for day to day use.

I recently installed Ubuntu 7.10 (server x64) on a server I have. I added Gnome desktop for ease of use dealing with administrative tasks (no one will ever convince me that remembering commands like "sudo chown -R user:group /var/whatever" beats have a nice friendly GUI, but I digress).

I have been very pleased with Ubuntu 7.10 with Gnome. It can still intimidate the uninitiated (s, but it seems that someone has finally got the Linux desktop right. Furthermore, I am stunned at how fast some operations are compared to Windows XP (although there are plenty of areas where Gnome still gets poor grades compared to Windows).

I am seriously considering wiping my desktop and dropping in Ubuntu, then adding a virtual machine with XP for any necessary stuff. I haven't dug deep enough to consider all the implications of the move, and I don't really have the time right now to deal with it, but I find myself frustrated with Windows bloat and looking for an alternative. I'm not a Mac guy (and I'm not replacing my hardware anyway) so Linux seems like the best and only option at the moment.

The biggest obstacle I see is driver support fr my ATI video card. The saga of Linux drivers for ATI has been well documented; it suffices to say that dual monitor support and high resolution, high performance, stable drivers are not exactly a sure bet with ATI cards. Is it worth it? Probably not right now, but that's more because of the driver issue and my general lack of free time than issues with Linux. Maybe I trade in my ATI card for some nVidia gear. 

Anyone else using Linux on the desktop? I'd love to hear how it is working out, especially with multi-monitor support. I'm not a gamer (at least not in the last few years), but I need high-res and good performance from a multi-monitor setup.  

More generally, I am wondering about the possibility of using Linux as a standard developer desktop for the organization, but if I can't make it work for my own use, I'm not going to force it on anyone else. 

In my previous post, I outlined my twist on the use of WITH, ROW_COUNT() and OVER to generate paged recordsets in SQL Server 2005. After creating the solution, I decided to run some performance tests to see how my method performed.

Test Bed

Client machine (custom-built desktop)

• Intel E6400 Core2 Duo
• 2 GB RAM
• 10K RPM Raptor system disk
• SQL Management Studio Express

Server (HP TC4400 laptop)

• Intel T7400 Core2 Duo
• 2 GB RAM
• 80 GB 7200 RPM system disk
• SQL Server 2005

I added 130,000 records to the content repository cmObjects table and queried to pull an arbitrary set of 20 records from within the full dataset, ordered by the objectname field, which is an indexed nvarchar field. Below is the query, followed by the client statistics captured from SQL Management Studio.

First query

•  Total Execution Time: 12812 ms

Second query

• Total Execution Time: 109 ms

 Third query (new page range)

• Total Execution Time:  125 ms

Granting that my laptop is not an ideal test bed for a server application, I am nonetheless satisfied with the results of the test.

I decided to up the ante and see how the technique would work on millions of rows instead of a hundred thousand rows. I created a new query to pull against the content properties repository table cmProperty. The talbe has roughly two million rows of data in it.  Below are the query and the client statistics:

First query

•  Total Execution Time: 3828 ms

Second query

• Total Execution Time: 3843 ms

 Third query (new page range)

• Total Execution Time:  7562 ms

Not great numbers, but perfectly acceptable given the server test bed. This query isn't a realistic use case, though, as there is typically no reason to pull an arbitrary set of records from the cmProperty table. 

My current plan calls for migration of 500,000 records into this content system, which comes out to 500,000 rows in the cmObject table and 8 million rows in the cmProperty table. I will run further performance tests with the full dataset and post my findings.

Recently I have been working on a content management system that I am developing, and I came across a performance challenge with large content sets. A typical query against a very large dataset might return tens of thousands, hundreds of thousands, or even millions of records from the database, but typically only a single page of those records will be displayed at any given time. The rest of the returned data represents wasted cycles and bandwidth on both the database and application servers.

I spent some time looking around for solutions to this problem, and I found several good solutions to return only one page of records from the database, but these solutions had limitations, either in that they required the use of stored procedures, or that they required making a second database call to get the total number of records matching the query. In particular, Ben Nadel posted a solution that generated some good discussion and links to other posts.

My solution, for SQL Server 2005, takes a slightly different approach than others I have seen. I use the WITH, ROW_NUMBER(), and OVER features of SQL Server 2005 to generate the paged recordset, and I use a UNION query to tack on an extra row with the total records matching the query. I haven't tried this out on other platforms, but the same strategy should work for any database platform that supports some form of paged recordset retrieval- MySQL, Oracle, and Postgres should all work.  

Here is a query of all the records in the object repository, but returning only 20 at a time. 

WITH cmObjects
AS
(
    SELECT         ROW_NUMBER() OVER (ORDER BY objectname) AS Row, o.*, t.TypeName, null as total
    FROM        cmObject o
    JOIN        cmType t on o.TypeID = t.TypeID
)
SELECT    *
FROM    cmObjects 
WHERE    Row BETWEEN 1 AND 20           

UNION

SELECT         null,null,null,null,null,null,null,null,null,null,null,null,null, count(*) as total
FROM        cmObject o
JOIN        cmType t on o.TypeID = t.TypeID

I have worked with XMail, an open-source mail server, for a number of years. XMail currently powers the mail services for emergentpath.com and a few other domains that the orgnization hosts. XMail comes with a very handy command line administration tool that allows you to administer the server from local or remote systems. Like all command line tools, it can be somewhat cryptic, and it certainly doesn't lend itself to automation. 

Based on the needs of my own organization, I decided to write a ColdFusion gateway to the XMail command line tool. The gateway is implemented as a ColdFusion component and includes all of the commands that the command line tool accepts. A few of the advanced features, such as pushing configuration files to the server, are not yet fully implemented, but most everything else is coded, if not entirely tested. If you would like to take a look, download the beta code.  

If time permits, we might put some R & D effort behind using this component as the front end of a full-fledged Xmail admin utility written in Flex.

I have put some additional thought into the troubles facing PHP, and what occurred to me is that we have seen, in the last ten years, a major shift in the way innovation in programming languages make it to the market.

In the past, new languages simply superceded old ones. Older technologies became relegated to supporting old systems, while new systems were created using new technologies. High level contructs replaced low level ones.

That model has changed. Software groups started to release new versions of programming languages and tools on a regular basis. Did it happen in response to customer needs? Because software companies realized that the languages and tools represented core intellectual property that they wanted to protect rather than just productivity tools? Did the rising complexity of software fuel a desire by business to stay on the same platform as long as possible? Those questions are outside the scope of my post, but I think everyone can speculate about the reasons for this shift.

New languages are no longer simply replacing older ones. New languages are occupying new space (and some existing space). The effect is an increasing fragmentation around the edges of the software industry. Call it the Tower of Babel effect.  At the same time, the industry is working extremely hard to standardize around a handful of general purpose technologies and languages like XML and Javascript, and commercial entities are lining up behind two major technologies- Java and .NET. That argument is a bit of an oversimplification, as large enterprises use a variety of technologies, but at a high level that is what is happening. In the industry, we have a largely unified center of software languages, and a totally fractured periphery. 

Because of this state of affairs, sponsorship by a major industry enterprise or group has become a necessity for mainstream programming languages. That does not mean that everything outside the corporate world is doomed. Open source languages like Perl, Python and Ruby can still thrive if there is enough organized sponsorship by universities, governments, and other non-commercial entities. But as languages become more complex and tools become more sophisticated, it will become more and more difficult for these languages to compete with commercial offerings from companies that spend billions of dollars per year on organized R & D.

In the competition for mindshare among software engineers and technology managers, the question for PHP and all other programming languages outside the center is whether that language continues to appeal to new audiences in an ever more crowded periphery. For languages in the center, the question is how to continue to provide enough innovation to keep the language at the center and not have it relegated to the periphery.

I thought about breaking down programming languages by industry, platform, and sector to uncover centers and peripheries in individual market segments, but that would take more time than I have to dedicate. If anyone knows someone looking for a research topic for a graduate degree, feel free to tell them to run with the idea. I'd love to see how things break down.