Microsoft submits virtualization drivers to Linux kernel

In a small but great piece of news for cooperation between the world's largest software comany and the Linux community, Microsoft has submitted drivers for the Linux kernel that will help Linux virtual machines running on top of a Windows Hyper-V host to run more efficiently.

Kudos to Microsoft for the move.

MySQL Workbench for Linux

Since I use Ubuntu Linux on my desktop for day to day development, I am always looking for good Linux desktop software technical tools to add to my toolkit. One of the categories that has lagged behind Windows is in database modeling and management. There are some freeware tools, but nothing that provides the features I need.

Fortunately, MySQL has released the MySQL Workbench for Windows, MacOS, and Linux.  There are community (read: free) and commercial editions of the product available. The community edition provides reverse and forward engineering of a model to a MySQL database and has a "sync with model" feature that is great. I have just finished porting a model of the Colony application platform into MySQL Workbench. Kudos to the MySQL team for a good release.

Bug in mysqld_safe wrapper

There is a bug in the latest release of mysqld_safe on Ubuntu that is causing cpu times to spike to 100% on the mysqld_safe wrapper. The bug has been documented on Launchpad, and a fix has been committed to the Jaunty (9.04) codebase, so it looks like the bug will be squashed soon. Still, this bug highlights the interconnected nature of software development and our reliance on the correct functionality of many significant components to modern server-based appplications.

 

Visual Artifacts in Ubuntu 9.04 VMs

I use a Windows XP virtual machine on my Ubuntu desktop whenever I need to access Windows software like Internet Explorer. I have found it to be a very useful tool in my toolkit, and overall I have been very happy with the functionality (which is, by the way, essentially the same thing as Windows 7 will offer with XP Mode).

Lately, however, I have noticed a lot of visual artifacts in my XP virtual machine. I can't recall seeing the problem under Ubuntu 8.10, but it might have been carried over, I don't know. At any rate, the most annoying thing about the problem is not just the visual artifacts (which is quite annoying), but the fact that the visual artifacts seem to have some substantial form, because mouse functionality (like clicking a link) doesn't work when one of the visual artifacts is in the way. 

Take a look at this section of a Google search result web page to see the problem:

VM artifacts

This one bug is probably my current pet peeve with Ubuntu 9.04, which overall is an excellent OS. Hopefully the bug will be resolved soon.To be fair, the problem may not be with the virtualization system at all. It could be a problem with the Virtual Machine Manager, it could be a display driver problem with my nVidia display card, or something completely unrelated. Still, it worked just fine until recently, so I expect that the Ubuntu team will figure it out before too long.

Upgrading to Ubuntu 9.04 - Go Live

Last night we upgraded our production systems to Ubuntu 9.04. SO far everything looks very good. I am waiting a few days to see how things continue to run before declaring victory, but so far everything looks great. Next on our infrastructure roadmap is setting up a cloud environment for use with some of our systems. 

Ubuntu Flash sound issues resolved

I finally seem to have resolved all issues related to Flash sound on Ubuntu 8.10. I found a very helpful guide on the Ubuntu Forums that provides instructions on fixing issues related to Pulse Audio on Ubuntu. The author points out that you should absolutely not remove Pulse Audio, as it is now part of the Ubuntu core and removing it may cause upgrade problems down the road (as in next month).

Update - Flash sound in Firefox with PulseAudio

Today Flash in Firefox started working with sound again. I have not change dmy configuration manually, though I did load at least a couple of Ubuntu updates over the weekend. It still seems to monopolize pulseaudio (similar to the problem with Skype, which is why I am using onboard sound with Skype), so I am forced to kill Firefox to get Amarok to play music through pulseaudio, but that's OK for now.

Pulseaudio in Ubuntu 8.10

As of the last couple of releases, popular Linux distro Ubuntu has switched the default sound system for the desktop to the PulsAaudio Sound Server. PulseAudio offers the promise of unified, abstracted access to a computer's sound capabilities, and from that standpoint it is a huge advance over previous solutions. However, the move to PulseAudio has been accompanied by a huge amount of frustration by end users with compatibility problems, crashes, and various conflicts with onboard sound, USB sound, and Adobe Flash and AIR.

Recently, I have experienced a number of these issues myself. (See the related posts for background on my transition to using Ubuntu on my desktop). In November I installed Ubuntu 8.10, and I have had some sound issues ever since. In the last few days, I have spent some time debugging these issues on my system, and I wanted to share my challenges and solutions.

Sound Hardware

I have two separate sound systems - onboard sound on my motherboard and a Bose Companion 3 USB sound system. For this type of configuration, the sound troubleshooting guides I have read recommend deactivating the onboard sound and using USB sound only.I tried that for awhile, but I had an issue. The Bose system has a microphone jack on the volume control, but Ubuntu seems not to see it, it only sees the output device. I need a microphone, so I enabled the onboard sound for that purpose. I have a headset/mic combo attached to the onboard sound system.

Skype

If you work remotely or travel on a regular basis like me, Skpe is a great solution for keeping in touch with people. It offers chat, voice over IP (VOIP) telephony, even video-conferencing. The latest Ubuntu client for Skype works reasonably well, but it tends to seize control of the pulseaudio process and eliminate sound for all other applications. My solution to the problem is to enable onboard sound (by removing any /etc/modprobe.d/blacklist entries and enabling the onboard sound in BIOS), assign USB sound to the first sound source in ALSA:

/etc/modprobe.d/alsa-base

install sound-slot-0 /sbin/modprobe snd_usb_audio

and setting USB sound as the default output sink in pulseaudio:

/etc/pulse/default.pa

 .nofail
set-default-sink alsa_output.usb_device_5a7_1020_noserial_if0_sound_card_0_alsa_playback_0 

 Note that your exact settings will depend on your particular hardware.

After that, in Skype -> Options -> Sounds Devices I set my onboard sound hardware (mine is defined as HDA NVidia (hw:NVidia,0)) as the device for Sound In, Sound Out, and Ringing. The ring will sound in my headset, but that's OK for now. Maybe I'll get one of those Skype phone sets from eBay. This solution works fine and it is the only solution I have found for my system that gets Skype working without taking over the pulseaudio system.

Amarok

I use Amarok for my music library.  I have Amarok configured to use the xine engine in Settings -> configure Amarok -> Engine. This setup works well, though I had to experiment with the settings for PulseAudio (making USB sound the default output sink) to get it to default to USB sound.

Adobe Flash/AIR

I do a lot of development with Adobe tools - ColdFusion and Flex in particular. I use YouTube, etc., so sound support for Flash tends to be important for me. As of right now, I have no sound support in Flash. I have installed the latest Flash 10 plugin (10.0.22.87) from Adobe and followed lots of recommendations from various guides about configuring PulseAudio and troubleshooting Flash sound problems. My best guess right now is that activating the onboard sound has caused a problem with Flash sound support. Since I need Skype more than I need YouTube, I'll be keeping my current configuration for now, but it would be great if Flash could use the PulseAudio system without any problems. 

Even when Flash sound support was working (with a previous Flash plugin), sometimes Flash would have trouble after playing a video, and at that point PulseAudio would lose the USB sound device completely. Killing Firefox (and any AIR applications like Twhirl) and restarting pulseaudio (pulseaudio-k; pulseaudio -D) enabled PulseAudio to find the USB device again.

Miscellaneous Settings

Here are a few things I have set according to the varous guides:

/etc/asound.conf

pcm.pulse {
type pulse
}

ctl.pulse {
type pulse
}

 ~/.asoundrc:

 pcm.!default {
         type asym
         playback.pcm {
                 type plug
                 slave.pcm "hw:0,0"
         }
         capture.pcm {
                 type plug
                 slave.pcm "hw:0,0"
       }
}

I am not 100% sure exactly how all of these settings interact with each other, but it seems for the most part to be a successful setup. Based on the number of reported issues I have seen with Flash sound support, I am going to wait until Ubuntu 9.04 is released in April to see if the problems are resolved.

Ubuntu continues to improve as a desktop OS, but from these experiences, you can see it still has a ways to go to be considered as an easy alternative to OS X or Windows (although the huge number of issues with Vista has certainly provided Ubuntu an opportunity to show its capabilities).

Don't let my experiences discourage you from using Ubuntu. You can run the Ubuntu LiveCD and give Ubuntu a try on your hardware without actually having to wipe out your current OS. If you are seriously considering switching, my advice for now is to either buy a new computer that comes pre-installed with Ubuntu or do your homework and install Ubuntu with supported hardware. 

Spamassassin - automating sa-learn with IMAP folders

Among the useful things we have found for our clients is a methodology for building a learning spam filter using Spamassassin and a mail server that supports IMAP folders such as dovecot. Simply adding Spamassassin with a standard configuration on incoming mail on a mail server can dramatically decrease the amount of spam users receive, but it will not catch nearly all spam sent to the server.

The reason for the lack of complete filtering is clear. Spammers play a cat-and-mouse game with spam filters, always attempting to modify messages in such a way as to avoid filtering. As filters change, spammers experiment until they find ways through, and they change their tactics as each new technique is detected. 

Because of this uncertainty with spam, Emergent Path recommends that clients who maintain their own mail servers implement the Bayesian filtering engine in Spamassassin and automate the learning process through the sa-learn script.

sa-learn is a command-line program that can be called and passed various arguments to classify messages as either ham (real messages) or spam (fake messages). Because it is a command-line program, it can be easily automated using cron on Unix/Linux systems. We recommend running a daily process on the mail server (depending on volume of mail and number of mail servers involved) that scans user-classified spam and ham using sa-learn to train Spamassassin.

A sample script might look something like the script below. This is a simple example and not necessarily a final production script:

 

!#/bin/bash

sa-learn --showdots --no-sync --spam /var/mail/domains/*/*/Maildir/.MakeSpam/cur/
sa-learn --showdots --no-sync --ham /var/mail/domains/*/*/Maildir/.MakeHam/cur/

rm /var/mail/domains/*/*/Maildir/.MakeSpam/cur/*
rm /var/mail/domains/*/*/Maildir/.MakeHam/cur/*

In this example script, each user who wants to tag spam creates an IMAP folder in the root of their account called MakeSpam. (The example assume a typical mail directory structure of /var/mail/domains/<domain_name>/<account>/Maildir/ for the root location of each user's mail folders.) For any spam messages that got through filtering to the inbox, the user drags those messages to the MakeSpam folder and leaves them. When the script aboe runs (via cron on the server), the messages will be classified as spam and then deleted. Over time this system will help Spamassassin improve its hit rate on spam messages.

Manually marking messages as ham that have been previously classified as spam may vary slightly depending on your SpamAssassin configuration. If Spamassassin is set up move spam to a Junk or Spam folder and simply add a header to the message, the user can simply move the message to the MakeHam folder, and when the script runs it will identify those messages as ham (good) and remember those settings for the future. If Spamassassin is set to create a new message and forward the original message as an attachment, the user may need to extract the original message from the attachment and place it in the MakeHam folder. 

Automated systems like this one can take time to develop and are sometimes tedious and error-prone to get right and keep right. We always recommend starting small with minimal functionality, proving that functionality over time, and adding to the functionality at a later date.

DIY NAS and The Effect of Open Source on Small Business

As I noted earlier, I dropped my attempts to use a SoHo NAS product in favor of building a NAS from spare parts and a couple of extra items (an SFF case, SATA card and extra drives). The total cost of the DIY NAS was about $700, and it did involve some time for configuration. Cost-wise, if you are billing your time, it probably comes out to a wash to go DIY or buy the Netgear ReadyNAS NV. The ReadyNAS NV is a step above the products I tested in cost, features, and functionality.

However, I have found other uses for the new NAS that make the configuration effort worthwhile, at least for me. We have been running an installation of SugarCRM, but a little while ago we had to re-purpose the machine it was running on, and we had not taken the time to re-install SugarCRM anywhere else. Enter the DIY NAS, a full-featured Ubuntu 7.1 server. In a couple of hours, we were able to install and configure a base deployment of SugarCRM Community Edition. (If you have worked with a CRM system, you know that installation and configuration is just the beginning of the effort for building and maintaining an effective  system). I expect theSugarCRM system to be low traffic in the short term, so pulling double-duty as a file server and CRM application should be fine. Long term, we will need to port the CRM system to more powerful hardware, but for now we have added significant capability to our business at minimal cost. We may also set up the NAS as our local LDAP server in the office, though as a small business we are not yet at the point where domain services are a real necessity. 

For small business, the combination of ever-faster computer hardware and open source software provides a low-cost avenue to business automation that was once the province of huge corporations. Today, any mom-and-pop shop with some technical skills or a few thousand dollars for consulting services can deploy enterprise-class business automation solutions. In one sense, it signals a loss of competitive advantage for big enterprise, and it knocks down an argument for scale in business, which is the cost and complexity of such solutions. There are still lots of other reasons why scale matters, but it is interesting to see how IT evolution has become an equalizer for small businesses.

More Entries

BlogCFC was created by Raymond Camden. This blog is running version 5.8.001.