Crossroads in FOSS Projects: Some Business Considerations

At our Seminar last month, Managing FOSS to Lower Costs and Achieve Business Results, several participants asked about the dynamics of FOSS (Free and Open Source Software) projects that reach a crossroads (a failure, a merger, loss of key personnel, etc). I had not expected that concern because with commercial software, it seems to me, the problem is more severe. When you have the source code and the right to modify and redistribute it, the source gives many more options (and its freedoms provide many more protections) than when commercial software goes bankrupt or gets bought by a competitor for instance.

But the reason for the questions may be a lack of understanding about how FOSS projects work. They involve individual human beings, perhaps just a single person or, more likely, several people from many organizations and even different cultures around the world joined in common purpose. For various cultural reasons, the project may be “owned” by an entity — usually a non-profit, but some are for-profit or even government owned, while others may simply be an “ad hoc initiative”. Some projects have explicit constitutions and defined processes for organizing the work and handling problems others are more informal.

At any time, any human social structure can experience a crossroads that could lead it to fail suddenly or wither on the vine in a gradual descent into “oblivion”. The cause of the failure will shape the results, but a very common situation is that conflicting visions or approaches for the project result in a “fork”. Then a sub-group of the original project takes the source code and starts a “new” project to develop the code in a new direction. Sometimes the original project “dies” and sometimes both continue resulting in two projects. Since multiple FOSS projects serving the same function or market incur inefficiencies due to duplicate development, there is a strong cultural value in the FOSS world to try to find a way to accommodate everyone in the project and prevent forks. When it works, the result is great software that meets everyone’s needs. But the reality is that often it is more effective to have multiple implementations of the same functionality so that each can be optimized for distinct objectives. Frequently one cannot know which approach will be best until many years of development and evaluation have transpired.

I recently learned about a FOSS project that forked when a friend asked me to copy some files to his new “My Book Essential”, a Western Digital product that provides 1TB of USB (Universal Serial Bus) storage. The My Book uses the poorly documented, non-free NTFS (New Technology File System). Linux has three projects that support NTFS: an in kernel driver, ntfsprogs (the Linux-NTFS project), and NTFS-3G. It turns out that all three were available for my Debian Lenny (5.0.3) system. First, I tried the in kernel support and learned that it was still read-only. Then I tried ntfsprogs which failed to mount the My Book:

NTFS-fs error (device sdc1): load_system_files(): Volume is dirty. Mounting read-only. Run chkdsk and mount in Windows.

I realized that since it was a new device it probably did not ship from the factory with a dirty volume. It was probably a bug. So I tried NTFS-3G which worked very well. In my research of the situation I was able to determine that both NTFS-3G and Linux-NTFS are under active development and have features missing from the other. So each has value and I’m glad my distribution included both. In Debian Lenny, the NTFS-3G driver has better support for writing files.

This illustrates one of the benefits of a crossroads in a FOSS project: you can end up with two good tools to add to your toolbox!

Posted by CJ Fearnley in Debian, FOSS Community, Tech Notes, 0 comments

Forthcoming Design Science Symposium and Systems Administration

Ever since I started doing systems administration, I’ve been interested in applying Buckminster “Bucky” Fuller’s comprehensive anticipatory design science to the task. Bucky extolled the virtues of a comprehensive approach. Put bluntly, the comprehensive perspective says “since what you don’t attend to will get ya, you had better consider everything. Or said positively: only by considering all elements in a system and all its interrelationships with other (relevant) systems can you ensure reliable on-going operation. In addition, the proactive or anticipatory approach is essential to prevent system complexities from impacting operations. I think of design as human initiative-taking to provide a service or artifact and science as experience-based learning. Evidently, design science is implicit in the work of systems administrators. I think the discipline of comprehensive anticipatory design science can be positively applied to the practice of systems administration.

So I am excited that on November 14 & 15, I will be attending the Synergetics Collaborative’s two-day Symposium on “Design Science” at the Rhode Island School of Design (RISD). Together with the organizing committee, we (I serve as volunteer Executive Director of the Synergetics Collaborative) have put together a program that will develop a deeper understanding of design science. So even though computer systems administration is not on the agenda, I think anyone with a problem-solving focus in their work (including systems administrators) would benefit by attending.

To find out more about this exciting event visit the Design Science: Nature’s Problem Solving Method Symposium home page here.

Posted by CJ Fearnley in News, 0 comments

Slides Available From Our Managing FOSS Seminar

Last Thursday LinuxForce hosted a seminar on Managing Free and Open Source Software (FOSS) for Business Results. The Seminar home page now has links to the slides from the event. Specifically, there are four sets of slides available:

Please let us know if you have any questions about the content in the slides or from the seminar itself.

Posted by CJ Fearnley in News, 0 comments

Contributing to FOSS: A Business Perspective

Last weekend I had the pleasure of presenting at the Central Pennsylvania Open Source Conference on the topic of Contributing to FOSS (slides available here).

In the talk I explored the many ways individuals can get involved in FOSS (Free and Open Source Software), briefly covering everything from programming to artwork to documentation. As diverse as these contributions are, the common thread is close collaboration with the project itself. In particular, following the procedures in place for contributing to the project is essential. The talk also reviewed some of the benefits of contributing to FOSS, which include career advancement and the ability to expand your professional network.

Although my presentation focused on individual contributions, these lessons also apply to how businesses benefit by contributing to FOSS. When a business approaches a project they should attempt to build a symbiotic relationship with the community. Such a relationship involves following the established community procedures so that your contributions can be easily adopted by the project. Useful scripts and code developments made within the company that can be useful to the greater public should be contributed back and packaging of popular software within the company can be submitted for inclusion and use by the greater community. Testing and bug reporting based on experience using FOSS on their production (or development) systems can provide important information for FOSS developers about the health and status of their projects.

Benefits for businesses that we at LinuxForce have seen first hand are referrals for projects based on documentation work completed on popular community websites (such as Debian-Administration.org) and feedback on our approach leading to improved best practices and building a reputation as experts. By sharing code with projects, others can build upon it to produce more functionality than your team could muster on its own, creating better software for everyone. Additionally, our involvement has allowed us to foster development of Debian packages for software that is used by our clients by, for instance, improving automatic database configuration support and making sure up to date packages are included in releases.

In conclusion, when a business contributes to FOSS they can help drum up business by building a reputation and doing real work within the community, and they help their customers by being on the forefront of development direction and discussions for software that is vital for their own organizations. Contributing to FOSS is good for business, good for your customers, good for the community, and good for the FOSS ecosystem in general.

Posted by Elizabeth Krumbach in FOSS Community, News, 0 comments

Customization, Upgradeability and Eternally Regenerative Software Administration

Mary Hayes Weier wrote an interesting article in this week’s edition of InformationWeek on "Alternative IT: CIOs are more receptive than ever to new software models". What is great about her article is how she captured the divergent views on IT models (such as SaaS, cloud computing, etc.) and gave nice vignettes of different organizations trying different parts of various models. I especially valued her use of cognitive dissonance to leave the reader thinking … better informed but without a firm conclusion.

There are so many parts of the article that I could blog about, but the one that touched the core of my thinking about “eternally regenerative software administration” was the quote by Bill Louv, CIO at GlaxoSmithKline, who said

"And here’s the rub: When you customize software, it’s difficult to implement future upgrades from the vendor"

Louv touched the very bane of eternally regenerative software administration! Software should accommodate both customization and upgradeability: these two elements of software administration are at the heart of my notion of eternally regenerative software administration: how to preserve customizations and provide smooth (near zero downtime with almost no glitches) upgrades through major release after major release. It is a big challenge, but in our experience the Free and Open Source Software (FOSS) communities are at the leading edge in finding solutions to these conflicting objectives. Here are some of the innovative ideas from the FOSS world which should serve as models or design patterns for all software developers (if only these ideas would become commonplace!).

First, Debian (a FOSS operating system which is the root of Ubuntu, Knoppix, Xandros and many other Linux distributions) requires that their official packages, a collection of software prepared for easy administration, must adhere to a very mature policy. Debian’s policy is a marvel in the FOSS world and to a very large degree is responsible for its strong support for both customization and upgradeability. I think Debian’s reputation for stability and maintainability is almost certainly due to their decision to develop a consensus-driven policy that its software must implement.

For example, the Debian package maintainer, Luigi Gangitano, for Drupal, a FOSS content management platform, did a great job making the software both customizable and maintainable. The package supports configuration of multiple virtual hosts which can all be upgraded at once! And the Debian drupal6 package stores the look-n-feel in /etc/drupal/6/themes/ so that each site’s GUI can be customized without interfering with upgrades. If only all web applications were built to be as maintainable as Debian’s Drupal package!

Another example is the overlay support included in RT: Request Tracker, a FOSS ticket tracking system. This allows putting replacement subroutines in special files in /usr/local/share/ which overlay or substitute the upstream code. This approach is more likely to break on upgrades, but it supports minimal changes to the business logic with a decent chance that upgrades will be smooth.

There are countless more examples from the FOSS world of innovative solutions to inter-accommodate customization and upgrades in support of eternally regenerative software administration. What are some of your favorite examples?

Posted by CJ Fearnley in Debian, Eternally Regenerative Software Administration, 1 comment

Congratulations Elizabeth on your election to the Ubuntu Community Council

I was thrilled when I saw that Mark Shuttleworth announced the election of Elizabeth Krumbach to the Ubuntu Community Council. Here is my “open memo” of congratulations to Elizabeth:

Elizabeth, you earned this honor to serve through your competent and tireless efforts to positively contribute to FOSS (Free and Open Source Software) communities like Ubuntu and its upstream, Debian. Collectively, it is the work done in the FOSS communities that has built a “game changing” software infrastructure which already delivers business results to illions of organizations around the world … day in and day out. Thank you for all that you do by contributing to these vitally important communities!

The Ubuntu Community Council plays an important role in the management and development of the communities which not only build the Ubuntu operating system but also contribute to the FOSS communities that intersect with Ubuntu in so many ways. Your good judgement and broad experience on the workings (both socially and technically) of FOSS communities will help Ubuntu continue producing software to meet the needs of more and more individuals and organizations and thereby grow the whole FOSS ecosystem.

From working with you the past few years, I know you are ready to lead Ubuntu in support of reaching toward the lofty ideal of eternally regenerative software (ERS). ERS is an emergent property of the Debian GNU/Linux operating system (which Ubuntu has inherited) wherein all the component software is integrated to facilitate easy upgrades (re-generation) through each and every (eternally) major new release of the operating system. I eagerly look forward to seeing how your contributions to the Council will foster efforts to improve the integration and eternal regenerativity of free and open source software in support of providing business results to all the organizations that have so wisely chosen to use FOSS.

Posted by CJ Fearnley in News, Ubuntu, 2 comments

Xen Virtualization: Migrating 32-bit domUs to 64-bit dom0s

Virtualization provides the facility to run multiple isolated computer operating systems on one piece of computing hardware. There has been a huge increase of interest in virtualization technology because recent advances in multi-core technology provide significantly more computing power in each machine with ever decreasing costs. Virtualization is one of the best ways to take advantage of these big changes in hardware.

Currently, Xen is the most mature FOSS (Free and Open Source Software) virtualization technology. Although we love the idea of KVM, since it requires a special processor extension on X86 systems, it cannot work on older hardware. So for at least another few years, we think Xen is the more flexible choice for FOSS virtualization projects.

The Xen infrastructure consists of the Xen hypervisor which “runs the show”, a domain 0 (dom0) which runs a special, privileged version of the operating system (typically Linux, but NetBSD and Solaris are also supported), and one or more domain U (domU) “guest” (or “User”) operating systems. We have found that Xen is easy to configure in many situations, but we encountered some complications in running a domU on a dom0 with a different architecture.

We recently migrated some 32-bit domUs running Debian Etch (4.0) from a 32-bit dom0 to a newer 64-bit dom0 running Debian Lenny (5.0). We did a direct move (using rsync) of the Logical Volume Manager (LVM) slices from the 32-bit dom0 to the 64-bit dom0. This means we’d now be running our 32-bit Etch domUs on a 64-bit Lenny dom0.

The first question was whether this would be possible. Absolutely! 32-bit domUs have no trouble running on 64-bit dom0s, we could even use the 64-bit Xen kernel in these 32-bit systems to avoid additional kernel installations we’d need to maintain on the dom0. The second question was whether we could properly load the 64-bit kernel modules inside our domU. Again, yes! But with a caveat: the domUs were 32-bit Etch, so the 64-bit Lenny kernel modules were not simply installable via apt. We realized that copying over the .deb package for the kernel modules and running dpkg -i --force-architecture linux-modules-2.6.26...deb would not be a maintainable way to handle the kernel module updates moving forward. So we weighed our options:

1. Serve these modules via a network file system (such as NFS) to each domU on bootup.

2. Deploy a script that would notify the domU and copy the new kernel modules .deb to it for installation. We could then install the new module package at our discretion.

We decided that the first option violated our strict security policy which calls for running as few services on the dom0 as possible. Since the second solution is scriptable and therefore automatable, it fit our vision of having easily maintainable systems regardless of the underlying complexity. So we installed the 64-bit modules prior to migration so that all the proper modules would be loaded as soon as we brought up the domU on the new Dom0. The result was a flawless migration of our 32-bit domUs to the new 64-bit dom0.

Posted by Elizabeth Krumbach in Tech Notes, Virtualization, 0 comments

Introducing RemoteResponder.LinuxForce.Net

If you know the history of LinuxForce, you know that we’ve been doing remote systems administration using FOSS (Free and Open Source Software) since our founding in 1995. And we’ve called our remote systems administration service Remote Responder for a long time too. But the website RemoteResponder.LinuxForce.Net is new.

The new site is part of our educational initiative to explain the issues involved in administering FOSS-based IT infrastructures to achieve the promise of greater reliability and ever-improving functionality while keeping costs low and meeting an organizations’ ever-evolving business needs. Check out our new website RemoteResponder.LinuxForce.Net and let us know what you think.

Posted by CJ Fearnley in News, 0 comments

Welcome to The Managing FOSS for Business Results Blog

Welcome to our new blog.

This blog is part of a new educational initiative to foster a deeper understanding of the capabilities and issues involved with administering FOSS (Free and Open Source Software) to deliver concrete business benefits. Although our subject will sometimes become technical, we will strive to address the business benefits at the beginning of each and every post. Therefore, we are confident that this blog will prove interesting and understandable to a broad variety of leaders, managers, and technicians.

We look forward to an engaging discussion in this blog as we explore the new possibilities available to organizations to manage their IT (Information Technology) architecture with FOSS! We hope you check in regularly or subscribe to our RSS feeds.

Posted by CJ Fearnley in News, 0 comments