News

Front Lines is a guest viewpoint section offering perspectives on current issues and events from people working on the front lines of Canada's technology industry. Dr. El Emam is the author of "The ROI from Software Quality", and an Associate Professor at the University of Ottawa, where he is a Canada Research Chair in Electronic Health Information. In 2004, Dr. El Emam was ranked as the top systems and software engineering scholar worldwide by the Journal of Systems and Software based on his research on measurement and quality evaluation and improvement.

Much ado has been made about the benefits and quality of open source software, or OSS.

However, CIOs need to take a hard look at the touted advantages of OSS — namely that it is free, high quality, secure and that the code can be modified easily — and determine if these claims are real. They need to evaluate the risks of OSS to their business and to the IT capacity of their organization.

Security is a moving target

Security is an industry-wide issue, and building secure software is difficult - open source or not.

Not surprisingly, when the development team for a product focuses on security, then the product tends to be more secure. This applies to both open-source and commercial software. More secure products have less vulnerability, so if vulnerability is discovered, the organization supporting it is able to send out patches immediately.

There are many examples of good and bad in the OSS and commercial worlds. To manage security risk properly a CIO needs to determine what practices exist to produce secure code, practices for security testing and the qualifications of those responsible. If the development organization has been audited to verify that they have good practices, then that is a big plus. This must be done on a case-by-case basis due to the amount of variation in the security of software within the OSS and commercial communities. Being open or closed sourced by itself does not ensure that the software is secure, it is also important to evaluate each of the suppliers.

Add the code?

The ability to modify the programming code is one of the most-cited benefits of OSS. However, customizing code is something most IT departments are loathe to do. Modifying open source is generally not a good idea, because organizations are then stuck maintaining a large custom application. Usually, this is much more expensive than licensing a commercial application.

Modifying software code also means that as the main thread of the application evolves the organization will not be able to apply new features and security patches very easily. There is a possibility of contributing the customized version to the public OSS version to narrow this gap, but there is no guarantee that the modifications will be accepted by the core development team of the open-source project. The organization has thus effectively inherited the responsibility of maintaining a large custom OSS application.

You get what you pay for

Many open-source supporters make a broad statement that unlike commercial software, OSS is peer-reviewed by hundreds or thousands of developers. In reality, the number of peers reviewing most open-source projects is about the same as those involved in reviewing commercial projects, or less. For example, in one survey of OSS developers, only nine per cent said that all of their code is peer reviewed, but about half say that much of their code is peer reviewed. These numbers are similar to what one will find in commercial projects based on another set of surveys. Hence, this is not a valid distinction. Even in head-to-head comparisons between software defects in OSS and commercial applications, there is no clear evidence that open source is inherently of higher quality than commercial software.

Meanwhile, many of the best peer-review practices are not commonly used in open-source projects. For example, most OSS development teams do not use checklists as a tool for defect detection. Also, OSS testing practices tend to be weak. Test coverage tends to be low, test plans are rarely produced, and in many cases a range of internal tests is skipped in favour of beta testing by end users.

There is evidence that as some of the more popular OSS applications increase in size, their structure deteriorates. Such big system problems are well-known in software engineering circles. Experience tells us that more quality issues with these OSS applications will emerge over time as they grow.

In specific application domains, many open-source developers are domain specialists. For example, in open-source biomedical applications most of the people writing the software are clinicians, geneticists, and biochemists. While they may be good programmers, many of these individuals have not received software engineering training. Just as software engineers do not practice medicine, biomedical specialists are not trained to write code. This has to raise questions about the quality of that software.

Licensing value

Finally, surveys show that one of the main reasons that people adopt open source software is because they believe it is free. Everyone loves a bargain, and with corporate IT budgets getting tighter, "free" certainly sounds like a good deal. The downside of free OSS is that it does not come with any contractual obligations of support.

When you purchase commercially sold software, you buy a licence to use the vendor's intellectual property. Licensing software entails some contractual obligation on the vendor, including providing support.

Such support contracts help purchasers get the utmost value from the software by guaranteeing a level of service and maintenance. The licensing model works so well, open source vendors have adopted it for themselves. And though they charge for "free" software, the licences are generally cheaper than those of commercial vendors.

Conversely, commercial software vendors are lowering licence fees to compete with OSS vendors. Some are also opening up their proprietary code to certain customers with an interest in building applications using the vendor's IP — governments and academic institutions, for example. With the two models beginning to blend somewhat, it pays to do your homework and find the right software to run your business — nothing is really free.

Whether an enterprise chooses OSS or commercial software in the end, it pays to consider each application on its own merits. CIOs need to examine if managing the code is what they really need. If they do choose to go the OSS route, it is instrumental that they purchase vendor support. They will need someone to turn to when something goes wrong.