Skip to content.

Course Management System (CMS) Selection Project

Spring 2005 through Fall 2006

Go to global navigation menu.

Project Objectives

• Gain support from key stakeholders for selecting and migrating to the CMS.
• Create a dialogue on how and why to use CMS.
• Understand the role of CMS within other systems, and understand how we want the systems to inter-operate.
• Reduce confusion for faculty and students by providing a single entry point to course-related web sites.
• Enhance the opportunity for integrating the University’s various information systems (course management, library, digital asset, student information); provide better value to faculty and students for the resources used.
• Understand the cost of different scenarios moving forward, and develop a financial model to understand the investment required in CMS implementation to achieve the desired academic value.

Go to global navigation menu.

Project Scope

The scope of the project includes:

• Establishing a process for selecting the CMS
• Evaluating and selecting a single, centrally-supported CMS
• Developing a preliminary plan for implementing the selected software and migrating current existing course-related web sites

The selection process should be an open one, with the overall process and key decisions widely disseminated to the campus through a coordinated communications effort. Faculty, staff, and students should provide input to the process and be aware of the criteria used for decisions made in the process.

The selection of a CMS will not include evaluation of content management or portal system capabilities. By content management, we mean an extrinsic system with workflow control, version control, and customizable meta-tagging, organized around learning objects. Some CMS’s include intrinsic content repositories that allow a sub-set of content management features; these features of CMS’s are within the scope of the project.

Open-source projects such as Angel and Sakai will be included and evaluated in the project. However, based on input from the Fall 2003 Assessment Project, projects which require “write your own” code to develop CMS functionality will not be included.

Go to global navigation menu.

Process

We adopted a multi-step process to look at the field of possible vendor solutions to UI’s needs for a single, enterprise CMS. This process moved from open-to-closed, starting with a broad list of possible vendors, and subsequently narrowing down the vendor list at key points of the RFI / RFP process. The specific steps included the following:

• Identified candidate CMS vendors, and sent out RFI with minimum requirements (Apr 2004) (Attachment 3 documents the course management systems currently in use at the CIC campuses)
• Received RFI responses based on self-selection model, and validated responses against the minimum requirements of the RFI (3-6 May 2004)
• Determined which vendors to invite to give a campus presentation (3-6 May 2004)
• Based on previous responses and presentations, determined which vendors should receive RFP (24 May – 30 May 2004)
• Released RFP (late Jun 2004), and received proposals (late Jul 2004)
• Evaluated RFP vendors through review of proposals, on-site demonstrations, hands-on evaluations of products, interviews with adopters of systems, and visits to vendors’ headquarters. (Aug – Oct 2004)
• Recommended CMS to Provost and Deans(Oct 2004)
• Negotiated contractual arrangements (Nov - Dec 2004)

Go to global navigation menu.

Open Source Considerations

Open source software is a growing movement within both higher education and the corporate world—a movement which is already providing benefits to the end users. According to a Gartner Group analysis from December 2003, “by 2007 80% of e-learning platform functionality will be available through open source (0.7 probability)”. The report goes on to state “E-learning is emerging as the focal point of higher education's rising interest in open-source applications. Nevertheless, it will be several years before commercially supported open-source software e-learning products will become available. In the short term, enterprises that pursue OSS initiOSS vs. potentially high internal support costs.”

In the past year, all significant open source course management system (CMS) efforts have combined into the Sakai project (www.sakaiproject.org). The Sakai Project is a $6.8M community source software development project founded by The University of Michigan, Indiana University, MIT, Stanford, the uPortal Consortium, and the Open Knowledge Initiative (OKI) with the support of the Andrew W. Mellon Foundation.

After considerable research and attempts to work with Sakai during Spring 2004, the ELearning Core Group has decided the University of Iowa should not pursue Sakai or any other open-source CMS project as part of the current CMS selection project. Sakai does not address any of UI’s e-learning needs as documented by the E-Learning Assessment Project of Fall 2003 (www.uiowa.edu/~provost/elearning) and would, in fact, add considerable risk and effort to the consolidated enterprise CMS project. A report of our investigation into Sakai as a potential CMS solution for Iowa is available upon request.

Go to global navigation menu.

Evaluation Strategy

The CMS products were evaluated on a number of criteria, as specified in the RFP. Input was gathered from a number of sources. Sub-groups were formed from the Advisory Committee to oversee the evaluation of the following aspects of the products.

Product Features & Usability

To test the usability of the three products, five Usability Labs were set up in computing centers around campus. The staff of these centers attended training sessions provided by the product vendors and facilitated the testing. They provided technical assistance to the testers and answered questions about the use of the products. Three groups of users comprised of faculty, staff, and students were provided with rating forms to guide their testing processes:

• Approximately 50 faculty and staff members from a variety of University colleges and units completed task sets in a single, randomly-assigned product. Most testers had previous CMS experience, though about 25% had little or no experience and were characterized as novice users.
• Approximately 20 students, graduate and undergraduate, representing various fields of study and levels of experience with course management systems, completed task sets in a single, randomly-assigned product
• Approximately 16 faculty and staff from a variety of colleges, departments, and units, all experienced CMS users, tested all three products with no specifically-assigned tasks.

Technical Analysis

Technical evaluation of the three products was conducted by a group of seven members of the CMS Selection Advisory Committee. Additional staff members were invited to participate in the evaluation at various points. To carry out their evaluation, the group:

• Gathered and refined a list of CMS technical categories to evaluate.
• Reviewed the vendor proposals; gleaned technical content for an initial evaluation.
• Gathered technical documents from vendors (installation/configuration, application programming interface, etc.). Evaluated technical documentation quality and further refined understanding of technical attributes.
• Conducted face-to-face technical sessions with vendors.
• Interviewed adopting institutions.

Distance and Non-Traditional Learners

A small, but growing, number of programs at the University of Iowa reach out to distance and other non-traditional learners. These learners present unique needs, and so have special requirements of course management systems. A sub-group evaluated the products based on these special needs.

Financial Analysis

A subgroup of the CMS Selection Advisory Committee conducted the financial analysis of the cost proposals submitted by the vendors. The goal was to calculate the cost for the initial setup as well the four years of operations following the initial implementation, for a total five-year cost for each solution.

Company Evaluation

Since the vendor selected in this process will be an important partner for the University in the coming years, selecting a vendor that aligns with our vision and direction for e-learning is important.

As part of our assessment of each company, three University of Iowa staff — Steve Fleagle, Interim CIO; Molly Langstaff, Director, ITS–Academic Technologies; Tim Evans, Business Analyst, CIO Office — visited the headquarters of each of the three vendors. The visits typically lasted six hours. Discussions centered on company vision, financial standing of the company, partnership opportunities, and support strategies.

The Core Group also gathered input from other sources, for example:

• Conference calls with campuses that have recently implemented the vendor’s product
• Feedback from vendor supplied references

Go to global navigation menu.

Course Management System (CMS) Selection Advisory Committee Members

This group of faculty and staff will be advising the E-Learning Core Group on the selection of the CMS. They are listed by department/college.