IMEJ main Wake Forest University Homepage Search articles Archived volumes Table of Content of this issue

1. Introduction
2. The Packages
3. IOS System
4. Interaction Concerns
5. What Are Students Learning?
6. Learning to Learn
7. The Interface Design
8. The Content
9. Conclusion
10.

References

Printer-friendly version


Courseware Quality and the Collaborative Classroom: Implementing IOS Courseware to Generate Seminar-style Interactions
Mary Flanagan, University at Buffalo
Chris Egert, University at Buffalo

Abstract
Administration, parents, and faculty ask many questions about the value and purpose of courseware use in higher education. As the designer and active user of such courseware for my classrooms, I am deeply invested in this debate. Like most universities, my home institution sponsors a few distance learning initiatives. Distance education and online learning are touted as economically sound and convenient (Gubernick Ebeling 97). How can these concerns be addressed by the software designer, the interface designer, the learning community, and the instructor? How can the decisions used to address the technology use in and out of the classroom be adequately assessed? In this paper, we demonstrate how we addressed these concerns in the implementation of custom courseware called "IOS."


About the authors...




1. Introduction
Administration, parents, and faculty ask many questions about the value and purpose of courseware use in higher education. As the designer and active user of such courseware for my classes, I am deeply invested in this debate. I teach at the University at Buffalo (UB), a school of 26,000 students which is neither on the cutting edge of technology in the classroom nor last in line. As the largest school in New York’s "SUNY" system, UB has the advantage of being able to offer a variety of different sized courses and a diverse selection of courses in its many departments.

State University of New York at Buffalo



Like most universities, we do have a few distance learning initiatives on campus. Distance education and online learning are touted as economically sound and convenient (Noble 99). According to Harris, there are three powerful reasons why Internet-based distance education is proliferating in the US. First, he notes that since the technology is available, the need is evident; second, online education is an important financial market for universities; and third, access to education is enhanced and opportunities are expanded from a political point of view through distance learning opportunities (Harris 99). David Noble's thoughts on the "Digital Diploma Mills" and his series of online instruction and the commercialization of higher education criticize the economical soundness of distance education, and the debate rages on about faculty job futures, technological takeover, and the commercialization of higher education (Noble 99).

SUNY Buffalo Distance Learning Initiative

David Noble's Digital Diploma Mills



The quality of education in the trend towards online education interests me more the political reasons behind this shift. I teach courses in UB's Department of Media Study. Course size is generally under 25 students per class, and in-class interaction tends to be high. Students, parents, and other educators frequently have overlapping concerns about the use of courseware in higher education, and generally ask four types of questions about courseware use:

1. How do you choose the package? With so many pieces of courseware out there, in addition to textbook publisher’s efforts to make web/texts, how do you find the technology solution to meet your needs?

2. How do you decide which tasks to assign in the courseware? What are your needs with the courseware? What is the benefit of using the technology for particular tasks?

3. How will students learn to use it, especially if they are not computer literate? Will students from non-privileged backgrounds suffer from the introduction of courseware into the curriculum?

4. Why will using the courseware help students learn in the course? Will the technology obscure the real content of the course?

These questions are realistic and valid concerns in response to the sometimes-mystifying push for technology in the classroom that has infiltrated most colleges and universities. How can the software designer, the interface designer, the learning community, and the instructor address these concerns? How can the decisions used to address the technology use in and out of the classroom be adequately assessed? In this paper, I demonstrate how I answered these concerns for both "in person" courses and distance learning courses through the implementation of custom courseware called "IOS: Interactive Online Seminars." The advanced level seminar has been ignored by massive software development efforts because of the specialized needs of the course types (institutions often want large introductory courses online first) and cost justification for creating such specialized software. In addition, seminars going online are often assumed to be accommodated through web sites and email between the student and instructor (Flanagan and Egert 98).

UB's Department of Media Study


2. The Packages, The Processes: Comparing Course Management Systems
How do you choose the type of software to use or create? When is it appropriate to use courseware packages? With so many textbook publishers’ efforts to make web/texts or electronic supplements to textbooks, as well as numerous courseware programs – or course shells -- out there, how do you find the technology solution to meet your needs?

First you must decide how you want to interact with the students online—have you time in your schedule for real time (synchronous) interaction to simulate seminar style interaction, or will you need to find an equally engaging interaction model? How will you document student interaction in the chat? Students will have to be trained to show up at the appointed place and time, and with live events, a few will have technical difficulties during synchronous meeting times.

Second, you must decide upon the amount of content you wish to create or have someone else provide. Could you use a commercial textbook website in addition to your course materials? The most likely answer is no if you are teaching advanced seminars. Is it important to have all of your materials on line, including video presentation or animated diagrams, or is an outline or assignment/question list adequate as an accompaniment for a seminar?

Third, determine how much you want the students to create and how this material will be displayed and managed. Seminar style interaction relies on class discussion and the ideas and materials created by the students, with the faculty member acting as facilitator. As the faculty mentor guides a discussion, there is an agenda: to ensure that course objectives are met and that all participants have equal opportunities to benefit from the discussion (Albrektson 95).

The fourth factor in choosing courseware is centered on the outcomes the instructor desires. The instructor needs to identify what outcomes the courseware will help foster or generate. For example, seminars generally do not require online quizzes and multiple choice tests, but rely much more on close interpersonal interaction, in-depth discussion, and the learning process. Seminars tend to attract a more experienced and engaged set of undergraduate and graduate students as they are generally offered at advanced levels, and this background will help foster discussion. Finally, you must consider your software selection in relationship to the setting of the class: an in person, "live" class might use the web for different purposes in your curriculum than a distance learning course.

Some commercial packages could be used to facilitate seminar interaction, but I chose to create custom courseware, IOS, Interactive Online Seminar system, with the help and expertise of a computer science graduate assistant. Like the commercial products Top Class, Web CT, Learning Space, and eSchool, IOS is a course management software package which organizes coursework and facilitates interaction on the web.



3. IOS System and Capabilities
The IOS: Interactive Online Seminar system is a technological solution to a familiar problem – successfully addressing students with diverse backgrounds through courseware design. Like the popular commercial products Top Class, Web CT, Learning Space, and eSchool, IOS provides a web-only, cross platform, security-driven course system with asynchronous comment and submission features. IOS, like the other platforms, incorporates student monitoring, student posting of work, and class input features. It differs from other packages in that it provides an easy to understand user interface, a simple upload model, and simple functionality. Most importantly, IOS provides an outlet for course interaction that is fundamentally different from that of threaded discussion in its different way of organizing discussion, course hierarchies, and collaboration, including its unique mark-up ability. (The instructor can mark, edit and change student work.)

IOS was designed to initially meet the needs of the Advanced Multimedia Design Seminar, a course taken by seniors and graduate students in illustration, graphic design, and multimedia at UB in 1997. The class participants needed to be able to share their graphic and multimedia work and interact with the work of other students. They had to be able to contribute and receive meaningful feedback frequently. Since the students were advanced, the freedom for students to work asynchronously, have a record of their progress, and update their work was crucial. The IOS system has now been utilized over a period of two years at the university level in a variety of courses including courses in media production and others which emphasize criticism and theory. Over this time, the system’s user interface and additional aspects have been studied through surveys, interviews, and usability testing. Server usage patterns have also been tracked in order to correlate them with user's actions while using the system. In order for this application to be useful, it had to combine technological advances with sociological models to support co-dependent work (Grudin 91).

 



IOS consists of two applications: an administrative component and a student component. The administrative component is a database application that allows a class instructor to create record entries for courses, students, and projects. The administrative application can monitor the progress of a student in a class by reporting on the current submission status of the student work, monitor the level of participation within the comment area, and track the student’s usage patterns within the system. The instructor uses the administrative application to dynamically change information about the class, such as adding or deleting assignments, changing due dates, and locking student work as examples for future classes. Finally, the administrative system is responsible for maintaining a file structure capable of storing student submissions and criticizing text. The second half of the IOS system is the web-based student interface. The web approach was chosen since it offered the flexibility for cross-platform operation and for student submission of their work from both school and home. The IOS system utilizes Java Servlets (Davidson 99), which can be best described as server-side applets, for most system tasks. Servlets are responsible for accessing database information and for dynamically generating HTML content so the user is free from the FTP process--the servlets are responsible for facilitating the upload. When a student participates in an IOS session, he or she is presented with a link-based navigational interface that allows for the selection of a course, student, and project. These navigation pages are dynamically generated by the servlets. When the student visits a project area, the interface appears as several frames which allow different areas for menu items, multimedia content, and text-based comments and discussions. The student then has the option to browse, post a comment, or submit multimedia content. The system is password protected so that only class participants can post comments, and only the owner of a particular page can submit content in his or her own area. All student comments are tagged with the poster’s name and the time of posting on the server so the instructor can monitor use of the pages and events such as multiple postings by a single student, the time of posting, etc.

How does this system measure up? Compared to Web CT, TopClass, and others like them, IOS is certainly limited in its features. Yet it is limited due to the needs of the seminar style course. Packages such as Top Class, for example, incorporate many valuable features, but most of these features are not useful to enhance seminar style learning and interaction. One such feature is the multiple choice test function and automatic test grading.

(11 KB) Screen shot of the instructor's opening page.



4. Interaction Concerns
When IOS was initially utilized, it was used in a small class of advanced multimedia students. These students could easily monitor the progress of their classmates and could critique all of the work presented because of the scale of the course. As the IOS system was used in larger classes, it became more difficult for an individual student to monitor the work of other participants. In addition, when the courseware was used in conjunction with the in person class, students tended to cluster their comments around a few individuals in the class, leaving other students out of the communication process. In the distance class, only when a participant failed to turn in an assignment was he or she neglected in discussion, since these individuals had not posted work to discuss.



In response to the first problem, the growth issues inherent in the system, the interface elements used to track the status of individual work and mechanisms to enforce equity in the comment process had to be developed. In order to provide stronger feedback mechanisms, the interface incorporates two-part indicators in the project mode to indicate the submission status of a project. The first indicator reflects submission changes on the system. A class participant can view project participants, for example, and determine who has submitted new work and when. The second indicator is a student self-assessment. The student is able to provide commentary and an indication of the state of his or her work. By means of these two indicators, one can quickly access each class member's progress. The visual indicators are incorporated into the interface to allow quick access to work that has changed.

To address the potential for inequitable participation using the courseware, a measurement system was implemented. Although it would be desirable for students to comment on every piece of work, this is not often feasible in larger classes. To create balance, a comment requirement was added to the system. The comment requirement manager, housed in the administration section of the system, allows the instructor to detail the number of comments expected of each student per project. When this requirement is set, the system generates a list of participants per student for which he or she will have to comment. Status of both submission changes and student self-assessment is represented graphically. Students can comment as much as they wish once they have met the instructor-generated quota.

(13 KB) Screen shot of the IOS student status page.



In fall 1998 and spring 1999, I surveyed a total of 42 students; 35 out of 42 students returned surveys. When surveyed, 71 percent of these students did not want a secondary email account for the class. They wanted to be able to communicate as easily as possible with everyone, and to them that meant one email account for everything. To communicate personal messages to the student, the instructor utilized email via the student’s main email account.

According to Pritchard, Internet classes are, in many ways, far more intimate than a traditional classroom. "Participants expect more personalized responses, personal attention, and individualized experience. The instructor has to live up to those expectations with timely and accurate responses." (Pritchard 98) Newsgroup-style interaction, or discussion threads, can be vital learning areas if the instructor encourages (and enforces) their use. But I wanted to rearrange the idea of discussion threads and make the student’s work-- their responses to the readings in a media theory class, or creative assignments in a media production class-- to be the guiding discussion elements for the course. The instructor interacts on a peer level with the other students to pose questions to individuals and groups at the same time at the site of a particular work.

Chart showing student email preference. (Requires Flash plug-in.)



5. What Are Students Learning?
Why will using the courseware help students learn in the course? Will the technology obscure the real content of the course? There are several benefits both to using courseware for distance learning and as a component of an in person seminar-style course. In fall 1998 and spring 1999, I surveyed a total of 42 students; 35 out of 42 students returned surveys. The courseware "allows access to other people’s work without having to make a million billion photocopies," noted one student. But beyond the technicalities, over 25 percent of the students commented in surveys that the best aspect of the project was that they could upload and discuss visual content, see and talk about the work of their peers, and "get ideas from each other." The very learning process in the courseware—represented through the interaction model-- is the most important part of student learning. When promoting technology as a tool for active learning, Professor Stephen Heppell, an online learning researcher and head of Britain’s UltraLab, suggests it is more important to focus on how students learn rather than what they produce (Levis 99), and this is in line with the interaction style of a seminar which focuses on human interaction, debate, and active problem solving—the "getting ideas from each other" aspect of the software.

 

Chart showing student likes. (Requires Flash plug-in.)



6. Learning to Learn in a New Format
Using courseware can be daunting. Even journalists writing about online courseware face difficulties working with the various systems and software some courses require. Esther Shein of PC Week took a three week distance learning course at the University of Massachusetts to be able to write from the point of view of a student. She had to download software required by the instructor, and she notes, "It was a big deal. An FTP program? Something to unzip it? A screen grabber? Although this software was available at the UMass Help site, I didn't know exactly what I was downloading or where to open the file once I downloaded it, for that matter" (Shein 97, "Anytime, Anywhere"). If Shein, an eager professional, had difficulties, imagine the feelings of a technophobic student. How will students learn to use courseware and get the most out of it, especially if they are not computer literate?

One approach is by introducing them to courseware as a supplement to an existing in person course. Once a student has been introduced to the concept of courseware with a mentor and has worked through the process himself or her self, the student rarely continues to have trouble. One preventative measure, both in distance and in person courses, is to assign each student a "techPartner" so that there is always someone technologically savvy paired with a beginner.

Skeptics of courseware say it won't work if users aren't Web-savvy, and there also are personality inhibitors--for example, there might be individuals who aren't disciplined enough to keep up with material. (Shein 97) In my focus group in-person and distance learning classes at UB, I have thus far found that the education of students from non-privileged backgrounds does not appear to suffer from the introduction of courseware into the curriculum as long as all the students who indicate discomfort are given one-on-one tutorials with the software. Contrary to what might be expected, students who notified me at the beginning of the semester with fears about using the technology because of their backgrounds, experiences, or economic status did not, as a group, fare worse than other students; in fact, these students sometimes became enamored with communicating with technology. But the results show something of interest concerning courseware interaction trends and in person course interaction trends. Those students whose attendance scores were low in my "in-person" class were also the same students who did not use the course’s accompanying courseware or submitted material to the courseware intermittently at best. Thus, with a generous computer access policy at our university, students from any background seemed to have equal opportunity once they were familiarized with the computer and the program. The class, however, did have some students who failed to participate.



7. The Interface Design
The interface was the primary site for testing the IOS system. The system design required that a student navigate from course to student and then from student to project in order to view submitted content. The reason for the original design was to create a uniform navigational space and to create a strong relationship model between the class components. In classes conducted using web technology before the use of IOS, students often posted their media content to their own web page or to the Departmental server; this ad-hoc structure generated confusion and frustration for students trying to find work, and resulted in inconsistent navigational models; content location varied widely. Using IOS helped organize things and standardize naming conventions, but students, though instructed repeatedly, had the most problems with naming conventions out of all feature sets. The naming convention was to name the first page posted as "index.htm."

The concerns students had with the interface was not a result of the navigational model but of the posting subset. Out of the 35 students surveyed, 91 percent of the students saw the interface as "easy to use," but the survey also uncovered 31 percent—almost one third-- of the users had trouble with the naming conventions when posting their files to the system.

 

(11 KB) Screen shot of the IOS introductory page.

Chart showing student evaluation of the interface. (Requires Flash plug-in.)



8. The Content
As a course shell, IOS has little content except that generated by its users. The organization and manipulation of the content is an important task of the courseware. In some courseware packages, students spend a great deal of time lost in poor organizational information structures. IOS has two ways to provide unique and variable access to content: a portfolio mode, which organizes the navigation into particular "student sets" (looking at work which belongs to a particular student), and a project mode, which organizes the navigation around a particular class.

 

 



One of the most important content elements in IOS is an interactive white board which allows the instructor and other students to draw on top of a student’s work –the participants can draw, edit and change student work. This feature maps a color ink to a particular user and only registered users can participate. This assists media classes in particular since it is often useful to offer suggestions to graphic work visually. The asynchronous whiteboard applet is used in conjunction with the IOS system. The comments layer is superimposed on the original submitted work. Follow-up comments can either utilize the original media image or a combination of previous layers when formulating a response. The actual whiteboard application provides simple editing tools such as lines, fills, colors, and arrows.

 

(13 KB) Screen shot of the IOS Whiteboard.



9. Conclusion
IOS was created to facilitate distance learning needs specifically for a seminar style environment that could not be met by other forms such as email or chat. IOS was created to foster a participatory feel for an online classroom experience that encourages an interesting and interactive session. Tested in the classroom, it is a valuable interaction device and meets diverse students' needs in distance learning and face to face course settings. This open-ended software system can be used to meet a wide variety of course needs.




10. References
Albrektson, J. Raymond. "Mentored Online Seminar: A Model for Graduate-Level Distance Learning." THE Journal (Technological Horizons In Education) Oct. 1995:102-106.

Davidson, James Duncan, and others. Java Servlet Specification, v2.2. Palo Alto: Sun Microsystems, 1999: http://java.sun.com/products/servlet/index.html.

Flanagan, Mary. and Christopher Egert. "The Course Submission System: Providing 'Seminars' on the Web", Proceedings from the Association for the Advancement of Computing in Education Webnet 98 Conference. Orlando 1998: 313-317.

Grudin, Jonathan. "CSCW Introduction: Computer-Supported Cooperative Work." Communications of the ACM Dec 1991:30-35.

Gubernick, Lisa and Ashlea Ebeling. " I Got My Degree Through E-Mail." Forbes 16 June 1997: 84-90.

Harris, Dale A. "Online Distance Education in the United States: Tele-Learning: The "Killer App"?" IEEE Communications Magazine Mar. 1999: 87-92.

Levis, Neil. "Technology's Tool for Active Learning." Times Educational Supplement 4 June 1999:A31.

Noble, David. "Digital Diploma Mills." Internet. November 1998. http://communication.ucsd.edu/dl/ Accessed 12 Sept. 1999.

Pritchard, Carl L. "From Classroom to Chat Room." Training & Development June 1998: 76-78.

Shein, Esther. "A Firsthand Account of Cyberlearning." PC Week 10 Mar. 1997: http://www.zdnet.com/zdnn/content/pcwk/1410/pcwk0057.html.

- - -. "Anywhere, Anytime:Web-Based Courses, Virtual Classroom." PC Week 10 Mar. 1997: 115-117.



********** End of Document **********
IMEJ multimedia team member assigned to this paper Daniel Pfeifer