1. A Course Summary: Pre-Industrial Visual Cultures; to 1789
2. PVC Online
3. Structure of the Online Course
3.1  Content
3.2 Overall Content
3.3 Culture Topic Content
3.4 Synthesis Topic Content
3.5 Interface
3.6 Header Row
3.7 Left Column
3.8 Assessments
3.9 Policy for Written Assignments
3.10 Management Tools
4. Lessons Learned and Future Plans
4.1  Rethinking Assessments.
4.2 Student Reaction
4.3 A Priori Content
5. Conclusion
5.1  Current Modifications and Future Plans
5.2 Acknowledgments
6. References

Online Art History - Design, Development, and Review of an Interactive Course
M. Schmidt, W.H. Blackmon, D.R. Rehak, D. Bajzek
Carnegie Mellon University

Carnegie Mellon Online has been used to deliver several university courses to thousands of students at Carnegie Mellon University over the last two years. This paper illustrates the experiences gained from redesigning an introductory art history course using Carnegie Mellon Online to deliver supplemental course content. The technology, course content, course structure, assessment tools, and course management capabilities are described, as well as lessons learned from creating this course, and plans for the next version.

1. A Course Summary: Pre-Industrial Visual Cultures; to 1789

Pre-Industrial Visual Cultures to 1789 (PVC) is the first course in a three-semester sequence of art history courses required of majors in the School of Art at Carnegie Mellon University. It is geographically and chronologically the most broad of the three courses, introducing students to the ideas and artifacts of world cultures from Paleolithic cave paintings to the French Revolution and including Africa, India, China, Japan and the Ancient Americas. PVC follows chronology, but stops once a week in small groups to synthesize, focusing on topics which cross boundaries of time and space. As an example, the instructor and students recall images of myths and read myths from several cultures, discuss their similarities and differences, explore interpretations of cultural values based upon those myths; and then students write a myth for their own time and space. They learn that the past isn't so remote and incongruous with their own experiences, and that myths are relevant today. Relevance is key

Professor Schmidt and her students weave threads of continuity in visual expression across a patchwork of world cultures. They try to explain the anomalous. They wonder about the meanings of prehistoric works. They investigate the spread of Buddhism and its visual manifestations. They ponder the writings of artists. Each student finds issues, ideas, and artworks that resonate with his or her individual circumstances.

While the instructor encourages individuality, she also expects certain outcomes: that students learn a general framework of chronology and geography for the rich complexity of visual expression; that they learn to listen, read and write critically; that they develop vocabularies to articulate their own ideas about the visual arts; that they be tolerant of the varied expressions by others; and above all, that they gain insights into their own creative processes.

The structure of the course is twice-a-week lectures and once-a-week discussion sessions. Lectures focus on works of art in their cultural context. For example, a lecture on the art of India investigates how visual expressions reflect historical background, politics, geography, social structure, religious beliefs, and the role of the artist in India.

The discussion sessions focus on synthesis topics, such as mythology, the role of the artist, or patronage, which explore thematic comparisons across cultures. In these sessions, students are assigned to work in small groups to brainstorm ideas, discuss issues, and make summary presentations. For example, when we discuss archaeology as a topic, each student group develops a set of questions that an archaeologist would ask about an artifact, and then draws conclusions about the artifact that he or she is given. The assignment for the topic requires that students discuss what an archaeologist in the future would say about their own culture from unearthing a shopping mall or a gallery exhibiting their own student artwork.

Since the Spring of 1995, Carnegie Mellon art historian Mary Schmidt has taught this course by projecting images and text in the classroom with a custom-designed, infinitely modifiable digital database of images, maps, timelines, animations and sound. She also developed a text-based web syllabus with course overview, calendar, readings, links to relevant websites, and essay assignments.

About the authors...


Carnegie Mellon Online
Carnegie Mellon Online is a unique, database driven, educational tool taking advantage of the World Wide Web to provide student-centered, asynchronous learning. The system generates customized content, such as assessments and feedback, for each student and tracks the student through a course while enforcing course-specific policies and procedures. Descriptions of Carnegie Mellon Online courses are available at http://online.web.cmu.edu/.


2. PVC Online

In the Summer of 1997, Schmidt, Rehak, Blackmon, Bajzek, and a team of others collaborated to turn the classroom teaching tools and web syllabus into comprehensive web-based learning tools via Carnegie Mellon Online. The result was the delivery of a database driven, interactive website that contains all of the materials supplementary to the classroom experience, except for the required textbook and visits to the museum. The site contains an overview course description, calendar, topics list, images, timelines, maps, reading assignments, and written assignments. It quizzes students on their reading assignment before each class period and accepts their weekly, electronically submitted essays. On a daily basis, it gives them their personal status (Figure 1) in the course with regard to quiz, assignment, and test scores and calculates their cumulative grade. When students log in, the web site opens to the current topic for the day.

in line demo This Shockwave for Director demo (680KB) lets you browse the online art history course interface (requires Shockwave for Director plugin)

Sample of Student Status Page

Figure 1.Sample of Student Status Page

The first iteration of PVC for the Spring of 1998 exceeded some aspects of our original concept and fell short on some others. The assessment component proved to be a powerful tool. Students developed the sound practice of keeping up with the readings, and they improved their test taking abilities over the semester. They were prepared for class. No weekly essays for this class of 55 students were lost; no my-dog-ate-it excuses. Admittedly there were computer glitches; many students required extra help to set up their web browsers to upload their written assignments.

We planned a searchable database of images but due to time constraints, settled for a linear sequence of sample images for each culture topic. (Figure 2) Likewise, we planned a glossary but temporarily postponed its development. Both components are priorities for the future.

sample images

Figure 2.Topic Images Sample Screen

3. Structure of the Online Course
3.1 Content
The PVC Online interface structure gives students access, at any time of the day and from any campus or remote location, to guideline information about the course of study for the semester, to comprehensive materials for each day's topic, and to their status with regard to grades and assignments.

3.2 Overall Content
A course description displays a syllabus; gives who, what, when and where information; outlines course goals, policies, and procedures; and instructs students how to navigate the site. A calendar (Figure 3) and a topic list display the schedule for the semester in two different formats. A status chart gives students individualized reports of the entire semester's quizzes and assignments - what has been completed and when each was submitted, and what the cumulative grade is to date.

Sample Course Calendar Page

Figure 3. Sample Course Calendar Page

3.3 Culture Topic Content
For each culture topic there are a brief introduction to the topic, reading assignments, map, timeline, images, and a quiz. Students may also, at any time, access global information for the entire semester. For example, students may toggle between the global map and current topic map or between the timeline (Figure 4) of world cultures and the timeline for the current topic. The images on the web site are selected examples for which the instructor holds the copyright. For each culture topic, students are required to complete a quiz before they come to class. Quizzes are comprised of four randomly generated questions based on the assigned readings.

Sample Topic Timeline Page

Figure 4. Sample Topic Timeline Page

3.4 Synthesis Topic Content
For each synthesis topic there are a brief introduction to the discussion topic for the day, a reading assignment, and a description of the written assignment to be electronically submitted the following week. At the end of the description of the assignment are instructions for submitting assignments electronically.

3.5 Interface
When students log in to the PVC Online web site, the first page that comes up answers the question, "What are we doing today?" The interface is designed with buttons across the top for course overview information as well as navigation backwards and forwards through the semester's topics. In the left column are buttons specific to the topic for the day, with the addition of the ever-present global map and timeline. All of the information that students call up through navigation of the site appears in the central frame of the window.

The global timeline (Figure 5) provides the students with a visual representation of how the various cultures relate to each other with respect to chronology. In addition, the timeline can be used to navigate the culture topics.

Sample Main Timeline Page

Figure 5. Sample Main Timeline Page

3.6 Header Row
Buttons in the upper left of the header include Course Description, Calendar, and Topics List. In the upper right are Your Status, Today and Logout. The upper center frame displays the title for the current topic with left and right arrows to move forward or backward through the topics for the semester. Students can work their way through the entire complexity of the web site and return to the topic for the day simply by clicking on Today.

3.7 Left Column
There are two sets of buttons that distinguish between students' interactions with web site materials for lecture/culture topics and for synthesis/discussion topics - reflecting the differences in format between these two kinds of class sessions. The set of buttons for culture topics includes an introduction, readings, map, timeline, images and a quiz to be completed before class. The set for synthesis topics includes an introduction, readings, and a written assignment for six of the semester's topics.

3.8 Assessments
The assessment capabilities of Carnegie Mellon Online proved to be the most powerful asset of this undertaking. Students were prepared for class, made substantive contributions to class discussions, and excelled on quizzes, tests and written assignments. The primary catalyst was the requirement that students take a short quiz based on the reading assignment before attending class. Like paper-based quizzes, the instructor had to decide what kind of quiz was appropriate for testing the students' comprehension; she chose to use a four question, multiple-choice and true/false quiz.

If the quiz were a paper quiz administered in class, a time consuming task from two perspectives - taking up valuable class time and taking up the instructor's time for grading - an instructor would simply have written four questions per topic and photocopied the quiz for each student. However, since the Online technology can generate individualized, randomly generated quizzes for each student, the instructor divided each quiz into four sub-topics and created three multiple-choice questions for each sub-topic. For each student, the system generated one question from each of the four sub-topics, randomly permuted the answers, and generated the HTML form for the quiz.

In addition to creating the quiz content, the instructor had to determine the policy for the quizzes, such as:

  • When is the earliest time that a student can take a quiz?
  • When is the last time that a student can take a quiz?
  • When is a quiz due? What is the penalty for completing it late?
  • May a student choose to retake a quiz? How many times? Is it the same quiz or another randomly generated quiz? What is the penalty?
  • May a student see any of the web site's other materials during a quiz?
  • What feedback does the student get?

A quiz is due thirty minutes before the class meeting and may be started up to one week before the class. This discourages students from finishing a quiz just as the class starts. Late quizzes are penalized 10%. A quiz may be taken as late as two weeks after the class meeting, which prevents students from waiting until the end of the semester to do all the quizzes. There is a 10% deduction for a late quiz. Students who do poorly on a quiz may elect to take a second randomly generated quiz, which also carries a 10% penalty. Students are allowed to see all of the course's online material during a quiz.

After submitting a quiz, a student is not shown the correct answers or even the questions. The student's immediately generated status page (Figure 1) indicates only how many questions were answered correctly. This policy was instituted because, at this point in time, everything displayed on the web can be printed locally. The policy discourages students from compiling printouts of questions and answers and sharing them with current or future students. The instructor, however, has access to the individual questions and the student's answers for each quiz. Students may discuss individual quiz (Figure 6) results privately with the instructor, but have not expressed much need to exercise this option.

The instructor realizes that these policies do not prevent students from working together on quizzes. In fact students asked if it were permissible for them to do so. It was the instructor's intention that students read the text before coming to class. What better learning situation could there be than for students to read the material, discuss it among themselves, and collaborate on answers to quizzes? The security system does, however, discourage students from giving their password to one student to take quizzes for the group, and the individualized quizzes make it harder for students to copy other students' answers. These measures brought about the intended goal of these quizzes, that is, encouraging students to read, think, and talk about the material before coming to class, whether they take the quizzes individually or in groups. The outcome was more significant than expected.

Sample of student feedback

Figure 6.Sample of student feedback

This feedback is only available to the student if they request a meeting with the instructor.

3.9 Policy for Written Assignments
Students are also required to submit one written assignment per week for the first six weeks of the semester (and work on a long project/paper for the latter half of the semester). Written assignments are submitted as a file upload through the web interface. As with the quizzes, the instructor had to determine the policy for these written assessments.

Late submission of assignments is penalized 10%. The assignment submission page on the web site is available from 30 minutes after the class discussion until two weeks after the due date. This prevents students from submitting all assignments at the end of the semester. However, at any point during the semester, students may plan ahead and read the instructions for all of the written assignments for the semester.

It was decided that essays could not be rewritten and resubmitted. The system allows a student to submit a single essay. The instructor occasionally allows rewritten assignments, but they are handled on an ad hoc basis. Just as students have access to all the course materials during a quiz, students are allowed to see all of the course's online materials while working on an assignment.

After a student submits an assignment, an acknowledgment is given that the file has been accepted.

Students receive written feedback on their papers from the instructor. The policies for both written assignments and quizzes were established for the normal sequence of events that students encounter in a one-semester course. It was decided that anomalous situations that arise from time to time would be facilitated through an instructor's interface that allows a change in due date for illness and a change in quiz score for a well-argued point. The instructor's interface is described below.

3.10 Management Tools
Through a separate administrative interface, the instructor has electronic access to statistical information about quizzes, to individual student's status reports, to grade summaries for the entire class, and to screens which allow changes and corrections to accommodate idiosyncrasies, such as student illness, adding and dropping the class, and re-writing essays.

For the first time, the instructor is able to do something she has always wanted to do, but didn't have the time to do: compile a statistical analysis of students' answers to quiz questions. (Figure 7) The quiz summary displays in graphic format how many students scored 4 out of 4, 3 out of 4 and so forth; and displays how many right and wrong answers there are for each question. It gives her the opportunity to assess the clarity and fairness of questions as well as students' understanding of the material before she enters the classroom.

Since Carnegie Mellon Online compiles and reports each student's cumulative numerical score and percentage, reporting mid-semester and final grades for 55 students is much less labor intensive than in the past.

The instructor can change an individual student's deadline for quizzes and assignments in case of illness; change a quiz score after a well-argued rationale for a wrong answer; change a grade for a re-written essay; and add or delete students on the class roster.

Sample Faculty Quiz Analysis Tool

Figure 7. Sample Faculty Quiz Analysis Tool

4. Lessons Learned and Future Plans
4.1 Rethinking assessments.
Much of the time spent in moving the course materials to the Carnegie Mellon Online structure was spent in rethinking assessment tools. In previous years, the instructor would use less frequent tests based upon both the lectures and readings. With the new automated system, the instructor chose to use quizzes before class meetings to ensure reading. But this brought a new problem. Test questions from lectures and readings, which address issues of comparison across time and cultures, were missing. The second iteration of the course for the Spring 1999 semester includes classroom tests. Our future expectation includes new creative kinds of visual/verbal assessments as part of monitored web-delivered, mid-semester and final tests.

4.2 Student Reaction
Spring 1998 students were excited to be a part of this project. They showed we're-all-in-this-together enthusiasm, exhibiting patience when there were computer glitches, studying together, and talking about art history. Other faculty who taught the same students related that students thought it was great to have "all this stuff" available to them. Though these students were thoroughly assessed through twice-a-week quizzes and weekly essays, there were no complaints about the workload.

The instructor noticed a tremendous impact on student learning. Though it is fairly easy to assess students' command of the factual material of a course through objective questions, it is much more difficult to assess other expected learning outcomes such as critical reading, listening, and thinking skills. Frequent, short written assignments are crucial. So are discussion sessions. In large lecture classes, faculty who acknowledge the importance of writing and small group discussions, and who devote time to grading students' written assignments, frequently cannot devote the extra time to grading numerous objective tests that make students accountable for keeping up with readings. Anecdotal information from the instructor suggests the powerful impact that interaction with web materials, especially the quiz assessments, had on the quality of discussions, an essential component of humanities and social science instruction. In the last synthesis session, the instructor asked students to discuss patronage in all of the cultures and time periods covered in the course. She went around the discussion table designating for each student which culture or time period he or she was to cover. No one skipped a beat. Each and every student had something substantive to say. This doesn't happen in freshman courses where students don't do the reading.

4.3 A Priori Content
n the first version of Carnegie Mellon Online, all content had to be entered before the beginning of the semester. This placed a burden on the instructor to plan the course content, assessments, and policy well before the semester started. For this art history course, the course interface and policy were set about six months before the semester; the majority of the content and assessment questions were in place about four weeks before the semester, with the remainder of the time spent refining the content and testing the system.

On the other hand, the benefit of this process is that the instructor is free to spend more time with the students and to plan individual lectures. Furthermore, this allows students to see the content for the entire semester at any time during the course. It also prevents students from having an inconsistent experience with the course, namely, the possibility of seeing material, and then having it be changed by the instructor without the student's knowledge.

Though the benefit outweighs the burden, many instructors may be unable or unwilling to commit so much planning time before a class starts. The new version of Carnegie Mellon Online allows an instructor to add and update the course's content during the semester.

5. Conclusion
5.1 Current Modifications and Future Plans

The current version of this course includes the addition of classroom tests at three points in the semester. A future version of this course will have several more enhancements including a glossary, assignments based upon the web materials, and web-delivered mid-term and final exams with new kinds of visual/verbal assessment tools.

The most important addition will be an image database to replace the current linear sequence of sample images. Professor Schmidt has a comprehensive database of digital images relevant to this course. We will create an interface that will allow students to search the database based on several descriptors of the artifacts, including artist, location, period, style, theme, and type of image. The students will then have instant access to the found images. One of the exciting aspects of this database is that the term "image" is not limited to a single static image; instead, it includes any of the media that can be delivered over the web, including movies, three-dimensional models, and virtual reality models.

For all of the participants in this project, the journey has been an intensive collaboration to integrate pedagogy and technology for the creation of a comprehensive and effective online teaching and learning tool. We are enthusiastic about the results and are planning improvements for the future.

5.2 Acknowledgments
The Carnegie Mellon Online project is funded by Carnegie Mellon University.

6. References

[Rehak 97a] Rehak, D. R. (1997). "From Ivory Towers to Ethereal Webs" in Computing Futures in Engineering Design, Clive Dym, editor, Center for Design Education, Harvey Mudd College, Claremont, CA.

[Rehak 97b] Rehak, D. R. (1997). "Database/Web System for Customized Instruction" in "Educational Multimedia and Hypermedia, 1997", Proceedings of ED-MEDIA-97 & ED-TELECOM 97, Association for Advancement of Computing in Education, Calgary, Canada.

[Rehak 97c] Rehak, D. R. (1997). "A Database Architecture for Web-Based Distance Education" in "WebNet 97", Proceedings of WebNet 97---World Conference on the WWW, Internet, & Intranet, Association for Advancement of Computing in Education, Toronto, Canada.

[Rehak 97d] Rehak, D. R. (1997). "Carnegie Mellon Online: Web-Mediated Education" in "FIE '97", Proceeding of Frontiers in Education Conference, American Society of Engineering Education/IEEE, Pittsburgh, PA.

********** End of Document **********

IMEJ multimedia team member assigned to this paper Ching-Wan Yip