Assessment BannerIn today’s educational landscape, course-specific assessment is required. Regardless of whether you are a corporate trainer teaching an eLearning course or a tenured professor at a state college, every course you teach should have clear, measurable goals. Most often, these are called “Learning Outcomes” (LO)or “Student Learning Outcomes”(SLO).

Here’s examples from two of my most-taught courses. The learning outcomes were determined a few years back during state-wide meetings between educators and education administrators. These can be found at the Texas Higher Education Coordinating Board’s Annual Almanac web site.

SPCH 1315: Fundamentals of Public Speaking

Upon successful completion of this course, students will:

1.Demonstrate an understanding of the foundational models of communication.

  1. Apply elements of audience analysis.
  1. Demonstrate ethical speaking and listening skills by analyzing presentations for evidence and logic.
  1. Research, develop and deliver extemporaneous speeches with effective verbal and nonverbal techniques.
  1. Demonstrate effective usage of technology when researching and/or presenting speeches.
  1. Identify how culture, ethnicity and gender influence communication.
  1. Develop proficiency in presenting a variety of speeches as an individual or group (e.g. narrative, informative or persuasive).

SPCH 1311: Introduction to Speech Communication

(This is often known as the “survey course” for people in my field.)

Upon successful completion of this course, students will:

  1. Apply the principles of human communication including:
  • perception,
  • verbal communication,
  • nonverbal communication,
  • listening, and
  • audience analysis.
  1. Demonstrate how to establish and maintain relationships through the use of interpersonal communication.
  1. Apply small group communication skills including:
  • problem solving
  • group roles
  • leadership styles, and
  • cohesiveness.
  1. Develop, research, organize, and deliver formal public speeches.
  1. Recognize how to communicate within diverse environments.

 

Now, here’s where it gets tricky. You are expected to demonstrate two things: That you teach all of the listed outcomes, and that you identify areas of improvement. So, it’s not good enough to say, “They took my final exam, which covered all of those outcomes. There. Done. That’s my assessment.” No, you need to also measure how each outcome meets a certain minimum standard: a benchmark.

Further, some of you learning outcomes might be, shall we say, ambiguous. For example, look at “Recognize how to communicate within diverse environments.” Almost every word of that statement is open to interpretation. “Recognize?” Just recognize? What about understand or adapt to? What do they mean by “diverse”?

In addition, there is the problem of operationalization, which is a fancy word meaning, “how they heck can you measure that?” Take, for example, the SLO “Demonstrate how to establish and maintain relationships through the use of interpersonal communication.” How can I reasonable measure whether or not a student establishes and maintains interpersonal relationships? More importantly, how can I measure that I taught those skills to the student?

Take a deep breath because it gets more complicated. Let’s assume SPCH 1315 is taught every semester in multiple sections, with multiple faculty. Unless your department has a rigid, we-don’t-care-about-academic-freedom approach to teaching, you are probably going to have many different assignments and teaching styles in play. For example, think how many different types of “informative speeches” there are. Consider the SPCH 1315 outcome, “Identify how culture, ethnicity and gender influence communication.” Now imagine how many different ways there are to teach that in a course. So now you have the problem of collecting data across sections, which leaves the inescapable conclusion that there has to be common assignments across sections. One more point: Add online and partial online courses to the mix, and you find the complexity grows.

Finally, there is the problem of the sheer number of assessments. For SPCH 1311, there are seven assessment categories, with a total of 12 specific outcomes. Does that mean you should create and administer 12 assessments every time that course is taught? I hope not. In my case, we rotated assessments across three semesters: first semester was collecting data, second semester was analyzing data, third semester was use-of-results and assessment revision. If you are smart, you will stagger your assessments so that you are only assessing a maximum of three outcomes in any given semester.

Also, if you are clever, you will create assessments that overlap learning outcomes. For example, perhaps collecting and analyzing students’ persuasive speeches might satisfy learning outcomes dealing with creating speeches, doing critical thinking, and even tie-in to your school’s QEP!

After many years and many meetings, my colleagues and I at Brookhaven College came up with a spreadsheet to help with our assessment plans for SPCH 1311:

 

SPCH 1311 Assessment Matrix

SPCH 1311 Assessment Matrix

 

Notice how all five learning outcomes are listed across the top. In addition, four Core (GenEd) Objectives (critical thinking, communication, teamwork, and ethical decision making) are listed.

Now, notice on the left column how all of those outcomes are wrapped into five assessments: a listening assessment, an essay (speech teachers were thrilled to have to also teach composition), a teamwork evaluation (conducted after a required group project), a formal speech, and a PowerPoint presentation. Actually, the PowerPoint and speech could be combined, so it’s more like four total assessments, which were rotated throughout the academic year (see above).

Here are some examples of the assessments:

thumbnail of small-group-assessment

Small Group Assessment

thumbnail of persuasive-speech-assessment-rubric

Persuasive Speech Assessment

 

So, how do you actually create good assessments? By a lot of Googling and long, sometimes heated meetings. I’m going to gloss over this now, but take a look at the “External Links” page. I especially recommend the NCA and AAC&U web sites. In the future, I plan on creating a page here just about creating assessments, and I hope you will share some of your examples, too!

After you create the assessments, you gather the data. Find a statistician or social scientist in your midst, and decide on your sample size. Schedule when the assessments are due. Thank your adjuncts for the extra work involved.

Prepare for a lot of work in the analysis phase. Find the person who knows Excel the best, and put him or her in charge of collecting all the quantitative work. Or, have your institutional research person set up a Qualtrics site. Have your resident rhetorician handle any qualitative/content analysis work. For speech analysis, be sure to have a coder reliability training session.

Finally, you have your data, and you’re done, right? NO! You need to do two things: First, you need to write up your reports and submit them to your dean or assessment manager. Next, you need to plan for “use of results.” Set improvement goals, and have meetings so that everyone is on the same page. For example, one year we noticed that students needed more time management skills for their group projects, and we added more class time for that.

Done yet? Almost. You need to repeat the process periodically. This will help you determine improvement trends (often required for accreditation), and it will provide you with a longitudinal database.

I hope this helped you, but obviously I left a lot of information out. I hope that you will contribute to our discussion page. Share your mistakes, triumphs, and questions!

Assessment Links