Welcome to Wired Ivy Footnotes! Clippings from a previous episode, mulched with commentary from Dan and Kieran, to help your online course design and delivery skills grow.
KIERAN 00:15 Listeners who’ve followed Wired Ivy for a while now will know Dan and I are firm believers that course design needs to begin with the learning objectives, regardless of academic level and mode of delivery. Any test ride of course technology, content, activities, and assessment options that takes place before the learning objectives have been properly groomed is simply putting the cart before the horse.
And yet… It wasn’t until the previous episode went live that we realized we’re just as capable of accidentally flipping the proper order of carriage and Clydesdale… in other words, we failed to provide any guidance on how to develop learning objectives sturdy enough to carry all those other course components, in Episode 36 – Made to Measure, or any episode so far. This Footnotes episode is an initial attempt at closing the door… now that Black Beauty is no longer in the barn.
MUSIC STING
KIERAN: You know, I’m realizing that I’ve had a huge blind spot when it comes to course design. I couldn’t count how many times I’ve coached an instructor that the learning objectives should inform every other choice they make – content, activities, technology – and yet I don’t think I’ve ever included assessments in that list.
DAN: The learning objectives really should inform the assessment. Because in the end, what you’re trying to do is assess a mastery of a subject, what you’re trying to do is assess the learning outcome, the improvements that have been made in knowledge, and they really want to be tied together.
MUSIC STING
KIERAN 01:42 Dan, they say hindsight is 2020, and now that I’ve had a chance to listen to our last episode, Made to Measure, I realize we went straight into the importance of making the learning objective central to everything you do, but I don’t believe we’ve ever provided any guidance for how to develop those all important objectives.
I’m positive you’ve given this some thought. I’ve just never asked you to share your thoughts on that, so here’s your chance. When you sit down to create learning objectives for a new course, or update them for an existing course, walk us through your process for developing objectives that can integrate learning and assessment of learning.
DAN 02:16 You know, Kieran, I had a similar thought afterwards. It’s like, well, we went straight to the finish line and congratulated ourselves, as far as having this wonderful piece of advice, and we didn’t remember that in academics you always have to loop back and reiterate.
KIERAN 02:31 Yeah, it’s as if someone asked us, “How do I get to St. Louis from Mechanicsburg?” and we went straight for the turn-by-turn directions without ever asking, “Do you have a car?”
DAN 02:39 Yes, exactly. If we want the learning objectives to be integrated with the assessments and activities, then the assessments and activities are also going to inform how we articulate those learning objectives.
To the real point of your question of how do you create learning objectives for a course… there are a couple of regulatory things that you have to do when you do learning objectives. For one, for a course to be approved by the university curriculum committee it’s got to follow a certain format. They’re going to be looking for learning objectives that fit a pattern they recognize.
KIERAN 03:09 You might get a list of specific verbs to use, right?
DAN 03:12 Yes, exactly.
Most universities and colleges that I’ve worked with use Bloom’s Taxonomy. They want the verbs from the seven different categories in the taxonomy that show a progression of knowledge.
The other thing which is interesting for us in the online world is a lot of times our courses go through Quality Matters. I’ve had a chance to talk about the course learning objectives, as well as the lesson learning objectives with instructional designers who are very well versed in the Quality Matters rubric. That puts a certain constraint – is that the right word? – that puts a certain expectation of what the learning objective needs to do. You’ve worked with Quality Matters before…
KIERAN 03:49 I would go a little further and say it was designed to serve a particular purpose – to help online courses gain legitimacy in the eyes of the broader academic community, and make sure that they’re meeting a certain standard of rigor. And it does pretty well at that, I think, but I can’t say that it fosters innovation.
DAN 04:12 Yeah, I think exactly right.
My understanding of Quality Matters is, it’s put in place as a standard. We have to consider Quality Matters when we write our learning objectives.
KIERAN 04:20 It’s a good process to go through, even if that’s not exactly what you end up giving students as their syllabus for that particular semester.
DAN 04:29 I found that the process of working the learning objectives through the Quality Matters process was useful because of the dialogue I got to have about them. Someone helps you articulate specificity, for example. It’s not going to be the thing that necessarily creates an innovative approach to integrated objectives, activities, resources, assessment, so that’s really on us. We have to design courses to do that.
I do that in an iterative process. There’s really a hierarchy of outcomes and objectives that you deal with in a course. You have the course learning outcomes. How they’re worded in my courses – “at the end of this course, the student will be able to blank” – that’s an outcome. Course learning objectives, your CLOs, tend to be fairly broad. You might have three, five, generally not more than seven.
Then you build out your lesson learning objectives. What activities are they going to do that are going to push their knowledge forward? From there you built out your activities and your assessments. The assessment is the proof of the pudding. How are the learners going to demonstrate what they’ve accomplished? Is this the outcome we wanted?
I think if you’ve integrated your course in the first place, from the objectives to the activities, to the resources, to the assessments, then you’re not going to have so much of a issue… that’s a little abstract. Does that make sense from a general process?
KIERAN 05:49 The way I would rephrase what I’ve just heard you say is, you start with the class learning objectives. Those inform the individual module learning objectives. The learning objectives then are supposed to inform the content, the activities, and the assessment but by the time you get to the assessment, you might realize that the course learning objective or the module learning objective don’t lend themselves to what you want the student to come out of the course with. Is that…?
DAN 06:18 Yeah. See, this is exactly why we need to edit and review everything that we do, because it’s better when someone else says it. Yes.
The one caveat to that would be that the course learning objectives should be pretty well thought through. So if you get down to the granular level where the assessment is not reflective of a course learning objective, then you’re probably writing the wrong lesson. We’re really talking about the assessment being able to feed back into an activity, whether it’s a project, a paper, a lesson, something at that, that midstream level.
KIERAN 06:51 Okay. So it’s really more relevant to the learning objectives for the individual modules then for the course learning objectives. As we take courses through an approval process the focus is going to be more heavily on the CLOs, but teaching it’s going to be more focused on the individual learning objectives for each module. Do I have that right?
DAN 07:12 I would say that is, is spot on. It’s also logical if you think about what we talked about in Episode 36, that the assessments really want to be more incremental with the flow of activity in the course and the lessons. So instead of having these massive summative things, what we have are consistent assessments as the activities progress. We’re looking at mastery as it develops, so it also makes sense that those assessments are connected to the lesson outcomes or the project outcomes. Yes.
Instead of having these massive summative things, we’re looking at mastery as it develops, so it also makes sense that those assessments are connected to the lesson outcomes or the project outcomes. #highered #onlinelearning #virtualclassroom
Tweet
KIERAN 07:42 Let’s move from the theoretical to something more concrete. Can you give me an example of a course objective that has informed a specific learning objective, and then how you integrate the content, and activities, and assessment.
DAN 07:57 Sure. In fact, when you sent me a note saying, “we should talk about this because we didn’t really cover it in the last episode”, I thought, “oh, I really need to go back and look at some of my own courses!”
So I looked at one particular course, Sustainability Systems, and the outcome that I thought, that’s pretty good, is “construct conceptual maps of basic components of key sustainability systems.”
KIERAN 08:18 Can you give us some examples of what you mean by key sustainability systems? Since not everybody will have the same disciplinary context.
DAN 08:25 Great question! We’re talking about a combination of environmental, global, and social systems as well, connected with natural resources. Other people when they talk about sustainability might be specific to an industry, for example. For us, we’re talking about water, climate, agriculture, energy, land use poverty, urbanization, global material flows, biodiversity.
KIERAN 08:45 That’s helpful. So you’ve got this as a course learning objective. What’s the next step?
DAN 8:49 So, the next step, then, is to figure out how, in the various lessons, we’re going to have this learning outcome reinforced. How am I going to develop opportunities for them to practice and master this particular learning outcome?
The next step is to figure out how we’re going to have this learning outcome reinforced. How am I going to develop opportunities for students to practice and master this particular learning outcome? #highered #onlinelearning #virtualclassroom
Tweet
In this full semester course, there are 14 lesson plans and three projects. In more than half of the lesson plans we actually build up and create the opportunity to diagram maps of specific systems. So there’ll be case studies and they’ll actually be drafting out maps.
Now, a lot of people who do systems work are really looking for algorithms and trying to quantify it. Our outcome is the conceptual map. We’re really trying to understand what the relationships are between elements within a system, and what the flows are between elements in a system, and what the feedback loops are. That’s a pretty big challenge to be able to: A) understand it; and B) to map it. Once you start to be able to map it, you start understanding it better as well. That’s a very reinforcing process to do this as a learning activity.
KIERAN 09:46 This is a good example because I could see where this would be challenging to assess whether or not the student has achieved the learning objective. It’s not like just having them demonstrate that they know how to use a pipette.
DAN 09:59 You’re right, it’s not. And the other thing that’s really interesting is, even though I’m dealing with graduate students who are usually active already in their professions, it’s something that’s very foreign to them.
KIERAN 10:10 Interesting!
DAN 10:11 Many of them have never really worked or thought about organizing relationships in a multivariate systems approach. I start with an early lesson, not even with the diagram. It’s: Write me a narrative of how you see the elements in this land use system connecting to each other. I don’t need arrows, I don’t need boxes, I don’t need flows – just tell me how does land tenure, and soil, and water, and agriculture – how do they connect? If you can tell me the stories about how these various elements on the land connect, then you’re starting to conceptually map the system.
And then I give them examples of different ways you can map a system, because they come from different backgrounds and the way that they approach it in their profession might vary from their colleagues. By the end of the semester they’re able to create fairly sophisticated diagrams.
I look at the diagrams they have done, but they’ve posted these publicly to everybody in the class. So the real feedback comes when they get comments from their peers, and then they also comment on their peers’ work because they really learn from that kind of conversation. The nice thing, for me as a learning outcome, I have the actual product of their work. I have the diagram, but also, because it’s posted in a discussion format, they’ve been defending it, for lack of a better word.
KIERAN 11:29 Start out telling a story, to yourself as much as to anybody, just so that you can organize your thoughts. From there you start thinking about how you might be able to represent that in ways that could be more easily envisioned, as opposed to just a linear string of concepts. And then you put your work out for others to comment on – that’s where you start to see if how you’ve envisioned it is also understandable to folks who are not in your brain, which is always a helpful thing.
Are they getting assessment from you at each of those levels, or is this one of those situations in which they’re getting feedback, they’re getting input, but there is a larger point of assessment with a higher stake.
DAN 12:12 At each point they’re getting an assessment for having done that step of the work. They’re getting incremental assessment as we go along.
KIERAN 12:20 Okay. Since this is a Footnotes episode we’re going to keep it short. We’re not going to provide a ton of other examples. But, looking at this specific course learning objective, and the specific learning objective you’ve just described, how would students articulate the skill and knowledge set they’ve gained from this? And is that something we can use to better articulate the learning objective and its value to the students?
Ask yourself how would students articulate the skill and knowledge set they’ve gained from an activity, and is that something we can use to better articulate the learning objective and its value to the students? #highered #onlinelearning
Tweet
DAN 12:45 Yeah. This is a fun question because we’re about the fifth week in the semester – and this is where it generally happens but it happened already this week – where they start talking amongst themselves, in the open discussion forum, about how is this affecting your brain? And are you doing anything at work? One posted just in the last couple of days who said, “Oh yes, I already sent some stuff to my boss because I was seeing these connections that we’ve never really looked at.”
KIERAN 13:11 Excellent. Yes!
DAN 13:11 “So I sent a memo to my boss about this!” So, so we don’t have to wait for the 14th week. It really is an opportunity for them to put their learning in action right away. And I think the simplest thing on the resume is going to be that they can do systems thinking, and if they wanted to, they would have an artifact from the course because they could show a map that would demonstrate that. Some of the opportunities in their learning would be for them to actually map something of interest to them.
KIERAN 13:39 So I’m going to close with a question that’s specifically for you, Dan. Are you making changes as a result of having articulated, yourself, the need for integrating content and activities and assessment?
DAN 13:52 I did. I actually went back… As I’ve said before, I’m always looking to try to make the courses better and more efficient and have, quite frankly, denser value for the learners. On the lesson outcomes I did find a couple I thought, “Ooh, that’s squishy sounding, what does that mean? I think I can do better on some of them.”
KIERAN 14:09 We’re both writers, and a trick that writers use when editing their work is to start at the end and go backwards. I could see that being a really useful thing in learning objectives. Just as one example – to see if you are assuming that students have reached a certain understanding to do an assignment at the end, and then you realize you never actually gave them that in the earlier learning objectives. Look at it from a different perspective so that your eyes can’t just fill in what you intended to do instead of what you actually wrote.
DAN 14:37 First of all, I have to say that, as a writer, I love it when you can give an example that’s both metaphor and literal at the same time. That’s pretty impressive. Looking forwards and looking backwards, I think is a great way of doing that reiterative process, that looped process of learning.
MUSIC
KIERAN 14:57 Let’s hear what you have to say! Send us your questions, comments, and suggestions. You can leave a voice message at speakpipe.com/wiredivy, or send us an email to kieran@wiredivy.org — Kieran is spelled K-I-E-R-A-N, or Dan@wiredivy.org — Dan is spelled D-A-N.
And help Wired Ivy grow by sharing, subscribing, rating, and reviewing us on your favorite podcast app.
GUEST BIO
Wired Ivy co-host Kieran Lindsey is a former online graduate student, occasional online educator, and current Program Director for Virginia Tech’s Online Master of Natural Resources. The Online MNR has has evolved from in-person instruction to online, and under her guidance has grown to be the university’s 2nd largest virtually delivered graduate degree. Kieran serves as a special consultant to VT’s Graduate School Dean on virtual programming for non-traditional student audiences, and has provided ad hoc advising on the development and delivery of online programming to colleagues at various institutions. Kieran is also an urban wildlife biologist (Texas A&M – BS, MS, PhD), writer, blogger, regional Emmy Award recipient (documentary feature), and now a podcaster, too, as well as personal assistant to a ferociously cute wire fox terrier named Dashiell Riprock (aka Dash).
Daniel J. Marcucci advocates for sustaining a livable, rich biosphere—because the alternative is scary. He teaches Sustainability Systems, Coastal and Marine Systems, and Urban Water Systems courses and leads Global Study trips. Dr. Marcucci has two decades’ experience in higher education as an environmental educator, the last several years focusing on creating supportive asynchronous online learning communities. He has published in numerous environmental planning journals. He also has extensive experience as a regional and environmental planner. His current research explores individual landscapes and landscape as an integrative and holistic concept. Dan is a member of the American Institute of Certified Planners.
REFERENCE LINKS
- Bloom’s Taxonomy has been revised over the decades to reflect new research. Many colleges and universities publish guidances on how to use it.
- Quality Matters (QM) offers rubrics and processes for online course design.
- Donella Meadows was a seminal thinker about systems thinking and mapping.
We welcome your questions, feedback, and suggestions for future episodes. Join our Wired Ivy LinkedIn Group, tweet us @wiredivy, or look for the bright blue Talk to Wired Ivy tab on the right side of this screen to leave us a voice message!
Wired Ivy is wholly owned by Kieran Lindsey and Daniel Marcucci, and we are solely responsible for its content. Views expressed in this podcast and affiliated media are those of Kieran, Dan, and our guests, and do not represent Virginia Tech or any other institution. Our audio engineer is Star Path Images, and a license for our theme music, Breakfast with You, was purchased from SmartSound.
Want to be notified when whenever new podcasts and other content are published? Join our mailing list!