New paper: Examining the technological and pedagogical elements of select open courseware

First Monday journal logo

I’m pleased that my colleague Dr. Michael McNally (Ualberta SLIS) and I have published a new paper in First Monday titled “Examining the technological and pedagogical elements of select open courseware.”

The paper is non-traditional in both its focus and method, but it’s a step toward answering some research questions Michael and I have. Specifically, what does openness in education mean, what are the factors that constitute openness, and what does openness look like on a spectrum? In our previous paper (a thought experiment) we addressed these questions from a philosophical perspective. In this new paper, we wanted to know how ‘open’ open courseware (OCW) actually is. We focused on open courseware because we feel it’s an important component of open education, but it’s often overshadowed by open textbooks. This was a small study, where we randomly selected ten open courses – five from MIT and five from TU Delft. The sample is too small to generalize to all open courseware. Analyzing all the course materials in this sample was challenging, so a larger research team (or narrower focus) would be necessary to replicate this approach across a larger sample. We selected courses from these two institutions because of their similar focus on STEM; though, both institutions do have courses from other disciplines. Ultimately, we got a good mix of courses after running our randomizer formula in Excel. We analyzed the openness of each course using the framework we proposed in our previous paper. So, this paper is part analysis and part proof of concept. Below are our research questions from the paper.

Question 1: Based on the eight factors of openness in the framework, how open are the sampled OCW?

Question 2: Are the sampled OCW adequately designed for educator reuse or adaptation?

There are quite a few findings, but I’ll give a couple of highlights. First, we found the framework (called ‘Open Enough’) functioned reasonably well as a rubric for OCW evaluation, though it had some shortcomings. We proposed a revised version of the framework in the discussion. We also found that despite the name ‘open courseware’, the openness of these courses was a mixed bag, at least when evaluating them using our eight factors of analysis. Our results and discussion sections are long, but the paper includes a summary of the findings.

The level of openness across the sampled OCW was inconsistent. From a copyright perspective, the OCW were mixed as they employed an institutional licence across all content — a CC-NC-SA licence. In terms of adherence to accessibility standards, MIT courses were most open while TU Delft were closed. We determined accessibility openness by reviewing each OCW platform’s statement regarding which accessibility standards were observed. From a usability perspective, we concluded that TU Delft has the more modern interface, but MIT courses were more navigable. All courses (except one) were categorized as closed since the course content was offered in one language. The majority of support costs (course readings) were proprietary and many of the courses relied heavily on paid books. Two TU Delft courses were the exception to this rule; one course’s readings were completely custom (presumably written by the instructor), and another course made its proprietary ebook openly available (albeit with a closed licence). All of the sampled courses were very open in terms of digital distribution. Each course could be located using several federated OER search engines. Surprisingly, the sampled courses were categorized as closed in terms of file format, as all relied heavily on PDF content. We struggled with the cultural considerations factor, given the large volume of content and the factor’s subjective nature. Courses were split between closed and mixed/most open. Course content dictated this conclusion, as STEM courses materials were determined to be more culturally applicable, broadly speaking.

Given the limited editability of the course materials, we concluded that the sampled OCW were better suited for learner consumption rather than educator reuse. The lack of editability was surprising and severely limits these courses’ utility outside their originating institutions. These findings highlight the importance of prioritizing OCW for editability and adaptability, in addition to making them discoverable to the broader community.

Since writing this paper, MIT OpenCourseWare has redesigned its website which presents more opportunities for analysis, particularly usability and discoverability. This was an interesting project to be part of and I’m glad it’s finally published. I’d like to thank First Monday for publishing this piece, and the paper is available as an open access article from the journal website.

Citation: Christiansen, E., & McNally, M. (2022, Oct 5). Examining the technical and pedagogical elements of select open courseware. First Monday, 27(10). https://doi.org/10.5210/fm.v27i10.11639

Manuscript and figures

Italian immigration to Kelowna and its lasting legacy

Image of Kelowna Italian families during their 1st annual Italian picnic. The image was taken from the Kelowna Italian Club website.

For folks interested in Canadian History, my article “Italian Immigration to Kelowna and its Lasting Legacy” has been published by the Kelowna Italian Club! Just in time for Canada Day.

I originally conducted this archival research 10 years ago when I was a student at The University of British Columbia Okanagan. I received an undergraduate research grant in Summer 2009 which allowed me to study Italian immigration to the Central Okanagan and West Kootenays during the 1880s and 1920s. This piece talks about the lasting legacy of Italian immigration, and the important role played by some of the early Italian families, in the Kelowna area.

I want to give a big thanks to Archivist, Tara Hurley at the Kelowna Museums and Don Rampone from the Kelowna Italian Club for helping me update this research with additional historical documents and details!

You can read the full article on the Kelowna Italian Club website: https://www.kelownaitalianclub.com/kelownas-first-families/italian-immigration-to-kelowna-and-its-lasting-legacy

New publication in First Monday: “Open Enough?”

Open access logo

My colleague Michael McNally (University of Alberta SLIS) and I have published a new paper in First Monday titled “Open enough? Eight factors to consider when transitioning from closed to open resources and courses: A conceptual framework.” Having worked on this paper over the past year, I’m very glad to see it in print.

Abstract: Transitioning from closed courses and educational resources to open educational resources (OER) and open courseware (OCW) requires considerations of many factors beyond simply the use of an open licence. This paper examines the pedagogical choices and trade-offs involved in creating OER and OCW. Eight factors are identified that influence openness (open licensing, accessibility and usability standards, language, cultural considerations, support costs, digital distribution, and file formats). These factors are examined under closed, mixed and most open scenarios to relatively compare the amount of effort, willingness, skill and knowledge required. The paper concludes by suggesting that maximizing openness is not practical and argues that open educators should strive for ‘open enough’ rather than maximal openness.

McNally & Christiansen, First Monday

Article link: https://firstmonday.org/ojs/index.php/fm/article/view/9180/7808

Update on OER policy research: Fun with Excel and colour coding

After presenting at the Open Education Global conference in Delft, Netherlands, and submitting the research with my collaborator Michael McNally, I’m finally back to work on my OER policy study.

Currently, I’m gathering documents that represent the open education/OER policy or “policy directions” from Canada’s four Western provinces. I’m looking at documents from the provincial government and research university level, to answer the following questions…

  1. What are the focuses of the OER initiatives at both the government and research university level?
  2. Are there differences in emphasis between the four provinces?

I used criterion sampling (Patton, 2002, p. 238; Palinkas et al., 2013) to create clear inclusion and exclusion criteria for collecting documents. All the documents were collected via web search. Some examples of the documents I collected include strategic plans, provincial funded open education initiatives (eg. BC Campus), budget letters, Hansard minutes, task force reports, etc. I also used government press releases where information was thin – particularly in SK and MB.

In a nutshell, I gathered 140 documents and analyzed 75. All relevant sections of the documents (those that referred to open education) were coded using thematic analysis (Braun & Clarke, 2006). I only looked at the semantic level – meaning I took the text as written and didn’t read into it too much. I also coded the text at the sentence and small paragraph level because it makes it easier to preserve the context. Lots of notes accompanied these text snippets. In an earlier blog post, I showed how I organized all this data in Excel (as I don’t like qualitative coding software… clunky).

Today, I want to show some of my quantitative results. While this is a qualitative study, I think numbers and tables really help the reader. The limitation here is that there is a big difference in the number of documents I was able to collect by province. BC represents the lion’s share of the documents, and the number shrinks dramatically as one moves east – at least in this snapshot of data. Below, you can see not only how many documents I collected, and the number of text snippets by theme, but the percentage of text snippets according to province. I’m still working out the table titles or course…

Screen Shot 2018-06-03 at 3.59.37 PM

Screen Shot 2018-06-03 at 3.59.24 PM

Screen Shot 2018-06-03 at 3.59.59 PM

In the last screenshot, you can see all nine themes I identified. I highlighted the ones in orange that are most commonly emphasized across all four provinces. For each province, I chose the top three themes emphasized in the analyzed text. There’s lots of crossover but also some important differences. For instance, while all four provinces had “cost savings” as one of their top three themes, it’s the 3rd most emphasized in BC compared to the most emphasized in AB, SK, and MB. I’m assuming this is because BC has been working at open education longer.

I assumed “cost savings” would be a big part of the emphasis among all provinces, but I’m very happy to see “impact on learning” and “technology, usability, and accessibility” as strong contenders. I’m surprised that “quality control” is so poorly represented – given that it’s something emphasized so strongly by faculty, and it is well represented in the literature as a general criticism of OER.

Now that I’m getting into the writing, I’m really looking forward to describing these themes and discussing how the collected data compares to the literature.

Presentation – Creating Information Literacy Competencies for Psychology Undergraduates

I recently had the pleasure to lead a webinar for the a Psychology Librarians group in Canada – where I discussed a SoTL project my colleague and I are working on. The aim of the project is to survey psychology undergraduates during library instruction classes, to determine what information literacy skills they possess at each year in the program. Ultimately, we want to use this data to develop a more scaffolded information literacy program with competencies for each year of the psychology degree.

There is a summary of the project on the webinar slides which are available here: http://hdl.handle.net/11205/376 

The feeling of being behind

I’m certain anyone who’s worked in academia has felt ‘behind’. I would bet many early career academics feel this way constantly.

I admit that I have (and still) get myself into a headspace where I’m saying things like “I should be doing more” or “this should be done by now.” My university recently opened our new state-of-the-art library and learning centre – filled to the brim with exciting new technologies and teaching spaces. While I can’t speak for my colleagues, I feel there’s more pressure to deliver this Fall. However, a couple of recent experiences have grounded my expectations and made me take pause.

I recently read a 2016 blog post by Zack Kanter which outlines why we feel behind. Most of us have a vision of our perfect self and we constantly compare ourselves to this vision. Kanter’s summary is perfect.

The question that finally helped me break the cycle was: behind compared to what? Some alternate-reality version of yourself without flaws, a relentless Terminator on the Perfect Course of Life, chasing down and slaying goals and if you stop to catch your breath for one second the cyborg-take-no-prisoners-has-no-bad-days-or-relationship-or-family-issues-and-never-binge-watches-Netflix ‘you’ will just fly by and you will never be able to catch up no matter how hard you try?

I will tell you a secret. There is no other version of yourself, there is only the version sitting here right now. You are not behind (or, for that matter, ahead): you are exactly where you are supposed to be. So take a deep breath and relax.

The perfect version of myself has already published the two research projects I’m currently working on, planned all his classes, and has pre-read all the committee materials. But, putting unnecessary pressure on one’s self-doesn’t lead to greater productivity.

During a recent conversation with a colleague, this feeling of being ‘behind’ came up. She asked me about my current research projects and what teaching strategies I planned on implementing, to which I provided a lengthy explanation. Her reply was, “You’re doing a lot! You should slow down.” I was taken aback. Her comment was followed by a book recommendation – The Slow Professor. The book argues that “corporatization has engendered a pervasive time pressure” in academic life. In the book’s ‘slow manifesto’, the authors say this:

While slowness has been celebrated in architecture, urban life, and personal relations, it has not yet found its way into education. Yet, if there’s one sector in society which should be cultivating deep thought, it is academic teachers… Slow Professors advocate deliberation over acceleration. We need time to think, and so do our students.

It’s ok to stop and think. It’s ok to breathe. I still think being ambitious and productive are good goals. So instead of feeling behind, I’m feeling motivated. The difference? Tempering expectations. Assume what you want to achieve will take longer than you think. Learn to appreciate the small accomplishments along the way.

My process for qualitative web research

I’m a bit of a research methods geek. As I’ve become more familiar with qualitative methods and coding data, there’s a few tricks I’ve learned which I’ll discuss today. Disclaimer: I’m not a methods “expert” (though I’m not sure many people are), and I developed this approach from experience and exhaustive reading 👨🏼‍💻 of the academic literature.

I enjoy working with both numbers (quantitative data) and documents (qualitative data). Qualitative methods are interesting because they provide the researcher an enormous amount of flexibility, and the data often provides a great depth of understanding. The downside to qualitative approaches are that they’re a bit fuzzy. There’s also A LOT of qualitative techniques. You have to be comfortable with loose approaches to analyzing data and, occasionally, creating your own method – usually derived from one or more existing approaches – to fit your project. However, while the methods can be flexible, it’s important to employ your method as consistently as possible.

First, some background. I’m currently conducting a content/policy analysis of Open Education Resources Policy in Western Canada. You don’t have to know what that is. What’s important is that my data is 100% web documents. There are no experiments to run, no statistics to gather, and no people to interview. (Well… I chose not to interview this time around).

Within the documents I collected, I’m looking for snippets of text that make mention of my topic and I’m assigning those snippets a “code.” If you’re not familiar with coding I’ve included a couple of solid resources:

So, here’s what I’m looking at…

This is one of my Excel sheets. I prefer to organize data I’ve found using Excel because programs such as NVivo don’t seem as flexible. Each row is a separate document and the columns label the information in each cell.

Tip 1: Colour Code and Prioritize

It looks like a mess, I know, but there’s a logic. Green rows means I’ve read and coded the document. White rows mean that I’ve skipped those documents for now. These docs might be useful, but certain types of documents are hit or miss with regards to relevance. Conversely, others are DEFINITELY what I want. Time is precious, so working in order of relevance, not chronological, is key.

At this point you might be thinking “What a waste of time! Why collect irrelevant documents?” But, wait!

Tip 2: Collect everything you find as you go, and deal with it later

Time is precious, and it’s likely I’m going to do a “similar” project on the same topic (because it’s interesting). Re-finding documents I’ve already come across is a waste of time, so it’s best to collect them as you find them. Always be thinking of how documents could be useful to you one, two, three, or ten-years down the road. All documents that could be useful later are indicated by the dark blue cells on the left. This process also helps limit the scope of what you analyze. A minute saved is a minute earned? (I just made that up).

Tip 3: Multiple Excel sheets!

It’s imperative to have multiple sheets when coding with Excel. Snippets of text are on the left, and a code is assigned to each snippet.

Text snippets and codes are blurred because… it’s not ready yet!

I also have the document details on the right, in the same format as the previous sheet. Some people don’t agree with this approach because you’re not separating yourself from the documents when creating themes from your codes. My work around is to cover up these details when I organize my codes into themes. Just temporarily shade cells you don’t want to see in black. Problem solved!