The clicker experience at NUIG: Issues and concerns for staff

As I mentioned in my last blog post, I've been attending the iClicker Pedagogy Webinar series over the last few weeks, and find them very interesting. On 25th January, Roger Freedman (@RogerFreedman) spoke about Pedagogical Best Practices. On 15th February, Angel Hoekstra spoke about Pedagogical Strategies for Teaching with Clickers in the Social Sciences. This week, on 22nd February, Jennifer Kaplan spoke about her use of clickers in large Statistics classes. It's interesting to hear about the different approaches, and also to see the common themes.

In my post last week, I gave an account of the student feedback on the use of clickers in the College of Science initiative at NUIG. On the whole it was very positive, and it encouraged the College to expand the use of clickers to include 1st and 2nd year undergraduates. But, what about the staff issues?

Three group meetings were arranged during the academic year, when all staff involved in the clicker project were invited to come along and discuss progress, issues, problems, successes and to give suggestions. Initially well attended, the numbers coming to the meetings dropped significantly in the second semester. The issues that came up during the year reflect some of the findings of Roger, Angel and Jennifer.

Mandatory Use
Initially there was concern among some staff members that they were required to use clickers in their teaching. The strong message that came from the Dean's office was that, while nobody was forced to use the system, it was strongly recommended that staff come to training and consider how they might be used. Ultimately, uptake varied in difference disciplines.

Consistency of Use
There was a concern that, if clicker questions were not regularly used across all subjects, students might stop bringing them to class, thereby lessening the effect in those subjects that did use them. Although the use of clickers was not consistent across subjects, the survey of students at the end of the year indicated that 66% of respondents brought their clickers to every class, and a further 26% brought them to most classes.

Attendance Monitoring
There were a number of discussions and concerns around the use of clickers solely for the purpose of monitoring attendance. Everything in the literature indicates that this is not a good use of clickers, and likely to encourage the practice where students bring along 5 or 6 devices for their friends. At the same time, over time the data could be used to monitor trends in student participation, which could be used to identify students at-risk.

Through discussion, it was decided that clickers do not provide a reliable method of tracking attendance and that they were best used as a learning tool. At the same time, the use of clickers, where students find the activity valuable, can have the effect of increasing attendance.

At the end of the first semester, the Physics lecturers polled students (using clickers) about the possibility of awarding marks for participation, based on clicker use in class. The response was overwhelmingly in favour of using clickers and getting marks for participation. As a result, Physics students were awarded 5% of their overall mark for participation in the second semester.

This corresponds to what Roger Freedman described as low stakes clicker credit, in comparison to high stakes credit, where marks are awarded for a correct answer. Roger suggests that the choice of low or high stakes credit for participation can change the dynamics in a class. While there is no difference in the learning gain, high stakes credit can stifle student discussion.

Content and Flow
There was some initial concern that the time used in lectures for clicker questions would result in less time to cover content, and ultimately this did prove to be the case. However, this also raises questions about the responsibility of the student in the learning process, and how much they can be expected to read outside of class. This has the effect of starting to transform the underlying pedagogy to accommodate increased interaction and participation in class.

Use of clicker questions can give very valuable feedback to the lecturer who is concerned about content. A well-designed question can indicate whether a class is following a lecture, for example. As Jennifer Kaplan stated, you may be covering the material, but are they getting it?

Another, related, concern was that the flow of a lecture is broken when a clicker question is asked. Students may begin to chat, and it can be difficult to bring the focus back to the lecture. In fact, a clicker question should be disruptive, encouraging a student to think about what is being asked. In a peer instruction situation, students are encouraged to discuss the question in  pairs or groups.

Roger Freedman suggested that the best clicker questions are challenging, with multiple plausible answers, that reveal student confusion. Conflict leads to drama and gets the discussion going.

Conclusions
In particular, we found that the adoption of the clicker technology is unique to the culture and context of teaching within each discipline, and that this observation had to be factored into the training needs for different groups. The experience of peers is particularly persuasive, and we were lucky to have two academic staff members within the College who already had extensive experience of using clickers in their teaching. They both gave practitioner workshops at the start of the year, and were persuasive in their encouragement to use clickers.

Relating the NUIG experience to that of the experts in the webinar series, many of the issues (particularly around attendance monitoring, credit for participation, and the desire to cover content) are common. Some good tips I've picked up from the speakers are:

• Clickers are best used as a teaching tool, but even the best tools can be misused (Roger Freedman)
• Integrate clicker questions into your lecture, don't treat them as an add-on! You lecture less, and the students think more. (RF)
• Use clickers regularly, and tell students why you are using them.(RF)
• Explain regularly and often why you are using clickers and how the students will benefit. (AH)
• Experiment! See how it works best for you and your students (RF)

 Finally, I think the use of clickers can be an opportunity to change an approach to teaching, but this won't happen overnight. I was heartened by Jennifer Kaplan's answer to a question about the extra time required to prepare lecture materials to incorporate clicker questions. She responded that 5 years ago she didn't change her lecture material, but reworked some slides to incorporate clicker questions. Her approach has evolved and improved over time.

Tweet

The clicker experience at NUIG: student feedback

I've been "attending" some of the webinars in the iClicker Pedagogy Webinar series over the last few weeks. They are organised for 1:00 EST, which translates to 6pm Irish time. If I race home from work, I can just about catch them while watching over the kids.

On 25th January, Roger Freedman (@RogerFreedman), Physics lecturer and clicker enthusiast, talked on the topic of Clickers in the Classroom: Pedagogical Best Practices. Roger gave a lovely presentation, making a compelling case for using clickers and demonstrating that there can be a significant learning gain from integrating them into teaching. The webinar was probably more suited to people getting started with clickers, but it was very interesting to see some of our experience here in NUIG being mirrored in what Roger was saying.

In September 2010, 762 clicker devices were distributed to incoming first year undergraduate Science students and Foundation Medicine students at NUIG. In addition, 40 RF receiver devices were distributed to academic staff involved in the initiative. The aim of the project, led by the NUIG College of Science, was to use clicker devices in lectures to promote student intellectual engagement and also to increase attendance. CELT was involved to provide advice, training and support for academic staff during the year.

In March 2011, we invited student feedback on the use of clickers using an online survey. A total of 272 students responded, giving a 35% response rate.

What did they like?
When asked about what they liked about using clickers, a large number of students (71%) made some reference to active learning ("It made me sit up and think"). Smaller numbers of students mentioned anonymity as a positive, while 2 students mentioned social aspects - that clickers helped them get to know other members in the class.

What did they not like?
When asked what they didn't like, the responses were more diverse. A significant number (19%) said that the clicker questions were disruptive, that other students would start to chatter and it could take some time for the lecturer to regain control. Some students (12%) thought that the questions were not always being used well (questions too easy, questions too hard, trick questions) while about 7% of respondents thought that the clickers weren't being used enough.

None of the students appeared concerned about not covering material in lectures.

Attendance and Participation
In his talk, Roger noted the difference between Empowering and Compelling students to use clickers. They work best as a learning tool, and students prefer the formative feedback they offer, over grading or tracking attendance.

In the second semester of the NUIG Science clicker initiative, students were awarded 5% of the overall mark in Physics for participation in class using clickers. In the survey, when asked what they liked about using clickers, 18 students said they liked the participation marks. However, when asked what they didn't like about using clickers, 19 students said they didn't like the participation marks, because sometimes they forgot to bring the clicker, or it didn't work due to flat batteries. In contrast, when asked what would improve the experience, 22 asked for participation marks in all subjects, 2 asked that they be used to track attendance, while just one students said that there should be no marks for participation.


 Outcomes of the initiative
In the survey, 79% of respondents said that the system should be used with first year students again, with 9% saying no. In fact, the College decided to expand the scheme and in September 2011 all first and second year students were provided with clicker devices.

One of the aims of the project was to promote intellectual engagement. From the student responses to the survey, active engagement is reported as an outcome. This may be due to the use of the clickers themselves, the changed teaching practice as a result of the use of clickers, or just a novelty factor associated with the devices. The student feedback supports findings from more substantive studies in the literature that clickers can play a positive part in student learning.



Tweet

Call for Papers: EdTech 2012

EdTech 2012, on the theme of Digital Literacies, will take place from May 31st to June 1st, at NUI Maynooth.

This year's keynotes will include Lesley Gourlay @lesleygourlay (who is external examiner on our Learning Technologies module here at NUIG), Martin Oliver, Doug Belshaw @dajbelshaw and Martha Rotter @martharotter.

The call for abstracts is now open, and submissions are welcomed, before 30 March, on the following themes:

• Teaching digital literacies
• The student experience 
• Learning resources - collaborative and social
• Creativity and learning design
• Online practitioner competencies 
• Assessing digital competencies
• Sustainable models of innovation
• Technology-supported assessment approaches

More details about the conference are on the ILTA website.

Check out Lesley Gourlay's keynote at the CELT/NAIRTL conference last year at NUI, Galway, on the topic of The tyranny of participation? Critically exploring notions of student engagement.

NDLR Resourceful Coffee Morning 2011

NDLR Resourceful Coffee Morning 2011, a set on Flickr.

On Tuesday 6th December, in collaboration with the NDLR, we hosted a Resourceful Coffee Morning in CELT.

The purpose of the event was to

• Find out more about Open Educational Resources for Higher Education
• Browse through digital resources relevant to your teaching
• See how other academics are using digital resources in their teaching practice
• Learn about collaborative projects and communities of practice

Catherine Bruen, manager of the NDLR service, came from Dublin to demonstrate the potential of the service.

Here are some photos from the event.

Threshold standards for Blackboard courses: Innovation Prevention?

Image: Simon Howden / FreeDigitalPhotos.net

At the Durham Blackboard users' conference last month, there was a panel discussion on threshold standards for VLE courses: whether we should have them; at what level; and what they should include. Others have written about the discussion including Julian Beckton's excellent blog post and Matt Cornock's summary.

Here are my thoughts on the issue of threshold standards  for Blackboard courses at NUIG.

Since Blackboard went mainstream in 2007, we have never required that academic staff use Blackboard to support their teaching. We make it easy for them to use, authentication is via ldap so no new userids or passwords have to be remembered, the courses are created and ready for use, the students are automatically enrolled in the correct courses, we provide training and support. Uptake has been massive, with very few staff not using Blackboard and, I estimate, very few students who don't have resources (at some level) available to them on Blackboard. So why would we need threshold (minimum) standards?

For a start, we couldn't enforce such requirements. Every single module code on our records, whether active, defunct or redundant, has a corresponding Course created on Blackboard. So, we have a huge number of Bb courses, many of which are empty. Moreover, we have no central system that records who is responsible for what module code. So instructor associations with Bb courses is still largely a manual process. Who would be responsible for a non-conforming or empty Blackboard course?

Blackboard is used to support teaching in different ways at NUIG. For distance and blended learning, Blackboard is central to teaching and learning activities. For primarily face-to-face teaching, Blackboard might just be used to provide administrative information, with readings and lecture notes. There is a whole spectrum of potential use between these two points. The important thing is to focus on the teaching and learning and to use Blackboard ( or other appropriate tools) to support these activities.

I can understand the point about consistency for students, so that there is a standard way for students to access materials. It is true that some Blackboard courses are very disorganised, with documents thrown into folders, using a mix of formats and no logical thinking behind any of it. I would suggest though, that what is needed is good signposting, and some thinking and planning on the part of the instructor on the course. After all, we don't dictate minimal standards for other teaching tools: minimal requirements for PowerPoint slides; the threshold form of a 1 hour lecture; standard teaching methods. But we do expect that these elements are organised and planned.

What of innovative, creative and effective course designs that don't fit into the standard template? Do we run the risk of having people subvert the requirements, making organisation more confusing for students, or driving the innovators out of the VLE altogether? Are threshold standards for the VLE an invention of the Innovation Prevention Department (IPD)?

At the School or Programme level, it may well be worth considering a standard, but flexible, Bb course structure. This is particularly true where distance or blended learners are involved.

Course information, learning outcomes, reading lists, staff details, handbooks, timetables, assessment information, are all examples of information that should be available to students. The VLE is an excellent place for them, but there may be other, more suitable distribution methods. The requirement is that they are provided somehow, not that they are all contained in Blackboard.

Returning to the image of the VLE as a Trojan horse; at NUIG Blackboard is an easy first step into using technologies to support teaching. I believe that by imposing minimal required standards on Blackboard courses we would be creating an impediment, giving lecturers a reason to reject the VLE completely.

There are other things we can do to encourage better use of Blackboard. For example, each course is created with a standard menu, including menu items for Course Information, Learning Outcomes, Feedback. This is a reminder that these elements are expected, though many staff just ignore them. We can also put together guidelines and recommendations for effective Blackboard courses, in a non-threatening, non-judgemental way. See, for example, Leeds University's 10 Tips for improving a Blackboard course.

So, in CELT, let's continue in our role as guides, supporting academics in their use of Blackboard, and avoid becoming the VLE wing of the IPD.

Tweet

Getting started with Turnitin

Last Friday I gave a workshop on Using Turnitin to Deter Plagiarism for academic staff at St Angela's College, Sligo. St Angela's is a college of NUI Galway and occupies a beautiful site on the shore of Lake Gill, just outside Sligo, and it is always a pleasure to visit there.

The workshop was aimed at teaching staff getting started with using Turnitin. As we went through the practicalities of setting up a Turnitin assignment in their VLE (moodle) and discussed strategies and options, I stressed a few points. These are the take-home messages that I think are key when starting to use Turnitin.

1. Focus on the teaching: the 12 people in the group were concerned teachers. The last thing they want or need to become is "plagiarism police", with a focus on plagiarism detection. Turnitin is best used as a teaching tool, so make sure that the strategy you use, and every option you choose, is based on improving the learning for your students. 

2. Turnitin does not detect plagiarism, it highlights matching text. It is very good at what it does, but it cannot tell you that a student has plagiarised. You, as the teacher, are the one who decides if the matching text indicates a problem, or otherwise.

3. Turnitin can only be used to help detect plagiarism of text, or cut-and-paste copying. It doesn't identify plagiarism of ideas, and won't be of any use with ghost-written materials.

4. Originality reports have to be interpreted. You have to examine a report to understand what the coloured highlighting is telling you. There are many reasons why text might be highlighted: direct quotes, bibliographic references or just writing on a particular topic using the correct language for the discipline. Highlighted text could be an indication that the student has actually performed a good literature survey!

5. The similarity index should never be used as a measure of plagiarism. Do I really need to explain this?

6. You have to follow up, when you find a problem. This could be in the form of general feedback to a class group, where there are common issues across the class; or individual feedback to address a particular piece of writing. Where a serious problem exists, it will be necessary to use formal procedures. Feedback should come first though, see point (1).

7. Giving students access to originality reports can be a great learning tool, when they are learning academic writing. But it has to be in a supported environment, where they can understand how to interpret what they are seeing. See point (4).

For more on strategies for using Turnitin, see my presentation from the E-Assessment Scotland conference in Dundee last August.

Do you have any take-home messages for teachers starting off with Turnitin in the classroom? Please add to the list by writing a comment. Feedback on my list is also welcome.

Tweet

Testing: Kaltura for Blackboard

I'm currently engaged in testing out Kaltura, a platform for video, as part of a 3 year extended pilot at NUI Galway. It has a rather nice integration with Blackboard, which we're hoping to launch within the next few weeks. If any members of staff are using or creating their own video and are interested in being part of an initial pilot study in the next few months, please get in contact with me.

For the moment, I'm testing the embed feature. So, here's a short video recorded during our guerilla video workshop with the participants in the PG Cert in Teaching and Learning in Higher Education.

Using the VLE as a Trojan Horse

The Calman Learning Centre

Last week I travelled to Durham for the 2012 Blackboard Users' Conference. Coming immediately at the end of the Christmas/New Year break, it was a shock to the system, and a rude kick start to my brain. But I survived, and came away with my head buzzing. Now, if I can just get time to implement some of those ideas...

The two keynotes, in particular, got me thinking about training and support for our academic staff in their use of technologies for teaching and learning.  The learning technology team at CELT aims to provide pedagogical support for staff in their use of technology, although often we end up dealing with queries about user accounts and basic support for our VLE, Blackboard. Our mantra is that we want to encourage each member of staff, from their current technology use, to do something a little more. We always keep in mind that the teaching/learning is key, and we're just using the technology to (better) support it.

With this in mind, Gráinne Conole's (@gconole) metaphor of the VLE as theTrojan Horse makes perfect sense to me. Since we went mainstream with Blackboard almost 5 years ago, the uptake by staff members has been very positive. Encouraged by automatic enrolment of students in the correct courses, the possibility of easy communication (announcements, email) and demand from students themselves,  academics at NUI Galway have embraced the VLE, albeit (in some cases) with reluctance and/or trepidation.

We have a significant number of early-adopters, champions and people willing to experiment, and these people really keep us on our toes. But we also have a large group of staff who just use Blackboard as a repository, making minimal use of its functionality. Our challenge now is to use this as a basis to encourage them along and to try something new. Gráinne described the VLE as the "nursery slope" for academics: a safe and supported environment where they can learn the basics of technology for learning. But, how to do this when the average academic is already overworked and doesn't have time to even turn up for basic training?

The other image that I liked in Gráinne's keynote was the notion of the VLE+. It's no longer a case of the PLE vs VLE: we can extend our VLE with a myriad of extra functionality to support teaching and learning activities. This afternoon I met with our learning technologies team, to plan our activities for the next few months, and we discussed all of our Blackboard connectors which make up our VLE+. At the moment, the list includes: Campus Pack (Learning Objects); Turnitin; Echo360; QuestionMark; Kaltura. This all provides a seamless environment for staff and students - to the extent that all our support tickets say "Problem with Blackboard", when it's probably not a Blackboard issue at all.

Is this a bad thing? As raised by Nick Pearce (@drnickpearce) in his presentation Beyond Good and Evil, is "closed" necessarily evil?

Ray Land (now Professor of Higher Education at Durham University and Director of Durham’s Centre for Learning Teaching and Research in Higher Education), in his keynote addressed exactly this point. Higher Education is historically closed - with teaching taking place behind closed doors. The newer, digital world, challenges this; it challenges the authority of the academic and breaks down barriers. While there are some that embrace the digital world, for many academics, this represents "troublesome knowledge", and requires a change of culture. The digital world is constantly changing and evolving. For learning technologies, this means a new tool every day (at least) that could be used in teaching and learning. It's difficult, even for those of us who work in the area, to keep track of developments. The VLE is a way to contain that volatility, and slow things down.

Footbridge, Durham

In the same way that teachers will always have a role in leading students through a new landscape (while not necessarily being the source of all knowledge), our learning technologies team will have a role to play in leading academics through the fast paced, digital world, using the VLE+ as the safe and supported environment.

Up next, when I get a chance, some thoughts on the panel session around threshold, minimum standards for VLE courses.

Welcome to the class of 2011

This afternoon I am meeting the new group of students on the Learning Technologies module, part of the PG Diploma in Academic Practice.

It looks like we have a great group of people and I look forward to working with them over the next couple of months, as they explore the use of technology in their teaching.

E-Assessment Scotland 2011 - my prezi

I was really pleased to be invited to give a presentation at the annual E-Assessment Scotland conference in Dundee last week. The conference itself is a free event, and attracted around 300 delegates, mostly from the UK. I'll write a blog post soon (when Blackboard is fixed) about the conference.

In the meantime, here's my presentation, which I prepared using Prezi. It was my first Prezi, and I learned a lot in putting it together - but that's another blog post.

Supporting Students' Writing Skills using Turnitin on Prezi

Tweet

Blackboard 9.1 is coming

The upgrade of our Blackboard system to release 9.1 is progressing according to schedule. In the last few weeks, we’re received many enquiries about when the new system will be available and what changes staff can expect.

When will I have access to my courses?
It is intended that the new Blackboard system will be available for use during the week of 22nd August.

There will be a new URL to access the upgraded system, but otherwise you will log on using your username and password as before.

You will have access to all your courses for the academic year 2011-12. So, if you are an instructor on a Blackboard course currently, or if you requested access to a course within the last few weeks, you will have access to that new course on the new system.

In keeping with existing policy, the new courses will be created as empty shells, ready for you to develop as you wish.

What about the teaching materials in my current course? How can I get it into my new course?
To facilitate the re-use of teaching materials, the new Blackboard system will also have courses for the academic year 2010-11, to which you will have access. These are not full courses, and will not include student enrolments, so cannot be used for teaching purposes. However, much of your content from the current Blackboard system is being migrated into this area, so that you can import it, or copy components of it, into your new Blackboard course for 2011-12.


Can I still access the old site?
The old Blackboard site will continue to be available until October 2011. This is to facilitate any courses that are active over the summer period, and to facilitate access to 2010-2011 materials and student work.

What benefits are there to the upgrade?
There are many benefits that you will experience with the new Blackboard, many of which will become clear as you use the system. Just some immediate advantages are:

Improved Interface

• There is a new drag-and-drop interface making it easier to navigate the system and complete tasks with less clicks.
• The OK confirmation button (which displayed when creating content in Blackboard 8) has been removed.
• The course control panel is located on the main page of the course for easier editing.
• A new “edit mode” makes editing your course easier and gives you a (95%) student view of your course.

Course Statistics

• The course statistics function will be fully operational.

Improved Group features

• The new group features allow instructors to create any number of groups at once.
• Students can be randomly assigned to groups, manually assigned to groups, or asked to sign-up for groups themselves.

User Information Sessions: What’s New in Blackboard 9.1?
In preparation for the new release, CELT is facilitating User Information Sessions. During the session we’ll go through the main changes in BB 9.1 (working on the test server) and we’ll provide you with your own sandpit Blackboard test module which you’ll be able to access during and beyond the drop-in session.

A list of upcoming CELT events, including these Information Sessions, can be found on our CELT events site.

CELT will announce a full programme of training sessions. These will be a combination of short “What’s new in 9.1” for experienced Blackboard users as well as a new version of our popular “Introduction to Blackboard” workshops for new users.

We will also arrange a number of drop-in sessions to coincide with the start of teaching, for hands-on support and advice.

We are currently working on some on-line resources and will update our own Blackboard Support site in time for the new system.

Ken Robinson keynote at BbWorld 2011

In CELT, we are busy preparing for the introduction of Blackboard 9.1 in NUI Galway. More information about the new system will become available over the next couple of weeks and you can expect a full schedule of information sessions and training opportunities.

In the meantime, here is a recording of a keynote given by Sir Ken Robinson at Blackboard World 2011 two weeks ago.



Tweet