Workshop Feedback and Moving Forward

I’ve now looked through the feedback from the workshop evaluation forms. Stupid problem number one was that several people only completed one side of the evaluation form. It is clear that they did not realize the form had two sides. I will need to be much more clear at the next workshop that there are two sides to the form. I’ll also update the form to say or something like that at the bottom.

Themes that emerged from the workshop feedback:

  • Resource: participants would like some kind of resource (handouts and/or eBook) to support the workshop. This resource should include a list of apps that are mentioned with their descriptions.
  • More hands-on: participants would like more hands on activities and less theory.
  • Slower pace: Pace of presentation was too fast. Some participants felt like there was too much information presented

In addition, the pre/post test indicated that the following items were already known by most participants, such that they need not be covered in the workshop:

  • Turning on and off the iPad
  • Using the iPad to buy an App from the App Store
  • Organizing icons on my iPad

As I mentioned previously, my plan is to restructure the skills lab to be problem-based learning scenarios. These will be supported by some form of resource as well as me (or the technologist) being present. It will be sort of liked a “flipped” classroom model, when learners are given a challenge and can work either individually or in small groups to work through the various activities. This will increase the amount of hands-on time, but also hopefully, help people feel like they won’t forget everything they learned the minute they walk out of the room. The resource will also contain a bunch of the theoretical information that originally covered in the skills lab. This information could just as easily be read, rather than presented. The key is to spend the face-to-face time doing hands-on activities.

My other plan is to provide some form of resource. I’m not 100% certain what that will be. Originally, I had planned on it being this website, but I’m realizing that the navigation on this site is much more conducive to looking up things you already know. They need a form that looks more like a book that they can easily follow in a linear fashion. So, I think I may end up building an iBook. I also think I need a few worksheets – pieces of paper that provide a checklist of activities for people to progress through. This model has the benefit to adapt to the level of the learner. Beginner learners can start at the top of the list, where more advanced learners can skip over the stuff they already know.

There will be lots of work involved in putting this resource together. I should be able to get enough together to prove the concept for the next workshop – and then hopefully have more time over the summer to fill in all the blanks.

Do I make an iBook?

One of the issues that came up at the initial workshop was a lack of resources for participants. This was not helped by the fact that this website did not yet have resources associated with it. But also, learners wanted a list of apps, they wanted things they could take home.

I have always wondered whether the website was the right medium for this type of content.

In addition to the need for a resource, the other issue is that the workshop needs to be more hands on. The theory part of the workshop needs to be filtered out – and put elsewhere. I thought the website was a good place for it, but I’m finding myself struggling with structure – but also, knowing that they likely won’t ever access it on the website. I think an iBook might be a better choice. An iBook would allow me to add in theory in a book format. They would be more familiar with a book, flipping through pages and reading the content. We use iBooks for other faculty development programs – so the use of iBooks is not unfamiliar – plus the iBooks tutorials are needed for other fac dev programs – on other projects, not just this one … so I could do up the iBook Tutorials and integrate them into several iBooks.

With our other program we also provide the iBook in PDF format. Here, I don’t think I would do that. All the content would be available in the iBook format and on this website. I want to force learners to use the iBook rather than a PDF version – I also think the multiple versions causes a confusion sometimes, as sometimes people will download the PDF into iBooks and then not know why the iBooks isn’t working properly.

So, there is that – once I create a bunch of content, I shall create an iBook to support that content. The big question now is, can I get that done in time for a workshop on May 15th? – assuming we can make the May 15th workshop happen!

Conceptual framework

When trying to visualize the writing a paper on the current design using the conceptual framework I identified back when I wrote the paper: Considerations for a Professional Development Program to Support iPads in Higher Education Teaching, I realized that I have having trouble making things fit, but also including all the things that mattered in the current design. That made me realize that what I really need to do is articulate a new emerging conceptual framework – a picture that does a better job capturing the variables that directly affect the design. In this post, I’m going to talk about some of these variables – I’ll need to spend some time with pen and paper brainstorming all the topics and building a new picture of how the course looks.

First, from the original paper, I have:

  • TAM – more specifically the goal to increase an individuals perception of the ease-of-use and perceived usefulness
  • Technology – the idea that a workshop needs to cover both product technology and idea technology (it occurs to me as I write this, that this is a concept I should have introduced in my emerging technology class, to help students with their technology presentations – but alas that isn’t related to this post)
  •  Time – the idea that calendar time needs to pass in order for a person to adopt a technology – the idea here, is both related to cognitive load – too much information in too short of a time is not useful, but also that just plain time needs to pass. People need time to integrate a new technology – this doesn’t happen overnight …
  • Individual Beliefs – these are can be a barrier – or they can help – with adoption. The idea here is that if you are asking people to change beliefs as well as use technology, you have more work to do. But also, that to get people to change their beliefs, peers help.
  • Organizational structures – these can help or not.

I also had evaluation, but that was really a throw in to address the scope of the comprehensive exams question, and not something that truly fits within the framework. Additional things that I think need to be considered include:

  • Sustainability / resources – this effects many aspects of the design. You can create something great if you have infinite resources, but the reality is you don’t. Under sustainability, you need to think about physician availability, and the costs associated with running the program
  • Organizational structures – this is playing out a role in my project too … it is mentioned above but I felt I needed to say it again. This is in part because things are changing … people are new … organizational policies are changing … so you have a moving target here.
  • Adapablity. This has a few components. It involves adapting to  the changes in the organization, but also needs to address the adapting of the technology itself. An iPad program needs to constantly evolve, as new mobile apps are available every day, but also the older apps are improving. Something that you dismissed 6-months ago may now be leading edge. Building this into the program is not trivial.

OK.. I have more thoughts in this area, but they are not flowing right now. I will need to continue with this later.

Content remixing

I am thinking about the skills lab content again. One of the bits of feedback is that they want more hands-on – and it occurs to me that this is not just the case vignettes but also in the skills lab. The focus in the skills lab was a little wrong – it needs to be more activity centric. Sort of like the Icebreaker activity – which was successful – I think also the skills lab needs more “challenge” type activities – and the theoretical content can be left for the eBook or something – similar to what we do with Essential Teaching Skills – we need to keep the focus of the workshop on activities (on doing).

So, the skills labs need to be re-structured. They need to be activity driven. Content that is best explained via lecture or text, should be left to the text – the workshop needs to focus more on hands-on.

So, in looking at workshop one, what is the skills lab content? Editing and commenting in iBooks. Perhaps it should look more like:

  1. Organizing icons (creating folders, renaming folders, changing which page on icon is on)
  2. Finding apps (searching iMedcal Apps, searching the library – finding good apps)
  3. Using iBooks (which covers some gestures as well as adding content and sending an email)

This becomes enough – and I write it up as a challenge sheet and everyone gets a chance to “do” each thing and tick it off the list when they do it. That way, they are doing hands-on and I’m just helping them work through it – filling in the blanks and asking questions.

Now I’m wondering about a link between problem-based learning and teaching technology. I will need to go check the literature on this.

First Workshop – Initial reactions

My initial reactions after the workshop were that it worked well. I don’t think it needs two physicians, but it does need more structured activities. I need the activities for the icebreaker and skills lab written down. They need to be written out better. Perhaps for the skills labs, I need actual activities (hands-on workbook) that people go through, rather than me talking and explaining something. A few things I showed worked well, but others seemed like me talking too much, and I wonder if I caused more confusion than I needed to.

In looking at the feedback, people wanted a handout. They felt they were writing things down too much. The workshop definitely needs some kind of resource (this website, but also, perhaps, a one or two page handout that people have in the workshop itself).

We also had a large group (20-people). The workshop would be better with a smaller group – or broken up into smaller groups. The length of the workshop worked well. For most of the participants, they reported “just right”, with only a few wishing the workshop was longer.

The ice breaker activity worked remarkably well. The timing for the workshop was good (right on 1h30m according to the powerpoint timer) – however, we did 15 minute ice breaker activity, 35 minutes each session, and 5 minute wrap up.

So, some quick updates for the next workshop:

  • only two presenters (ice breaker could be led by either presenter – technology educator and physician demonstrator)
  • handouts with list of apps being presented (perhaps some info on context)
  • structured activities for the skills lab – getting them doing more hands-on in this section

Other feedback is that they wanted to know more about medical apps they could use. One challenge here is that many of the medical apps being presented are apps that cost money. I haven’t really figured out how to manage that conflict.

Pre-workshop jitters

In a couple of hours we will run the first workshop. For the most part, the logistics are all organized; however, I woke up this morning to complete white out. It seems to be clearing up a bit now, and the weather network is telling me that things should be clear sometime between noon and 1pm. An hour ago I could not see the buildings across the street!

2014-03-22 10.03.31The workshop is up in Wakefield – about a 30 minute drive when the weather is good. Fortunately, the workshop participants are already there – they were up in Wakefield all day yesterday as part of the annual DFM community retreat. The only people driving up today are myself, one of the physician facilitators, and the videographer. I hope everyone makes it up there safely.

The content and organization of the workshop are prepared. I had hoped to have the website in better shape prior to the workshop. However, a lot of the content of the workshop will find its way onto the website – which is why there is a videographer at the workshop itself. I hope that in presenting the material, I get a lot more detail than I have in my transcripts from my initial interviews. The act of teaching the material to peers will be useful. In an ideal world, I would have been able to get the group of early adopters into a room together and had them explain to each other how they use the iPads – that would have allowed for rich knowledge transfer. Actually, I may look at proposing that as a concurrent session at the Fall retreat. Depending on how many concurrent sessions they have, we could do one or two iPad workshops, plus an iPad knowledge sharing event. My only concern with such a thing is ethics approval – but if I were to treat it as a separate study, I could get approval just for it …

I need to remember to document the room setup at the workshop. Ideally we would have had two rooms, but that didn’t happen. So we have one rather large room, but I still worry about when we break into two groups whether we will end up competing with one another (from a volume perspective). I won’t be using a projector for the skills lab, as I would rather focus on the hands on portion in a more intimate setting. The demonstrator will be using a projector so that everyone can see what he is doing. I wish there was an easy way for me to record what he is projecting directly – so I get good images of what is on the iPad – but I cannot see how we could manage setting that up.

We have two projectors in the room. One will show the PowerPoint slides which are on my computer, and the other will be available to show iPads.

Skills Lab Notes

Tomorrow I shall be presenting the first iPad skills lab. I would have liked this website to have contained more content by then, but it is what it is. One thing that occurred to me is that I could create a series of checklists for skills lab facilitator, just to act as a guide. Presumably the person doing the skills lab will already know how to do the different things, but they’ll need a list of what to cover in the works. So, for my reference here is the list. We’ll see how much of it we actually cover in 30-minutes:

  1. Configuring your Apple ID
    • Multiple Apple IDs
    • Configuring Apple IDs for iCloud versus App Store
    • Relationship between Apple ID and apps your purchase
  2. Downloading and updating Apps
    • Need for Apple IDs
    • Ideas on where you can find good Apps (e.g. Residents, iMedical Apps)
    • Mention library has some free app / subscriptions
  3. iPad gestures
    • Response to gestures depends on the app
    • Number of fingers matters
    • Touch versus touch and hold
    • Four finger swipe
    • Pinch and expand
    • Flick versus swipe versus drag-and-drop
  4. Organizing icons
    • Putting apps in folders
    • Editing folder titles
    • Searching for apps
  5. Finding things (searching)
    • Searching for apps on the device
    • Finding good apps
    • Check out the library
    • will have vignettes and tutorials
  6. Using iBooks, including: adding comments, editing text comments
    • Downloading eBooks
    • Deleting eBooks
    • Navigating (chapters versus pages)
    • Orientations
    • Changing font size
    • Highlighting
    • Adding comments
    • Emailing comments

Technology Educator not Technologist

In describing the roles, I had listed the role that I shall initially fill as the “technologist”, but in reality that isn’t the role. A technologist may actually be a poor choice for that role, as technologists are not necessarily any good at teaching technology. They may be good at using technology, but teaching technology is a completely different skill set.

The most important characteristic for someone in this role is patience and empathy. You need to make the workshop participants feel comfortable learning and give them permission to play – more specifically they need to feel OK to try and it needs to be OK for them not to succeed the first time.

So, I’ve change the title of this role.

Technology Educator

The purpose of this page is to describe the requirements of the Technology Educator. The primary role of the Technology Educator is to teach the skills lab portion of the workshop.

This website contains a collection of skills lab tutorials. The Technology Educator needs to be comfortable adapting the skills lab portion of the workshop to the current level of workshop participants. In addition, the Technology Educator needs to be familiar with teaching new technologies using a hands-on skills lab approach.

Physician Facilitator (Demonstrator)

The purpose of this page is to describe the requirements for the Physician Facilitator (Demonstrator).  The primary role of this person is to provide examples of how the iPad might be used in context. This person may or may not be seen as an ‘expert’. They must be confident in describing specific examples of how the iPad is used.

This website contains a collection of Case Vignettes that outline how the iPad is used in clinical medical education. These case vignettes have been collected throughout the delivery of iPad workshops during the Educational Design Research study. They are provided as examples only. It is important that the Physician presenting the case vignettes use vignettes that they are comfortable presenting. The physician in this role should be encouraged to use their own examples. This is one area where each iPad workshop will differ – as each facilitator will present a unique perspective. Examples are expect to change over time as new applications become available and as the ways in which we use iPads changes. Because the Physician Facilitator (Demonstrator) is encourage to use their own current case vignettes, the Case Vignettes portion of the workshop becomes self-evolving.

The time allotted to the Case Vignettes presentation is 30-minutes. In a typical workshop, case vignettes would happen twice. The first group of people attending the vignettes would be more experienced iPad uses and should be encouraged to share their own vignettes. The second group are less experienced users. It is important that vignettes are demonstrated at a slow enough pace for learner comprehension and time is allowed for learners to ask questions.