Category Archives: Research Journal

Importance of Titles

I noticed the other day when I posted video clips to Vimeo, if I used more descriptive (longer) titles, then more people re-tweeted my videos. This behaviour has encouraged me to be more descriptive in my titles.

For example, rather than “Enabling Lock Rotation”, I am now using “Preventing your iPad screen from rotating”. The old title was very “feature” specific – in that it was labelled based upon the name of the feature. The new title is more behaviour or user specific – in that it describes how the feature is actually used.

Iterations

In educational design research, improved design is achieved through multiple iterations of the design. Iterations occur at both the macro and the micro level. In this study macro iterations occurred at the program design level. With each new data point, the overall iFPD program design was adjusted to account for the new information. In addition, iteration occurred at the workshop level and the artifact level. Individual workshop design was conceptualized and re-conceptionalized to address the changing context in which the research was taking place. Finally, the artifacts themselves (website, eBook, and workshop handouts) went through multiple iterations of design to ensure the information they presented flowed appropriately and worked for the environment in which it was to be delivered.

In design, iteration is necessary and is constant, however, this poses an interesting conflict with research practices. It would be completely not practical to have to re-submit research ethics requests with each iteration. However, there does come a time in the project when enough things change, that the reapplication to ethics becomes necessary. I’ve now reached that point. I can no longer work within the constraints of my current data collection tools. In addition, the nature of participation in my project has changed.

Screen Shot 2014-06-02 at 17.23.27

So, now I need a new diagram that illustrates my data collection – because that will show the changes to the protocol. The review of the research questions looked good – the questions aren’t changing. I’m still trying to solve the same problem (statement of the problem is good), but the actual design of the program and the approach to solving the problem is fundamentally different. I think this is in part because the pre-curser information wasn’t available, so there is much more focus on that information, but also because the organizational context has shifted – the people involved in faculty development are different (new fac dev director) and the team of physicians that I’m working with is different. I had anticipated having more physicians volunteer to work as physician facilitator – that did not happen. Funding changed, which meant I had a reduced set of people to work with (Fac Dev budget doesn’t allow for compensating community physicians).  We have also moved further along in the adoption cycle. Rather than the tablets being new and interesting, we are shifting to the slope of disillusionment. The devices have been in people’s hands long enough that the newness effect isn’t enough to keep then interested in the program. With that, we are seeing a drop of in interest in attending workshops. That at least might explain it … I will need to ask the question on the survey at the retreat.

Research questions

Looking back at my proposal, my research questions are:

(1) How can iPads be used to support teaching and supervising residents in Family Medicine?

(2) What artifacts (e.g. facilitator guides, facilitator presentation material, participant resources) support the implementation of an iPad Faculty Development Program (iFDP)?

(3) What best practices relating to design, delivery, and evaluation can be derived from the implementation of an iFDP?

(4) Does iPad adoption increase in FMP who participated in the iFDP?

What is interesting, is that although there needs to be changes in the protocol and changes in the way in which the iFDP is created, the actual research questions don’t change (this is a good thing) .. this also helps me to ensure that I am not losing focus on this project, I am simply adapting to the needed changes.

I do, however, need to find more time to work on the project. I am finding myself running from one thing to the next, and rushing through things. I need to step back and try to ignore the time pressures .. and allow things to unfold in the time they need.

Re-focusing

After the delivery the first workshop, the overwhelming comments were that the physicians wanted a resource to support the workshop. Seeing the information at the workshop was useful, but they wouldn’t retain that information.

After much thought and consideration, I began the development of a website – but that format didn’t see to work – so I began the development of an eBook.

The eBook is now evolving. I have two (or three) physician champions who are working on the eBook with me. Is it now looking like the eBook will be the star of the show – that is, it is will be focal point in the program, with the occasional workshop that uses the eBook. The eBook development really addresses the research question: How can iPads be used to support teaching and supervising residents in family medicine?

The latest outline of the iBook draws upon our experience with Essential Teaching Skills (another faculty development program delivered within the department of family medicine) – there are three sections: iPad essentials, cloud essentials, and clinical teaching essentials.

The website will be redesigned to align with these three concepts – largely because the current flow of the website it tied to workshops, which aren’t being attended.

Our current hypothesis is that people will only attend iPad workshops if they are adjunct to another event – such as the community retreat or annual retreat. In those events the workshops are well attended, but when we schedule workshops on their own, no one comes. We will try to schedule more workshops, and make them available at sites – but if we don’t get enough people they will be cancelled. The eBook is still valuable in and of itself, and so the study needs to show the value of the eBook as a resource to help improve the use of iPads in clinical teaching.

Now I need to go back to ethics and change my protocol and consent forms – the way this research project is playing out is very different than I anticipated. I had anticipated partnerships from peer facilitators, but really the partnership is one peer facilitator and a couple of other champions who are helping to support the project. Life is getting in the way … and of course that isn’t helping matters.

Program Evaluation

Reading through Guskey (2002), I am reminded that I need to be asking questions relating to the structure and organizational support in my end of session surveys.

Questions to ask on the survey:

Would you have attended this workshop if it were offered on its own (that is, not as part of the retreat)? Please elaborate on your willingness/ability to attend professional development workshops that relate to the use of technology such as the iPad.

eBook and Website design considerations

In another project, I’ve been producing iBooks. The eBooks have video, which had to be close captioned, and posed a problem. When I added the captions, the video file format meant that the files were huge. This, in turn, made the iBooks huge. One book was 1.5GB (actually, it got to 7GB, but the iBooks Store won’t publish anything over 2GB). One of the solution to this problem is to stream the videos from YouTube (or somewhere else) rather than to embed the videos. This requires a trade off – to view the video clips, the reader needs to be online. If you are using a data connect, this can get expensive (I will need to figure out how this works).

I polled the physicians in attendance at the Faculty of Medicine Fac Dev day – there were about 50 people there. Overwhelmingly they preferred the streaming option to the embedded option.

Why does this matter? Because as I add more and more videos to the ipad-fm.ca eBook, the larger the eBook will get. I’m already finding the eBook to be bigger than I’d like.

So, I explored embed options. I began with Bookry.com, but found their widgets to involve too much advertising. I really disliked the look of it. I tried iAd, but I couldn’t get it to accept a video URL, only an embed video which defeats the purpose. I tried iboosgenerator.com, which was free, but the quality of the videos streamed was horrible – the pictures were fuzzy. I tried Bookwidgets, which presented a nice widget – with good quality. I was ready to purchase a licence, but didn’t know which one I should buy. It is confusing because I’m the sole producer, but I am producing as a consultant, but all my clients are educational institutions. Their models didn’t line up with my use case, so I emailed them. They kindly gave me a free license since my primary use was for my PhD Thesis project – that was really kind of them.

So, I shall use BookWidgets (they also have a bunch of other cool widgets – I will be replacing the quiz widget with their widget, but I need to ask about accessibility needs in their widget.

 

Iterations and revectoring

So this project is re-vectoring again. The workshop schedule for Thursday has been cancelled. Already it had a very low sign-up rate – only 5 people signed up and one of them was a resident (I think one was a nurse). Then when the reminder went out two people cancelled. Things came up, and it wasn’t too hard for them to prioritize something else over the iPad workshop.

I interviewed DY yesterday – in part to get her expertise, and her opinion on the program. The perspective is different – when the questions aren’t focused on the assumption of the program, we start to see that the program is ill conceived. There is a lack of sense of need. People are busy, and so without a pressing need, people won’t make a special trip. Now, if you add a workshop onto something they are already doing, then they will come … they just won’t come solely for the workshop. That in and of itself is interesting. I will need to ask that specific question …

So, now I am revectoring. Had a meeting with CJ today to go over changes in the program – and talk about how things don’t at all look like they did at the proposal stage. Afterwards, it occurs to me that the iterations in DBR are happening, they just aren’t happening at the workshop level – they are happening at the program level. Each new bit of information is causing the program to be re-structured – redesign. It isn’t each “workshop” that is being iterated, rather it is the program as a whole. It is like ‘click’ a piece of the puzzle just hit me – I needed to be thinking about the bigger picture – because it is the bigger picture where the insights are happening.

Another important insight is that I don’t have a true “early adopter” champion. I should have seen this gap sooner, as all the literature talks about the importance of it. I thought I could make do with the folks I have, but I see now that I don’t have the  champion support that I need. One of the assumptions of this workshop is that iPads can improve resident education. This is proving to be a false assumption, that this a challenge. There might be a few earlier adopters out there for whom this is the case, the problem is, I don’t have any of them at uO DFM – so the organization I’m working with does not have the champion earlier adopter that I need. Without that, the program just be “Rebecca’s” pet project – it will not be the program it needs to be.

So, here is a summary of what I’m emailing to CJ – talking about the iterations and where I’m currently at …

I was thinking about research and iterations. When we talked today, I found myself thinking that I did not see a problem with where things were. It occurs to me that it is in part about perspective as to what makes an iteration. If you take my research at a “program” perspective, rather than a workshop perspective, then the design of the program has already gone through several iterations. I think that is the better way to describe it. The workshops are just a portion of what makes up the program. My focus is on the program as a whole, rather than the individual workshops.

So, the first iteration is the one based upon literature and my knowledge at the time. This is what I wrote in the proposal.

Iteration two happened after I did the interviews. In meeting with Jay and Mad before presenting the first workshop, we restructured how the workshops were formatted – this represents the second iteration of the program. This version of the program was written up for the MainPro application. It already looked quite different from the version that was in the proposal.

Iteration three happened after we finished the first workshop and responded to the evaluations. I’ve build an iBook, a website, and handouts for the workshop. I also make some changes to how the workshop was to be delivered. The plan was to test this version of the “program”, in part through delivery of the workshop, but it got cancelled.

Iteration four is happening now – as a response to the workshop being cancelled. This is causing a re-reflection on the program as a whole. I will look at what the current program looks like based upon the new information, and create a new design. The testing of the new design will begin at the retreat.

In essence, what is happening is the “program” as a whole is being re-designed as a result of each piece of new information. Something happens (either interviews, a workshop, or in this case the cancellation of a workshop), that causes a design change in the program.

Of course, this demonstrates an issue with DBR, and one of the questions that comes up – when do you stop iterating? When do you stop making changes to the program, as this could go on forever. It is actually a known issue with this type of research.

So, over the summer, I shall re-conceptualize the “program” based upon the information that I currently have, including the interviews I did last week with Mad and Dee. My thought is that iteration four will be the last iteration of this project – at least as far as my dissertation is concerned – although depending on how things go, I may be convinced to do an iteration five. I am not short on data.

Initial Design and Train the Trainer

The initial design of the iPad workshops was based upon what we had learned in the Essential Teaching Skills (ETS) program (MacDonald et al, 2013). The ETS program used a train-the-trainer model for the initial role out of the program. Each of the units within the department were asked to send one person to attend the train-the-trainer. The attendees were then expected to teach the course at their units. This has had a mixed review. It was successful at getting the content delivered to each of the units, and to create a body of expertise. However, the trainers were not confident in delivering the workshops, and some of the trainers were not good at it. This meant the the quality of what was delivered at the units was mixed.

At the time I created the initial design for the iPad workshops, the train-the-trainer model was seen as a success. We are now, after further reflection, looking at abandoning it for the ETS program. The change we are now considering is to have champions at each site attend the pilot delivery, however, they will not be expected to teach the course. Rather, we will be selecting two or three people who will be champions of each ETS course. After the first year, each course is offered once or twice per year. As a result, we do not need an instructor at each unit, rather we need a select few instructors who can go to where ever the given course is needed.

For the iPad program, it became apparent during the initial round of interviews that a train-the-trainer program would not work. This is largely because so few preceptors felt comfortable with facilitating iPad workshops. Actually, very few people saw their skills as enough to even be interviewed! A theme during the interviews was “I’m not an expert” or “I don’t have much to say” – and yet, they all had useful things to say.

I’m wondering now – although it may be too late – if the new model would work better. If offering the course and asked each unit to send one person for the course – would that be effective? If after sending one person per course, that person could then judge whether or not the course would be valuable for their unit. It might actually make for a better dissemination process.

Connecting with Wes at uCalgary – quick thoughts

Great meeting today via go to meeting with Wes from Calgary.

He has a database of apps – but it is internal only for access – it is great place to put apps once people are up and running.

Helped to describe where the iBook fits – it is an eBook designed to help physician transition to using the iPad for teaching – so the focus of the eBook would be on “the first six months” of the iPad user.

The app database is great once the physician know what they are looking for.

We talked about the need for “evidence base” when reviewing medical apps – which we currently don’t have in the eBook – we should look at a way to provide “evidence” or review criteria for the medicine related apps – this is currently a gap in the eBook.

We are going to put together another grant application to create a web app that allows us to test the evaluation tool that we are developing. The evaluation tool will start with what Wes has done and we’ll figure it out from there …

Going to have to find some collaborators.

Will ask Wes to author a couple of chapters in the iBook – helping to fill in some of the gaps.

 

iBook update

My last update submitted to the iBooks Store was April 21, and it is still listed as “under review” (so two weeks). I think the next time I submit it, I’m going to change the category to Medicine / Education – as the next version will have medical apps in it. Hopefully that will improve the turn-around time on the review process.

As I processed Jay’s presentations, I found myself struggling with the structure of the eBook. I wasn’t sure how best to layout the eBook so that it flowed. I’ve now moved the apps into a flow that aligns with the structure that Jay has been introducing in his presentation. First, apps that replace physical objects, then apps that enhance physical objects, and finally apps that do things we couldn’t do with physical objects. The idea is that one is not “better” then the other, it is just a progression that represents a way to think about different apps. The categorization helps when evaluating new apps.

With the creation of the eBook, I’m struggling with the structure of the website. I’ve been structuring the website to align more with the workshop structure (so Ice Breaker activities, Skills Lab Tutorials, and Case Vignettes). The eBook is more of a reference that flows from start to finish – so the Apps are just placed in the eBook based upon the categorization structure rather than activity structure. I think this is actually going to work / make sense – since people don’t read the website linearly – the website is really just a resource that is accessed when someone wants to look something up. I do visualize someone “browsing” through the pages of the eBook.

My goal for today is to finish the edits to the eBook for the workshop. With any luck, I’ll send in a new version / update to the Apple Store and cross my fingers that the update gets approved within a week. If I only knew what was causing the delay, I might be able to do something to help move it along.