Program Evaluation

Reading through Guskey (2002), I am reminded that I need to be asking questions relating to the structure and organizational support in my end of session surveys.

Questions to ask on the survey:

Would you have attended this workshop if it were offered on its own (that is, not as part of the retreat)? Please elaborate on your willingness/ability to attend professional development workshops that relate to the use of technology such as the iPad.

eBook and Website design considerations

In another project, I’ve been producing iBooks. The eBooks have video, which had to be close captioned, and posed a problem. When I added the captions, the video file format meant that the files were huge. This, in turn, made the iBooks huge. One book was 1.5GB (actually, it got to 7GB, but the iBooks Store won’t publish anything over 2GB). One of the solution to this problem is to stream the videos from YouTube (or somewhere else) rather than to embed the videos. This requires a trade off – to view the video clips, the reader needs to be online. If you are using a data connect, this can get expensive (I will need to figure out how this works).

I polled the physicians in attendance at the Faculty of Medicine Fac Dev day – there were about 50 people there. Overwhelmingly they preferred the streaming option to the embedded option.

Why does this matter? Because as I add more and more videos to the ipad-fm.ca eBook, the larger the eBook will get. I’m already finding the eBook to be bigger than I’d like.

So, I explored embed options. I began with Bookry.com, but found their widgets to involve too much advertising. I really disliked the look of it. I tried iAd, but I couldn’t get it to accept a video URL, only an embed video which defeats the purpose. I tried iboosgenerator.com, which was free, but the quality of the videos streamed was horrible – the pictures were fuzzy. I tried Bookwidgets, which presented a nice widget – with good quality. I was ready to purchase a licence, but didn’t know which one I should buy. It is confusing because I’m the sole producer, but I am producing as a consultant, but all my clients are educational institutions. Their models didn’t line up with my use case, so I emailed them. They kindly gave me a free license since my primary use was for my PhD Thesis project – that was really kind of them.

So, I shall use BookWidgets (they also have a bunch of other cool widgets – I will be replacing the quiz widget with their widget, but I need to ask about accessibility needs in their widget.

 

Iterations and revectoring

So this project is re-vectoring again. The workshop schedule for Thursday has been cancelled. Already it had a very low sign-up rate – only 5 people signed up and one of them was a resident (I think one was a nurse). Then when the reminder went out two people cancelled. Things came up, and it wasn’t too hard for them to prioritize something else over the iPad workshop.

I interviewed DY yesterday – in part to get her expertise, and her opinion on the program. The perspective is different – when the questions aren’t focused on the assumption of the program, we start to see that the program is ill conceived. There is a lack of sense of need. People are busy, and so without a pressing need, people won’t make a special trip. Now, if you add a workshop onto something they are already doing, then they will come … they just won’t come solely for the workshop. That in and of itself is interesting. I will need to ask that specific question …

So, now I am revectoring. Had a meeting with CJ today to go over changes in the program – and talk about how things don’t at all look like they did at the proposal stage. Afterwards, it occurs to me that the iterations in DBR are happening, they just aren’t happening at the workshop level – they are happening at the program level. Each new bit of information is causing the program to be re-structured – redesign. It isn’t each “workshop” that is being iterated, rather it is the program as a whole. It is like ‘click’ a piece of the puzzle just hit me – I needed to be thinking about the bigger picture – because it is the bigger picture where the insights are happening.

Another important insight is that I don’t have a true “early adopter” champion. I should have seen this gap sooner, as all the literature talks about the importance of it. I thought I could make do with the folks I have, but I see now that I don’t have the  champion support that I need. One of the assumptions of this workshop is that iPads can improve resident education. This is proving to be a false assumption, that this a challenge. There might be a few earlier adopters out there for whom this is the case, the problem is, I don’t have any of them at uO DFM – so the organization I’m working with does not have the champion earlier adopter that I need. Without that, the program just be “Rebecca’s” pet project – it will not be the program it needs to be.

So, here is a summary of what I’m emailing to CJ – talking about the iterations and where I’m currently at …

I was thinking about research and iterations. When we talked today, I found myself thinking that I did not see a problem with where things were. It occurs to me that it is in part about perspective as to what makes an iteration. If you take my research at a “program” perspective, rather than a workshop perspective, then the design of the program has already gone through several iterations. I think that is the better way to describe it. The workshops are just a portion of what makes up the program. My focus is on the program as a whole, rather than the individual workshops.

So, the first iteration is the one based upon literature and my knowledge at the time. This is what I wrote in the proposal.

Iteration two happened after I did the interviews. In meeting with Jay and Mad before presenting the first workshop, we restructured how the workshops were formatted – this represents the second iteration of the program. This version of the program was written up for the MainPro application. It already looked quite different from the version that was in the proposal.

Iteration three happened after we finished the first workshop and responded to the evaluations. I’ve build an iBook, a website, and handouts for the workshop. I also make some changes to how the workshop was to be delivered. The plan was to test this version of the “program”, in part through delivery of the workshop, but it got cancelled.

Iteration four is happening now – as a response to the workshop being cancelled. This is causing a re-reflection on the program as a whole. I will look at what the current program looks like based upon the new information, and create a new design. The testing of the new design will begin at the retreat.

In essence, what is happening is the “program” as a whole is being re-designed as a result of each piece of new information. Something happens (either interviews, a workshop, or in this case the cancellation of a workshop), that causes a design change in the program.

Of course, this demonstrates an issue with DBR, and one of the questions that comes up – when do you stop iterating? When do you stop making changes to the program, as this could go on forever. It is actually a known issue with this type of research.

So, over the summer, I shall re-conceptualize the “program” based upon the information that I currently have, including the interviews I did last week with Mad and Dee. My thought is that iteration four will be the last iteration of this project – at least as far as my dissertation is concerned – although depending on how things go, I may be convinced to do an iteration five. I am not short on data.

Initial Design and Train the Trainer

The initial design of the iPad workshops was based upon what we had learned in the Essential Teaching Skills (ETS) program (MacDonald et al, 2013). The ETS program used a train-the-trainer model for the initial role out of the program. Each of the units within the department were asked to send one person to attend the train-the-trainer. The attendees were then expected to teach the course at their units. This has had a mixed review. It was successful at getting the content delivered to each of the units, and to create a body of expertise. However, the trainers were not confident in delivering the workshops, and some of the trainers were not good at it. This meant the the quality of what was delivered at the units was mixed.

At the time I created the initial design for the iPad workshops, the train-the-trainer model was seen as a success. We are now, after further reflection, looking at abandoning it for the ETS program. The change we are now considering is to have champions at each site attend the pilot delivery, however, they will not be expected to teach the course. Rather, we will be selecting two or three people who will be champions of each ETS course. After the first year, each course is offered once or twice per year. As a result, we do not need an instructor at each unit, rather we need a select few instructors who can go to where ever the given course is needed.

For the iPad program, it became apparent during the initial round of interviews that a train-the-trainer program would not work. This is largely because so few preceptors felt comfortable with facilitating iPad workshops. Actually, very few people saw their skills as enough to even be interviewed! A theme during the interviews was “I’m not an expert” or “I don’t have much to say” – and yet, they all had useful things to say.

I’m wondering now – although it may be too late – if the new model would work better. If offering the course and asked each unit to send one person for the course – would that be effective? If after sending one person per course, that person could then judge whether or not the course would be valuable for their unit. It might actually make for a better dissemination process.

OtoSim Companion

App: OtoSim Companion
Cost: Free App ($4.99 to unlock)
Function: Enhances


Teaching ear examination in the clinical setting can be very challenging. The first problem is that it may be difficult to provide the learner with the opportunity to see all of the pathological entities that they might encounter in practice. The second problem is that most books present a complete picture of the tympanic membrane rather than the small segments that are often seen with the otoscope. There are stimulators that can be purchased to offer trainees a wide range of clinical content. However, the OtoSim Companion app for the iPad provides a cost effective alternative. By selecting an image and then having the learner examined the image through an ear speculum from approximately 10-15 cm away from the iPad screen, it is possible to simulate what they would actually see through an otoscope.

The OtoSim Companion app contains a series of images with descriptions that include both common presentations and conditions that occur rarely and as such are unlikely to be encountered by a trainee. The simulation allows learners to observe a much wider breadth of clinical material than they would be likely to see during their regular rotations.

The app itself is free, but the free version only contains a few sample chapters. The remaining chapters can be unlocked through an in-app purchase for $4.99.

Challenge Activity

In groups of two, one person open the OtoSim app and choose an image. The second person, using an ear spec, examine the image and determine what they are seeing.

SimulusDM

App: Simulus DM by Simation Medical Ltd.
Cost: free
Function: Provides something new


The SimulusDM App provides simulation around various diabetes treatment options for patients . It allows the user to select a patient from several patient profiles with diverse variable (type of DM, age, weight, past treatment and lab results etc.). Once a patient profile has been determined, the app user can modify the patient’s glycemic physiology through modifications in lifestyle choices and medications. The app will then provide a visual representation of the effects of the modification on blood sugar over the course of three months. . The app supplies information in both metric and US units.

Note that this app requires that you be connected to the Internet. It is memory intensive, so you may wish to close any apps you do not need before using this.

From a teaching point of view, one of the biggest benefits to this type of simulation app is that you can allow residents to make treatment decisions that you would not be feasible or appropriate in real time clinical settings. It allows residents to see the consequences of various management decisions and learn from their mistakes without risk to patients.

iPad User Manual

A common complaint amongst iPad users is the perceived lack of a ‘user manual’. Apple does make an iPad User Guide, conveniently available online for each release of the iOS (operating system). The manual is available in PDF format or as an eBook for iBooks, available for free in the iBooks Store.

[tboot_button_group_vertical size=”md” ]
[tboot_button color=”primary” url=”http://manuals.info.apple.com/MANUALS/1000/MA1595/en_US/ipad_user_guide.pdf” title=”iPad User Guide for iOS 7.1 Software (PDF Version)” target=”blank”]iPad User Guide for iOS 7.1 Software (PDF Version)[/tboot_button]

[tboot_button color=”primary” url=”https://itunes.apple.com/ca/book/ipad-user-guide-for-ios-7.1/id709634245?mt=11″ title=”iPad User Guide for iOS 7.1 Software (iBooks Version)” target=”blank”]iPad User Guide for iOS 7.1 Software (iBooks Version)[/tboot_button]
[/tboot_button_group_vertical]

 

 

Connecting with Wes at uCalgary – quick thoughts

Great meeting today via go to meeting with Wes from Calgary.

He has a database of apps – but it is internal only for access – it is great place to put apps once people are up and running.

Helped to describe where the iBook fits – it is an eBook designed to help physician transition to using the iPad for teaching – so the focus of the eBook would be on “the first six months” of the iPad user.

The app database is great once the physician know what they are looking for.

We talked about the need for “evidence base” when reviewing medical apps – which we currently don’t have in the eBook – we should look at a way to provide “evidence” or review criteria for the medicine related apps – this is currently a gap in the eBook.

We are going to put together another grant application to create a web app that allows us to test the evaluation tool that we are developing. The evaluation tool will start with what Wes has done and we’ll figure it out from there …

Going to have to find some collaborators.

Will ask Wes to author a couple of chapters in the iBook – helping to fill in some of the gaps.

 

iCloud

By Rebecca J. Hogue & Madeleine Montpetit

iCloud is an umbrella term used for the various cloud-based services provided by Apple. iCloud services are integrated into all your Apple devices (iPhone, iPad, Mac). iCloud services are actively being developed and evolving, and as a result, new features are being delivered on a regular basis.

iCloud provides services that help to make your iPhone, iPad and laptop more secure. At a minimum, we strongly recommend that you enable iCloud for the Find My Phone functionality.

Please read the following article specifically addressing how you can make your iPhone and iPad more secure when using iCloud in the medical context: iPhones, iPads and HIPAA-Compliant Practice: Locking Down Your Apple Device.

In addition iCloud can be used  to synchronize some of your data from your iPad and iPhone to the cloud. When you enable iCloud on your device, you can configure iCloud synchronization for each type of data individually.

Which services you choose to use are up to you. This will depend on your personal strategy for backup and cloud-based synchronization services available. At a minimum, I highly recommend you turn on Find My Phone as it aligns with the HIPPA-Compliance Practice.

[tboot_accordion_bootstrap name=”UniqueName”]
[tboot_accordion_bootstrap_section color=”primary” name=”UniqueName” heading=”Security and privacy” number=”1″ ]

As a general rule, before using any cloud-based application you should consider the security and privacy policy of the company providing the app. Most companies that provide cloud-based services provide an overview of their security and privacy policies on their websites.

For example, if you are concerned about the privacy and security of any of the data you choose to store in iCloud, review the iCloud security and privacy overview.

[/tboot_accordion_bootstrap_section]

[tboot_accordion_bootstrap_section color=”primary” name=”UniqueName” heading=”Password management” number=”2″ ]

Passwords abound in today’s internet enabled society. To make our internet use more secure, it is recommended that we use different passwords for each website we access. It quickly becomes impossible to remember all the passwords required to manage our internet use safely. Password Managers are software applications that were created specifically to allow you to securely store and conveniently retrieve your passwords, allowing you to implement unique random passwords for each website you use. In addition, some of the password manager tools allow you to securely store credit card numbers and other important information.

Recently Apple has introduced a new feature known as Keychain that provides password management services. This feature provides many of the same functions as some of the leading password managers; however, it only works on Apple products and software (e.g. iPhone, iPad, Mac, Safari).

If you are going to take the time to implement a password manager solution, at this time we recommend that you use a full service solution such as 1Password. Applications such as 1Password integrate with all mobile devices and all web browsers on both Mac and PC platforms. This means you only need to remember one password (make sure it is a good one) in order to access each of your unique website password.

[/tboot_accordion_bootstrap_section]

[tboot_accordion_bootstrap_section color=”primary” name=”UniqueName” heading=”Find My Phone” number=”3″ ]

Find My Phone is an iCloud service that allows you to track the location of any iPhone, iPad, or Apple computer that is configured with the Find My Phone Service.

To configure the Find My Phone service on your iPad:

  1. Open the Settings app.
  2. Select iCloud.
  3. At the bottom of the list, ensure Find My Phone is selected.

If you have not enabled iCloud, you will need to first activate iCloud. Select Settings > iCloud and follow the prompts (see Managing your Apple ID).

The Find My Phone functions are activated by logging in the iCloud website (http://iCloud.com). After logging in, touch Find My Phone, then select a device from the My Devices list at the top center of the page. Once you select a device, the Find My Phone options are displayed.

FindMyPhone

[/tboot_accordion_bootstrap_section]
[/tboot_accordion_bootstrap]