If it can be measured, it can be valued

Training is often viewed as a cost by companies and therefore is on the chopping block when costs need to be cut. Much like the arts in K12 education, training is seen as a "nice to have," but not a core function of the organization.

Why is training viewed as a cost and not a strategic investment? Because learning professionals don't measure the impact of training on the performance of the organization.

Think about the last time you delivered a training program in an organization. Or the last time you participated in training, if you aren't an L&D professional. How was the success of the training measured? 

How many completed the training?

A passing score on an assessment? 

A subjective "smile sheet" ranking where learners reported how valuable they thought the training was?

Do these training programs measure improvements in employee performance? Decreased time to market/increased productivity? Customer satisfaction ratings? Increased revenue? Reduction in costs? Increased employee retention?

No? WHY NOT?

If training is a strategic investment, then it is absolutely critical that we show the value of the investment. Attendance is not value; increased sales is. Assessment scores are meaningless if people don't perform their jobs better. 

If our role in an organization is to improve performance, then shouldn't we be measuring performance? 

Only when we start measuring the right things will training be valued. 

You can't measure learning, but you can measure behavior

I'm overstating it a bit with the title of this post, because sure, you can measure knowledge acquisition by pre-testing and post-testing, or iterative assessment. I know, I know...we can measure how much someone knows because we have standardized tests! (I really hope the sarcasm is evident in text...)

I spent the last three days at the mHealth Summit in Washington, DC and 19 hours manning the Ayogo booth, talking to amazingly interesting people about the potential of games to improve health outcomes. What mattered to everyone? It wasn't what people know...amazingly, most everyone actually knows what they need to do to be healthier. The challenge is to get people to actually DO those healthy things that will help them better manage their diabetes, reduce their risk for cardiovascular disease, etc.

When it comes to health, but really when it comes to ANYTHING, there is a knowing-doing gap. We all know this...so then why are we as a learning profession settling for assessing knowing? Knowing is not doing. The proof is in the behavior, and behavior can be easily measured.

We live in an age where everything we do is tracked. Do you carry a cell phone? Your wireless carrier knows where you take that phone all day, every day. Do you use a credit card or bank card? All of your purchases are tracked. Do you log onto the Internet? Every site that you visit is logged and recorded (yeah...I know...you delete the history. That just means your kids won't see those sites your visiting...but your Internet service provider still knows).

All of that data, and more...everything you post on Facebook, Twitter...everything you email...anything you do is trackable now. And more ways to track behavior are being created every day...sleep monitors, pedometers, glucose monitors...there is data EVERYWHERE and its all about you. And me. And the guy sitting in traffic next to you who's using his gps.

With all of this data, we can start making predictions about future outcomes. We can target specific communities or subsets of employees, populations, learners. We can provide information to the most relevant audiences in the most appropriate places.

As learning professionals, we should be thinking more closely about the implications of that data and what it means to know so much about a person's current status and the implications for her future status. Can we change the future? Why yes...yes we can. We can observe current behaviors, predict future outcomes, and use our expertise in learning and performance improvement to change behavior to improve those future outcomes.

We have access to so much behavioral data. How do we get people to change their behavior, when we know that people operate in a world of short-term benefit over long-term reward? We're not going to change those behaviors through knowledge training...we'll only change them through behaviorally-focused training. Games, simulations, contextualized practice...immersive learning environments are the bridge between having access to data and changing behavior for better results.

We can, and already do, measure behavior in almost every aspect of our lives. Learning professionals need to stop focusing on knowledge and start focusing on behavioral change as the basis of our design practice or risk obsolescence (see: Instructional Design is Dead). Our jobs aren't about making sure people know things...they are about making sure people can do things better. We can design those experiences and measure those outcomes. If we aren't doing that, we're not doing our jobs.

Gauntlet thrown. 

America's Army and gender bias in game design

Coming out of the #ifest Twitter stream, I once again heard how America's Army was a shining example of how games had been used to improve recruitment efforts. I posed the question...has it improved recruitment of women at the same rate it has improved the recruitment of men? So far, all I've heard back is *crickets*.

For two weeks, I have been looking for data or research on America's Army that mentions gender as a research parameter, but so far, I've found nothing. If you know of any research, I'd love to see it. My hypothesis? Recruitment of women was not as greatly improved after they played America's Army. If that's the case, what does that say about the relative value of recruiting women vs men into our military?

What makes a game successful? Is it ok for public institutions (government, schools, etc.) to measure the success of a serious game without looking at differences in outcomes along the most basic parameters (gender, class, race)? Is it ok to say a game is successful in achieving its goals if we don't consider those issues as part of the discussion?

I'm tired of hearing the marketing spin and the hype around how games can change the world if we're not even asking the most basic questions about WHO games are changing and HOW they are changing them. You won't find a bigger advocate of games for learning and as a vehicle to raise awareness and support behavior change. But not all games are created equal. We have to be vigilant and constantly questioning our design to ensure we're achieving the outcomes we seek. Ignoring questions of gender, class, and racial bias in serious game design makes me question the motives of the design itself and the motives of those promoting a game's "success."

As always, I welcome anyone's comments who can prove me wrong...

Return on expectations

Very often, learning professionals are challenged to come up with justification for training investments by providing ROI figures. Let's be honest, its almost impossible to accurately calculate the ROI for learning. So many factors impact a learner's performance, it is difficult to credit or blame a training initiative for a change in performance, particularly as it relates to monetary measures.

A viable alternative to ROI is measuring ROE, or return on expectations. What do you expect people to DO differently after training? Can you measure that change in behavior? This, in most instances, is a much easier way to measure the success of a training initiative. Have you reduced the number of help desk calls? Have you increased the amount of time a sales representative spends in the field? Have you shortened the average call length for call center reps, or increased call center satisfaction ratings? All of these measurements in some way eventually turn into money...why not advocate the measurement of direct training outcomes as a true measure of a training initiative's "worth" instead of the trickle-down ROI that can't as easily be controlled or claimed as a direct correlation?

Session blog: Using Virtual Worlds for Supporting Business and Enterprise

The following are my notes from a session at Virtual Worlds London...apologies for the sketchiness, I was taking notes live.

Dr. Sara de Freitas, Director of Research

SGI (Serious Games Institute)

Coventry University Technology Park

Trends

• by 2011 80% of all active internet users will have an avatar (Gartner)
  
• currently 80 vws, next year another 100, (100+ aimed at kids)
  
• studies demonstrating the efficacy of serious games for training
  
• wide uptake of social software (eg, facebook, wikipedia)
  
• lines between vws, games and social software are blurring
  

How can you use virtual worlds in business?

• support b2b collaboration and communication
  
• meet with customers remotely located
  
• provide training for sales staff
  
• provide mentoring and support staff

Demos:

*Business Nexus Island (SL)

*New hire orientation on OLIVE

Spending on Gaming:

by 2011, $12.5 billion US, $48.9 billion Global

Demo of serious game: 

Triage

***Data shows significant difference in learning between traditional learning and game play

"New" research on efficacy

Edgar Dale, Cone of Learning (Audio-Visual Methods in Teaching (3rd Edition) Holt

Chi et al 1989

Strengths of virtual worlds for learning:

• accelerated learning
  
• engagement
  
• motivation
  
• role-plays
  
• rehearsal
  
• longevity of learning
  
• experiential/exploratory learning
  

Will Virtual Worlds change how we do business?

Example:

*Virtual Philadelphia

KO commentary:

I thought this was the best session I attended. One reason, it wasn't a panel. Another, it focused on learning and research, and let's face it...that's my thing. I think the opportunity for data and case studies emerging in the next 6 months that demonstrate superior learning outcomes in these types of training programs is all but guaranteed. Once these outcomes have been proven, the future of enterprise learning will change forever.

Signs you are doing something right...

Everyday I feel like I'm drowning in stuff that hasn't gotten done, or things I'm doing wrong. But I had a few conversations today, that after reflection, are probably signs I'm actually doing something right. If you recognize them, they might be signs that help you feel like you're doing something right, too.

• Clients call you asking you to do more work for them
• Old colleagues email you asking for a job
• Clients call you asking for a job
• You confuse your accountants--because you, in this one instance, actually know MORE than them about how to manage your company's finances
• You understand the rest of the conversation with your accountant
• Someone you are wooing to come work with you says yes
• Someone who already said yes asks for more input into the start up process just because they want to be part of something cool
• You take time to meet your spouse and kids for lunch because you're the boss now and can do that stuff sometimes
• You schedule lunch, or drinks, with a competitor
• You schedule lunch, or drinks, with an old friend so you can gossip about stuff that has nothing to do with work