If it can be measured, it can be valued

Training is often viewed as a cost by companies and therefore is on the chopping block when costs need to be cut. Much like the arts in K12 education, training is seen as a "nice to have," but not a core function of the organization.

Why is training viewed as a cost and not a strategic investment? Because learning professionals don't measure the impact of training on the performance of the organization.

Think about the last time you delivered a training program in an organization. Or the last time you participated in training, if you aren't an L&D professional. How was the success of the training measured? 

How many completed the training?

A passing score on an assessment? 

A subjective "smile sheet" ranking where learners reported how valuable they thought the training was?

Do these training programs measure improvements in employee performance? Decreased time to market/increased productivity? Customer satisfaction ratings? Increased revenue? Reduction in costs? Increased employee retention?

No? WHY NOT?

If training is a strategic investment, then it is absolutely critical that we show the value of the investment. Attendance is not value; increased sales is. Assessment scores are meaningless if people don't perform their jobs better. 

If our role in an organization is to improve performance, then shouldn't we be measuring performance? 

Only when we start measuring the right things will training be valued. 

Your success metrics aren't unique (or, why are we making analytics so hard?)

Last week I had the pleasure of attending TechKnowledge 2013, including giving a workshop on game design with Karl Kapp and presenting a concurrent session on immersive design. Two action-packed days in San Jose, for sure, and for once I got to see some of my esteemed colleagues' presentations as well.

I was most excited to see Ellen Wagner present on learning analytics and get a sneak peek into some of the work that she's been doing with the Gates Foundation. I was not disappointed; I loved hearing Ellen's practical advice to the packed room on how to approach analytics and not be intimidated by big data and the necessity of quantifying learning...or improvement, really...for our non-learning business colleagues. I appreciated her encouraging us to be able to frame the value that we bring to the organization in business terms without abandoning our expertise that qualifies us to take on an important organizational leadership role.

Watching Ellen's presentation left me thinking (which is normally a very good or very dangerous thing).

Every presentation that I've seen on analytics and big data end with the same call to action: "now, go back to your organization and figure out what you should be measuring."

Photo by WH "Snowflake" Bentley

Here's my challenge to you, my esteemed learning colleagues: shouldn't we all be measuring (basically) the same thing? We all suffer from the snowflake syndrome (I'm just like everyone else! I'm unique!), but to be fair, both statements are true. The real question is how much are we alike and how much are we different?

About ten years ago, I was working with a gentleman who had done an in-depth analysis across 15+ pharmaceutical companies, looking at what made their sales representatives successful. He found that 80% of the success indicators were exactly the same across all companies. The unique 20% related to the type of product, the therapeutic market the drug was being sold into, and the corporate culture differences among the companies.

Now, if you're a pharmaceutical company, where do you spend your analysis energy, knowing this? Right. Don't worry about identifying success markers for the 80%: use the identified metrics, provide training and support for those areas, and spend your analysis time identifying what the other 20% are that are unique to your company. EVERYONE should be measuring that 80%...that's baseline. Figuring out the 20% is what differentiates you from your competitors and supports your brand.

Could the same analysis be achieved for call center support? Project managers? Database engineers? Are there consistent success metrics that are common across industries?

If we could identify the success metrics across within or across industries, we wouldn't have to start from scratch to determine our analytics. We SHOULDN'T have to start from scratch. Don't we know what these things are? Can't we say, "Hey, you want to measure your organizations' success and performance improvement? Here is 80% of what you should be looking at, and what it means. Now, figuring out the 20% that's unique to your organization is up to you." (I'm not saying that it's literally 80% for every job function or every industry...but you get the idea.)

As learning professionals, we should know these benchmarks for our industries. If we don't, that is our very first starting point: find the "80%." What makes us the same? What makes us unique? Identify it, benchmark it, measure it. Then? Spend your time analyzing what makes you better than that 80%.

Your "assessment" is probably bullshit

Every year, the first morning of DevLearn, I usually wake up with some crazy epiphany about learning and training that sticks in my brain throughout the conference. This year the revelation was "assessment is meaningless." I decided to try to spend the conference (when I wasn't presenting or hosting the Emerging Tech stage) trying to find examples of meaningful assessment.

My mission was by and large futile.

Let me start by clarifying: for adult education and training, knowledge might be important for dinner party conversation, and it might even help you in a job interview, but more than likely what you know isn't a fair representation of what you can do. Learning objectives are very different than performance objectives.

Why are we still writing learning objectives and assessing for knowledge acquisition when the only thing the business cares about is what employees can do?

If you're an instructional designer and you're writing learning objectives, please stop. STOP. What you learned in your graduate program or by reading Dick & Carey is meaningless to business objectives and business leaders don't know what the hell ADDIE is. Executives don't care if employees passed all of their knowledge checks with 100%. The bottom line is not improved by multiple choice questions. WHAT YOU ARE MEASURING AS A BENCHMARK OF LEARNING IS MAKING INSTRUCTIONAL DESIGN OBSOLETE. Stop stroking your own ego. Stop trying to justify your decision to make a click-through elearning module or to drag the salesforce out of the field for two days for training by showing they all passed a multiple choice test. NO ONE BELIEVES YOU and NO ONE CARES (yes, I'm yelling).

Businesses care about performance. What are employees doing, and what do they need to do better, to improve business function. Is what you're teaching employees helping the business make more money or saving the business money? SHOW THAT. Use those words. Measure performance. Make what you do meaningful to your organization by setting performance objectives and measuring against those.  Provide performance support. Stop making the easy decisions. Make meaningful and relevant design choices.

If you can't assess something meaningful, it might be better to not assess anything at all.

/drops mic. walks off stage./

You can't measure learning, but you can measure behavior

I'm overstating it a bit with the title of this post, because sure, you can measure knowledge acquisition by pre-testing and post-testing, or iterative assessment. I know, I know...we can measure how much someone knows because we have standardized tests! (I really hope the sarcasm is evident in text...)

I spent the last three days at the mHealth Summit in Washington, DC and 19 hours manning the Ayogo booth, talking to amazingly interesting people about the potential of games to improve health outcomes. What mattered to everyone? It wasn't what people know...amazingly, most everyone actually knows what they need to do to be healthier. The challenge is to get people to actually DO those healthy things that will help them better manage their diabetes, reduce their risk for cardiovascular disease, etc.

When it comes to health, but really when it comes to ANYTHING, there is a knowing-doing gap. We all know this...so then why are we as a learning profession settling for assessing knowing? Knowing is not doing. The proof is in the behavior, and behavior can be easily measured.

We live in an age where everything we do is tracked. Do you carry a cell phone? Your wireless carrier knows where you take that phone all day, every day. Do you use a credit card or bank card? All of your purchases are tracked. Do you log onto the Internet? Every site that you visit is logged and recorded (yeah...I know...you delete the history. That just means your kids won't see those sites your visiting...but your Internet service provider still knows).

All of that data, and more...everything you post on Facebook, Twitter...everything you email...anything you do is trackable now. And more ways to track behavior are being created every day...sleep monitors, pedometers, glucose monitors...there is data EVERYWHERE and its all about you. And me. And the guy sitting in traffic next to you who's using his gps.

With all of this data, we can start making predictions about future outcomes. We can target specific communities or subsets of employees, populations, learners. We can provide information to the most relevant audiences in the most appropriate places.

As learning professionals, we should be thinking more closely about the implications of that data and what it means to know so much about a person's current status and the implications for her future status. Can we change the future? Why yes...yes we can. We can observe current behaviors, predict future outcomes, and use our expertise in learning and performance improvement to change behavior to improve those future outcomes.

We have access to so much behavioral data. How do we get people to change their behavior, when we know that people operate in a world of short-term benefit over long-term reward? We're not going to change those behaviors through knowledge training...we'll only change them through behaviorally-focused training. Games, simulations, contextualized practice...immersive learning environments are the bridge between having access to data and changing behavior for better results.

We can, and already do, measure behavior in almost every aspect of our lives. Learning professionals need to stop focusing on knowledge and start focusing on behavioral change as the basis of our design practice or risk obsolescence (see: Instructional Design is Dead). Our jobs aren't about making sure people know things...they are about making sure people can do things better. We can design those experiences and measure those outcomes. If we aren't doing that, we're not doing our jobs.

Gauntlet thrown.