We talk a lot about evaluation in the learning world – well, we talk a lot about it everywhere, really – and we always come back to the same question. “How can I measure effectiveness? How can I create an objective measure that I can use against my program?”
Well I’d like to help you out with this question by asking you think about spaghetti.
Stay with me.
It’s a common joke online and at dinner tables – we never seem to be able to gauge the amount of spaghetti noodles we should cook. We either cook just enough to look like we heated it up on a hotplate in our meager college dorm room, or we are preparing food for a gigantic wedding, and we want the happy couple to take a bath in pasta. There never seems to be any in between. This problem occurs because the size of the pasta changes as it cooks. It takes on water, swells up, and boom – leftovers for six days.
Have you ever cooked the RIGHT amount of pasta? If you have, then kudos – many have tried and few have succeeded. If you have, what were your thoughts? As you ate the last bite, did you think to yourself, “Wow, that was the PERFECT amount. I’m full, but not hating-myself full, and I wish I knew how much this was so I can cook that amount again.” If so, how would you go about doing that?
The answer here is the same for anything that needs evaluation – you have to pay attention to your evaluation measures beforehand as well as after.
When we cook spaghetti, we need to measure the pasta BEFORE we cook it, and then evaluate afterward whether that was the right amount. If we start to evaluate our pasta needs after it has started cooking, it’s too late.
So let’s try this.
The next time you go to cook spaghetti, pick out your amount. It doesn’t matter how much or how little – this is your first experiment. Measure how much it is – weight it, hold it against your thumb, whatever. THEN, once you know how much pasta it is uncooked, cook it. Then, if it’s too much, try less next time. Or if it’s too little, try more. But the key here is to measure before you cook it.
It’s the same with any learning endeavor, or any project. Before you start the project, measure the current state. What are you trying to change? Sales numbers? Take a look at sales numbers now. Are you trying to decrease the number of support tickets? Run those analytics. This will get you thinking about your evaluation before you ever get started, which is the right time to think about it! And it will help you decide how MUCH you want these numbers to change. Then, as you develop the program and deliver the program, you can measure your metrics again. Did they change? By how much? Was it enough? Not enough? This will allow you to take a look at your deliverables and see what needs tweaking. Then you try again!
So like I said, start evaluating beforehand. If the pasta’s already cooking, we won’t know how to replicate the amount. (I suppose we could count each individual strand of spaghetti, but I couldn’t give a good ROI recommendation on that use of your time) But if you measure your pasta only after it’s been plated and eaten, you’ll only ever be able to guess.