Building the Survey: How Do You Measure Advising Practice?

In the last post, I talked about the study we're conducting — using the SSIPP framework to identify the advising practices that most contribute to student success. This time, I want to get into the guts of it: how do you actually build a survey that captures what advisors do?

I said in the last post that we've studied teaching for decades — that we've identified effective techniques, built training around them, and shared them across institutions. Advising has rarely been treated with the same depth.

One of the reasons for that gap, I think, is that measuring teaching is hard — but measuring advising is even harder. A class session has structure. There's a syllabus, a lesson plan, a room full of students you can observe. Advising, in most models, is a series of individual conversations, spread across a semester, whose content varies wildly based on the student in front of you. What does it even mean to say "measure that"?

That was the challenge we sat down with this summer.

Starting with what SSIPP actually says

The natural place to start was with SSIPP itself. The framework is elegant — Sustained, Strategic, Integrated, Proactive, and Personalized — and it does something a lot of advising frameworks don't. It describes the properties of good advising, not a specific technique. This is powerful because it means the framework can travel across advising models without prescribing a particular flavor of practice.

But that same elegance was our first problem. SSIPP tells you what good advising looks like at a high level; it doesn't tell you what an advisor does on a Tuesday afternoon. If we were going to measure practice, we had to bridge the gap between the framework and the behavior.

Fortunately, CCRC has done more than publish the original SSIPP paper. If you dig into their materials, you'll find checklists, self-assessment tools, and worked examples of what each SSIPP dimension might look like in practice. We used these as our starting anchor. From there, we did something that felt a little audacious at first but ended up being the most useful move we made: we crossed each of the five SSIPP dimensions with the natural time structure of an advising appointment.

Before. During. After.

So instead of asking "how proactive are you?" we could ask what a Proactive advisor might do before a meeting (initiating contact based on anticipated need), during one (raising concerns the student hadn't brought up), and after one (scheduling the next touchpoint rather than waiting). Same thing for Personalized. Same thing for Integrated. And so on.

Not every cell filled in equally. Proactive is almost entirely a "before" behavior; once you're in the room, the meeting is no longer proactive. Personalized, by contrast, showed up powerfully in all three time points — knowing your student before, using that knowledge during, and preserving it after. That variation was itself a finding. It told us something about the shape of each dimension that a flatter measurement approach would have missed.

The result was a matrix. Fifteen cells. Each one asking: what does this SSIPP dimension look like in this phase of an appointment? Some cells were rich; some were thin. Some cells we couldn't confidently fill in ourselves, and we left them as open questions for our partner institution to help us answer.

But behavior isn't enough

Here's where things got more interesting. If you just ask advisors "do you do X, Y, Z" and then look at whether the answers predict retention, you'll learn something. But you won't learn why an advisor isn't doing X, Y, Z when they aren't. And "why" is where the leverage is.

Think about it this way. If Advisor A isn't proactively reaching out to students, that could mean a lot of different things:

• She doesn't think it matters (a belief problem)

• She thinks it matters, but her supervisor doesn't (a norms problem)

• She thinks it matters, but she doesn't feel able to do it given her caseload (an efficacy problem)

• She thinks it matters, feels able to do it, but hasn't formed the intention to actually do it (an intention problem)

Each of those four scenarios points to a completely different intervention. If it's a belief problem, you need training. If it's a norms problem, you need leadership signaling. If it's an efficacy problem, you need workload relief or better tools. If it's an intention problem, you might just need a nudge.

If you only measure behavior, you can't tell those apart. You just see that the behavior isn't happening.

So for each SSIPP dimension, we added four "general" questions before we ever got to the behavioral items. These were drawn from Ajzen's Theory of Planned Behavior — a framework that's been used for decades in health behavior research to understand why people do (or don't do) the things they say they value. The four constructs map cleanly:

• Attitude: Do you value this dimension of advising?

• Norms: Do you think it's expected of you at this institution?

• Perceived Behavioral Control (Efficacy): Do you feel able to do it?

• Intention: Do you plan to?

One question for each construct. Four items per dimension. Twenty items total across the survey — a small addition, but one that fundamentally changes what we can learn.

Because now, when we identify a gap between what SSIPP prescribes and what advisors actually do, we can ask: what kind of gap is this? A belief gap? An efficacy gap? An intention gap? And that changes what we can recommend at the end of this study, in a really important way. It turns "here are the practices that work" into "here are the practices that work — and here's why some advisors aren't doing them, and what to do about it."

One more small but important thing

Something worth mentioning briefly. The survey is aspirational.

That means some of these items describe practices that no advisor is currently doing consistently, at any institution I know of. That's by design. We're not measuring compliance against a standard; we're measuring the gap between current practice and what SSIPP tells us is optimal.

That framing matters enormously for how advisors experience the survey. If you ask someone "do you do this ideal thing?" and they don't, they can feel judged. So we've been very explicit — with our partner institution's leadership, with the advisors themselves, and inside the survey instrument itself — that gaps are the point. Gaps are where growth happens. Nobody's being evaluated. This is baseline.

I mention this because it was probably the hardest part of building the instrument. Not the content. The framing. And we're still learning how to do it well.

What we've got

So — a survey built on SSIPP as the framework, structured around the natural rhythm of an appointment, layered with TPB to distinguish "won't" from "can't" from "doesn't." Seventy-two items in total, most on a six-point agreement scale, with a handful of open-ended prompts to catch the things our items missed.

We piloted it internally, revised it based on feedback from the partner institution's leadership, and launched it a few weeks ago. Results are starting to come in.

I'll write about those next.

P.s. If a survey like this sounds interesting, just reach out to us at connect@ISSAQ.net to see how we can collaborate!