In this episode of Roompact’s ResEdChat, we rebroadcast a past webinar discussing storytelling through data and assessment with panelists from the University of South Carolina and University of South Florida. Through this session, the panelists explore best practices to collect, interpret, and present data to campus partners and departmental leadership to create buy-in and provide the “why” to students and staff. Additionally, panelists share how innovative curriculum assessment strategies can enhance the student experience and support departmental and institutional priorities.
Guests:
- Kirsten Kennedy, Ph.D. (she/her), University South Carolina
- Courtney Durbin (she/her), University of South Florida
- Paige Hicks (she/her), University of South Florida
Listen to the Podcast:
Watch the Video:
Show Notes:
About ResEdChat
Roompact’s ResEdChat podcast is a platform to showcase people doing great work and talk about hot topics in residence life and college student housing. If you have a topic idea for an episode, let us know!
Transcript:
Aaliyah Deggs:
All right, good afternoon, everyone. Welcome to Res Ed 2.0, Innovative Engagement for Student Success and Academic Excellence, live from the University of South Florida. My name is Aaliyah Deggs. I use she, her, hers pronouns. And I’ll be moderating our last session, our last webinar, Evaluation and Planning Residential Education Assessment. I know there have been previous sessions where we talked about the importance and benefits of collecting data and assessment and we’ll further that conversation today. I am so excited to introduce our panelists today. First and foremost, we have Courtney Durbin, the Residence Life Coordinator at the University of South Florida. We also have the Paige Hicks, a Residence Life Coordinator at the University of South Florida. And with us, we have Dr. Kirsten Kennedy, the Associate Vice President of Student Housing and Wellbeing at the University of South Carolina, Columbia.
So as we navigate our new remote world together, we have a few housekeeping rules and we will go over that really briefly. We will be recording and hosting the stream today. Thanks to our friends at Roompact and we want to thank you all for being a part of our team and all of their contributions. Please use the chat function to ask questions, comment on anything you hear today, and share your knowledge as you may be an expert in our profession. The presenters will be reviewing the chat throughout the webinar. As time permits we will answer questions below. Any verbal or chat comments will also be a part of the recording. And lastly, please keep your microphones on mute to reduce any background noise. So is everybody ready to begin? Perfect. So we’re going to start off with our first question. What is the value of assessment within residential education?
Kirsten Kennedy:
Well, I guess I’ll start off with that. I want to start off with just a little story about how, when I first came to the University of South Carolina, I learned that we were doing a lot of assessment, but what I found is when I opened a closet, there were a bunch of books in there where our staff had gone around and collected data from the residents. There’s a lot of qualitative stuff, but they were just shoved in a closet, books, boxes upon boxes of books that they had filled out with. So, yes, we had collected a lot of data, but there was nothing that we could do with it because it wasn’t in a usable form. And so, the value of assessment is knowing that what we’re doing is making a difference and that it’s getting us toward the goals that we want for the department and for our students, but it’s also got to be usable data. It’s not enough to have it. You’ve got to be able to get it back in time for people to make some changes in order to do those mid-course corrections.
The other thing that I found was that when we did have assessment data, it came in October of the next year for the previous year, and so it didn’t allow for any plans be able to change course in order to make the experience better for students, so assessment really starts with having a good mission, vision, and goals. So Yogi Berra has a thing, “If you don’t know where you’re going, you might not get there.” It’s one of my favorites for assessment.
Paige Hicks:
I can go ahead and go next. Hi everyone, my name is Paige Hicks, she, her, hers pronouns. I think those are some great points, and if anything, I would add that the value of assessment within Res Ed is that it allows you to make data-driven decisions. And so not only can you say with confidence that X, Y, and Z program initiative or goal is able to influence the student experience and also student learning. I think that it also helps with getting staff buy-in as well, right? Because assessment allows you to really tell your story at your institution within your department and share that out with others. And so when staff are able to see, “Okay, assessment tells my story, what is my role within assessment,” it allows them to see that, “Wow, I’m able to really understand what my role is, and how I can positively make change, and suggestions within my own area.”
And if anything, I think assessment helps us with our students and our stakeholders. And so if anything, your students can learn like, “Okay, 40% or 80% of students that stayed on campus said that they were able to engage with a faculty member,” so it helps with marketing and getting folks to stay with us, but also to really reach out to our stakeholders as well. Now, especially now during this time with saying that our role within this department, this division, this institution is imperative. We’re really helping with that out-of-classroom learning, and assessment is essentially our receipts to show how.
Courtney Durbin:
I am appreciative of Paige’s analogy of receipts because I think what is important when we talk about assessment is that, ultimately, those receipts allow us to be able to continue to create a positive experience for students that are living with us. And as student affairs practitioners, we often talk about wanting to make sure that we are fostering high-impact practices and making sure that what we’re doing actually matters, and by doing that assessment, especially in a residential setting, we’re able to say students who are living on campus are being retained at the university, and that is something that’s huge, and that shows that our work matters because especially right now and with COVID going on, we’re going to have to be able to articulate that even more moving forward.
And I think the other thing about assessment and something that we always talk about here at the University of South Florida is that we should be scholarly practitioners as student affairs professionals, and by using assessment that really allows us to root our decisions in data, in research, and not just talking about anecdotal evidence or things that we’re seeing, which obviously that stuff matters and being able to say what we see and how we see student interact is important. But also being able to show and use numbers and be able to articulate that afterwards allows us to create more buy-in when we’re talking about the work that we’re doing. And then I think, ultimately, it allows us to have control over the narrative of what is residential education? What is our impact? And also who do we share that story with? And being able to communicate that with our residential students, our staff members that work within residential education, but also, parents and families, they care about these things too. And so being able to share our narrative and have ownership over what that narrative is also really important.
Kirsten Kennedy:
Great. And from an institutional perspective, because I’m a couple layers up from you all, they want to know that there’s value added in the residential experience. We’re an expensive proposition, when you look at the amount of staff that we have and how much all of that costs, including RAs, you’ve got to be able to articulate the return on investment. So it’s not only that you’re like, “Oh yeah, we’re making data-informed decisions,” that’s wonderful, but the institution wants to know what value are we getting. And that shows through in retention to the institution and graduation, which are metrics by which the institution is judged. And you’ve got to have staff be able to make that connection between what they’re doing all the way up to the upper levels, to, “Yeah, we’re impacting something that matters to the institution.” One of the things I like to say is Marriott can run this a whole lot cheaper than we can and if we can’t demonstrate our worth, Marriott will be running our operations, so just to further emphasize Courtney’s point.
Aaliyah Deggs:
No, thank you for that. I think Courtney, as you were speaking, I was literally writing down, share our story, share our narrative, and you said it right after, so thank you. I think you all brought up a great perspective of buy-in. And so the following question is how do you create a lens and culture of assessment as a entry-level professional?
Courtney Durbin:
This is probably my biggest passion here since I’ve started at USF. I think it is important, first and foremost, to acknowledge that everyone has different experience when it comes to assessment. You’re going to have graduate students who had coursework in assessment or who didn’t have coursework. They might not have seen assessment in practice or really even been involved with it as a graduate assistant. So, as we have those staff members transition into a new entry-level position, it’s important to just acknowledge that not everyone’s going to be in the same place but there probably are going to be staff who get excited and who are passionate about data and about how it impacts our work, and so I think from a leadership perspective, how they can create this culture is really tapping into those entry-level professionals who are passionate and tap into those skills and their strengths.
And so something that I’ve been really grateful for is here that Paige and I really have been a part of the movement to create this culture. And that’s because we want to show that our work matters, but also we know that the two of us can’t do it alone and that we need our peers and all of the other coordinators here to also have buy-in too. And so if leadership can tap into those entry-level professionals and say, “Hey, I know that this is something you’re excited about and you’re passionate about, let’s see what we can do.” And then, you let staff run with some of their passion projects. They’re able to articulate and get that peer buy-in, which I think is really important in allowing us to make it more accessible. I think sometimes people think of assessment as this big, lofty, ominous cloud over people’s heads, but it can be simple small data that allows us to see who’s living in our community or the residential GPA, and small data points like that also impact and influence our work.
And so something that we’ve actually done here at the University of South Florida is create community snapshots. And this is something that can be easily done at any institution. You just take a roster, and you pull that roster, and you’re able to create snapshots based on academic major, gender, start year for their academic term, and by using that, we were able to break that down for all 6,300 residential students here, but then also for each individual community. And if you reference the resource packet that I dropped in the chat earlier, there’s a picture of how it went from an Excel sheet to a community snapshot.
I think that was something that we were able to show at training with our professional staff and say, “Here’s what your community’s going to look like in the fall. How can this help guide what you’re doing as a residence life coordinator? How can you support your RAs and your community managers as they support students who are predominantly biology majors? Or maybe you have that one liberal arts study who might feel alone and isolated, and how can you make sure that you’re preparing your staff to work with that specific population?” And so I think being able to create that buy-in and show staff how assessment matters to what they’re doing is something that is really easy for us to be doing.
Paige Hicks:
I think that’s great, Courtney. Something else I think about when I think about creating that lens of culture of assessment is that it has to be something that’s valued top down. It needs to be something that’s really permeated within the fabric of that department, that organization, that everyone values it, and everyone sees their role inside of it, right? Assessment is essential to our work. And I think that the first place to start is to do this during training. Use data from prior years to inform how you show up, what are priorities for the year, how does that inform maybe your curricular programming approach. Something that I really enjoyed this year, that when we created the snapshots as a coordinator, as an entry-level professional here, I’m able to take that snapshot along with our educational plan and think critically about how am I going to support our students.
And so something that I did with my previous graduate student is that she created something called Walk Up Wednesdays where we were able to take that data and see, okay, so we have a lot of folks in this area that are Greek affiliated because I work with our Greek village here, but also wanted to get engaged. And so we use this item called MapChats, and for context, the MapChat is a curricular tool that we use here at the University of South Florida that allows us to critically ask questions around some of our learning goals for curriculum in a very succinct way. And so it’ll allow a student to really answer some of the questions where we want to know, and then for us to gather feedback. And so, from gathering that feedback, we saw a theme, and the theme was that folks wanted to get involved and engage with faculty members. They just didn’t know how.
And so now I’m able to show up critically for my students and say, “Okay, how do I get faculty members in my area?” And so we were able this year to invite faculty fellows over to engage with our students, and they were able to ask questions around, “How do I talk to a faculty member? What do I do during office hours? How do I ask for a letter of recommendation,” or, “How do I get someone to sponsor my research?” And so that is a small way to really empower coordinators, entry-level professionals, to really see a return on their investment, to be a knowledge holder of their community, and so when folks are able to not only see their value, they’re able to make meaning of their story, of that data so it can tell a story, and they can really create an experience for their students.
And also, in terms of COVID right now, we’re going to have to think outside the box too because we can no longer put on 300, 400-person programs and say, “Okay, now fill out my intercession assessment.” And so I think thinking outside of the box is going to be imperative right now, and so what are some strategies that you can utilize now? So for our summer B population, one of our lesson plans involved using GroupMe because folks are already in our GroupMe’s and utilizing the polls, and then from collecting the information from the polls, RAs or our student staff members are able to create many infographics and share that information with their supervisors. And so there’s a lot of ways that we can really engage our entry-level folks in this work is just thinking outside of the box and figuring out how.
Kirsten Kennedy:
I have yet a different view of all of this, so one of the things that I have been thinking about along my career is, first of all, leadership has to buy in, like they said. It also depends on what kinds of questions your leaders ask you. You come with a new project. One of my questions is, how do you know you even need this? What’s your evidence to support that? Going along the lines of a needs assessment. So one of the ways you get buy-in is that you’re going to be asked that, so you better do your homework before you go into your supervisor to ask for resources in some way, shape, or form. But I’ll also say that… You talked about an end-of-session assessment. I actually don’t believe in those. I’ll tell you why. And that is because students are over-surveyed, your chances of getting that filled out on a consistent basis. Some people are just like, “No, I don’t fill out your survey,” and I think there’s a lot with technology we can do to overcome that.
And my other question is, what do you do with that when you’re done if an RA puts that on, they get that feedback, and then what? When you ask them right after the session, of course, they retained a lot, but ask them what they learned four months from now, did they still retain those kinds of things? Did they still have the same lesson? But also knowing that asking them what they learned in and of itself is a tool to help them retain it longer.
One of the things that we’ve done at the University of South Carolina is we have this card swipe system. Everybody who does a program has to turn in an interaction form that lets us know which of the educational priorities this program was aimed at. And so we have a way of knowing how many programs did we do in each of the educational priorities, what the timing of that was. And we have an interactive dashboard. We know everybody who attended, we know all of their demographic information, and our hall directors get information like the best time to have a program in terms of attendance is Sunday night at seven o’clock, so if you have something really important to get out, that’s the date and time that you do it.
They can slice and dice it, and because it’s all real-time, you can check it every week and see how you’re doing on programming. And it has really increased the buy-in because it’s useful. And when they do their one-on-one conversations like you would do in a curricular approach, that’s entered into a database, and they can theme that so that if homesickness comes up a lot, they can make sure that they do a program about homesickness.
It’s very responsive. And so I think you need to be able to close. We talk about closing the loop. We need to do that often. I hate to do, so this is a book that I wrote and edited, in there is the thing that talks about how do you spin closing… If it’s a circle and you’re closing that circle, how do you spin that as quickly as possible so that you can make those course corrections like I talked about earlier? But I think that’s one way to get buy-in is to make assessment data mean something to the people who are collecting it, in particular, and to make it as unobtrusive as possible. So, I think that would be my answer for that. Others?
Aaliyah Deggs:
Sorry, you all gave a lot of great feedback. Thank you for that. Our next question would be, how do you effectively assess a residential curriculum or a programming model?
Kirsten Kennedy:
I think I’m the lead on this one. We did talk prior to coming here. So, a little bit about what I talked about. How do you close the loop? I would also say that in order for your program or your curriculum to be effective, you need to do an environmental scan at least once a year. So, for example, University of South Carolina, we had a president who was very much into leadership, and so you know how it is at an institution, you want to get in good with the president, you want to do things that mean something to that person, so we had leadership programs popping up all over the place. Well, we now have a three-star general as our president, and he is very much an into character and integrity and doing the right things, and so that’s necessarily changing the program model for residents’ life, so I would say to be effective, you have to be aligned with institutional priorities and institutional values.
You also need to make sure that everything that you’re doing maps back to the things you said were important, and what is the reason for that? So I have in the materials that you got is a map back. There are two parts to it. One is when you develop your curriculum or your program model, you want to make sure that every activity maps back to something, and some of that has to do with timing. So I know at the beginning of the year, South Carolina, we’re going to have a ton of barbecues because it’s going to… Not this year, but usually you’re trying to get as many people to come so that they can get connected and meet people, and we give them T-shirts so that they can identify that they’re all from the same community, all those kinds of things. So, don’t feel bad about spending money on T-shirts. I think that barbecue’s a great way, but if you’re having a barbecue in March, my question is, have you not done your job in community building that in March we’re still having barbecues? So this whole notion of scaffolding is an important one.
So not only do your activities need to map back, so do the questions that you ask on a survey. And in fact, when we were preparing for this, I was telling a story about how when I was writing the book, I had a researcher review it, and one of the things he said was, “You need to ask the fewest number of questions possible.” One, because you want them to complete it, but two, because it’s unethical to collect data that you don’t need, so that’s one end of the spectrum. I had a practitioner read the chapter and what he said was, “Storage is cheap. Why would you not ask anything that you could conceivably think that you need later?” But no matter how many questions you ask, you want to know that those questions are going to map back to something that you know, that you know you’re trying to measure at that point. So you don’t want to be asking, “Oh, it would be interesting if we knew X,” because you’re wasting questions. And the more questions you have, the less likely people are to complete the survey.
So, I think to effectively assess the programming model or the curricular model is, you want to be clear about what your educational priorities are, and you don’t want to waste time, effort, and energy on the interesting category because it’s a waste of time and potentially a waste of money and your resources.
I think that’s all I have to say about that. Oh wait, one more thing. You start with the study that you can do, like the ideal study with everything you want to find out, and then that gets chipped away, like, “Well, I’m not actually going to get a random sample. Okay, so I’m just going to take the people who will fill it out,” and then, “I can’t ask this question because that would invade their privacy too much, or it could cause trauma,” kinds of things. And you end up with a survey that you can do knowing that it’s not going to be perfect, but it’s going to be enough to get at what you need to know to be able to report out about your effectiveness.
Paige Hicks:
So, in my current role here at USF, I have the pleasure of working and serving with our curriculum team, and so we do a lot around design… Sorry, implementation, and assessment. And so when I think about how do you effectively assess the residential curriculum, in terms of my role, I think I work a lot with some of our components of curriculum. And so, the first thing I think I would mention is probably using a three-to-one card. And so all of the resources that we included, you can find this on slide seven of our PowerPoint resource that we sent you all, but a three-to-one card is something that’s a exit reflection, is used a lot in K through 12 education. It’s an opportunity for a teacher to assess comprehension and learning from their student and also take that information and see what were areas of interest and possibly identify gap areas.
And so we use three-to-one cards often after a community gathering because, of course, they meet, and then after we want to see what did they learn. And so three-to-one is essentially three, so tell us three things that after attending this session or this program, what are three new things that you learned? Two being, what are two new things that you would like to learn more about? And then one, what is one question you still have about the material presented, or the program, or the presentation? And from there, we’re able to do a little bit of data mining and understand our student experiences, and then, we can maybe even do some course corrective action in the moment and say, “Hey, it looks like we definitely met the mark here,” or, “It looks like we still have a little bit of growing and learning to do.” So instead of maybe waiting until the next school year, what can we do now to add and so we’re not having to just wait?
Another thing that we do is a one-minute reflection card. And so that’s the one all the way on the left, on the slide, on slide seven. And a reflection card is used a lot by faculty and professors, and so it is a rapid response tool, essentially in one minute or less, tell me what you learned from this event. It doesn’t require a lot of probing from the facilitator, but it allows someone to really truly understand, and this session that they just did with this and this PowerPoint or presentation, what did they learn? And you’re able to quantify that yourself to see what are students taking away from that session. We use this a lot after our social justice real-talk series, and so we’re able to see what are students sitting with, what are they holding, and what did they take away from it.
And the last thing I’ll mention is something that we call passports. And so typically, when we invite campus partners to come engage with us with some of our curricular needs if we have multiple campus partners coming out, we’ll essentially create a small little passport, and on the front, it’ll have all the tables or offices names represented. And from there we’re able just to see, we’re able to quantify, okay, it looks like one office a lot of students went to, and then another office a lot of students didn’t have the chance to frequent, so what are some things that we can do to make sure that we’re delivering those resources in a different manner, or what are some offices that our students really love to engage in?
Courtney Durbin:
And I think with that also utilizing those passports, we’re also able to connect some of those dots for our campus partners because I think sometimes when we bring them into our residential spaces, there might be some awkwardness or hesitation about how do I interact with students in the same place that they’re living? And so we’ve also been able to identify, we had campus police at one of our events, and students weren’t really engaging with them. And then what does that say about the bigger picture? Obviously, even when we’re thinking about police brutality. And so I think it’s important that we’re able to leverage information like that of there is an impact on our students, and they’re not wanting to engage, or they are very active in engaging with you, and what does that say about who’s at the table and what information you have this year?
And then additionally, in terms of resources and assessment, it’s also important that we’re leveraging assessment and tools that we already have, and so if your institution either uses the Beginning College Survey of Student Engagement, which is the BCSSE, or the National Survey of Student Engagement, which is the NSSE, all tongue twisters, those are two really good resources to be able to leverage to prepare for the residential population. And for us at the University of South Florida, our Office of Academic Advocacy holds the responses to to both of those surveys. And before the academic year starts, we will probe through what questions were asked on the BCSSE, and then we will pick out maybe four to five questions that we want to know how our residential students responded to them. And so this last pass academic year, we focused around how difficult our students expected college to be, how they were paying for college, what value they saw that USF was going to bring to them as a result of attending the institution, and then expectations about what their campus engagement would be like prior to even arriving on campus.
And so by looking at all of this information, we’re able to better plan and adapt our curriculum, that first community gathering, whenever our students are getting here on campus, what can our week of welcome and our first 50 days of programming and partnerships with other campus partners look like? So, we’re able to leverage some of that information to inform our work. And a lot of institutions are already collecting this data, but sometimes the residential education department might not have that information because it lives in another office typically, and so it might live in an office like that or institutional research just depending on who holds that information. So I’d definitely say if you have the opportunity to connect with folks about that information, it can definitely be useful to inform our work with our residential population.
And then, the other big kind of data that I would advocate for is utilizing Skyfactor. A lot of institutions are using Skyfactor to really understand effectiveness, and what are we doing well, and how are students experiencing different components of their learning as well as their satisfaction. And so residential education will look at that, and we’ve been able to identify because Skyfactor really does the work for us. It’s just us reviewing the information that we get from Skyfactor we’re able to easily pinpoint that our students really have a difficult time with their personal interactions, especially when it comes to conflict management, which who would have guessed that? And so, how does knowing that and also knowing that our students don’t necessarily feel like they’re learning how to manage their money as well as their time after living on campus with us?
So, how can we continue to think about those things whenever we’re building our residential curriculum? And so, we were able to leverage some of that information from our Skyfactor report and adapt our community gathering. Instead of in the middle of our semester talking about cultural appropriation, we are going to now focus on the roommate games and how we can get to know our roommates a little bit better. How do you actually address conflict? What do our students want to talk about? And we know that a lot of them rely on social media, or Snapchat, and things like that, and maybe that’s not the best way to talk about conflict and have a hard conversation. So continuing to think about what our students are doing and how they’re responding in surveys like that, I think are really, really important because those are surveys that already exist, and you don’t have to recreate the wheel even though we like to do that sometimes.
And then I would finally say, use some information that you already have. So, like I mentioned earlier, we utilize rosters, and we’ll pull a lot of data that is from those rosters, especially because we’re able to connect to Banner to get some of that academic information. But also, students have to apply to live on campus, and they have to apply to live in a living-learning community, and so what questions are we asking on those applications that can continue to inform our work, but then also maybe revisiting those applications because we can ask different questions to help us better prepare for those students and create a positive and meaningful experience for them, so I think those are all really good ways to continue to use information that we already have without having to start completely from scratch.
Aaliyah Deggs:
Paige, did you want to add something? All right, perfect. Thank you all. You all provided so much, so many resources and different perspectives and tools, so thank you for that. Now, we’re going to transition to look, what are some challenges that you should foresee when assessing a residential curricular model?
Paige Hicks:
I’m going to go ahead and take a stab at that one first. When I think of challenges, I definitely think about time, relevance, and resources. And so I’ll first start off with time, with thinking about how are you dedicating time to not only do assessment because it shouldn’t be seen as something like, “Oh, this is an add-on to my work.” It should be something that’s essential, and it’s something that’s done, but what are you doing after that? Because it is very easy to collect the assessment and then let it sit and stockpile and never use it. And so making sure that you’re creating time as an individual, even as a department, to really sit down with that data and learn, what am I going to do with this? And then to even share out your story with that.
So now that we’ve collected it, we’ve made meaning of it, who am I sharing this with, and who wants to know this story? And I think it’s important too to even loop back to the folks that helped you to collect that assessment, from you participating, this is how we are adding or changing our practice or our approach around a certain area.
When I think of resources, I think a lot of folks forget that some institutions allow folks to utilize their Office of Institutional Research, and so before thinking about asking a question or getting some of the feedback you’re wanting, see if that information already exists somewhere else. Are you able to capture that in another place so you’re not asking a student a question where you think it’s the first time, but it may be the seventh time that they’re receiving that because survey fatigue is real, especially with us all being at home, and/or practicing physical distancing? We want to make sure that we are really honoring the time of the students or the folks we’re asking to complete this assessment for us.
And I think the last piece is really relevancy, and so, one of our assistant directors in the department likes to say often that it is unethical to collect information or data that you’re not going to use. So, before asking that question, how does this map back when we think about that activity? How does this tie to something that I want to learn that’s going to positively change or impact the student experience or learning?
Courtney Durbin:
I think Paige was pretty spot on, and I would agree with all those sentiments. I think the other hard part about assessment and a challenge that we often see within residential education is just getting started, period. I know that that seems really simple and foundational, but like I mentioned earlier, assessment is often seen as this really big tasky thing. And it can be if you let it, but if you break it down to the simple parts, you’re able to start and take a baby step and then work your way up to get a completed project. And so that second resource that I actually sent out in the chat is how we break it down here at the University of South Florida, and I’ve been really grateful. I had a phenomenal grad school class about assessment where we really broke down the foundations and I’ve carried that throughout my time as a professional, and so a lot of it is just thinking from the ground up and starting with the end in mind.
So you need to name the program, the problem, the idea, whatever you’re wanting to start with, and then working even bigger. And so making sure you’re identifying the purpose and what those learning outcomes are, like why are you wanting to do this assessment in the first place? And then, thinking through what assessment methods are going to work. I think a lot of times it’s easy for us to say, “Oh, I want to do a survey because that’s all I know how to do,” but we can often think about other forms of assessment, which Dr. Kennedy is going to talk about in a little bit of using a focus group, or having an interview. Doing different types of assessment can also allow us to get really rich data, and sometimes, I think it’s easy for us to swerve towards that survey.
Then, finally, just doing the end part and closing the loop in what are you going to do with the data? I think we often get overwhelmed with because now we have all this, and it’s sitting in the closet, and how can we make sure that we’re sharing it with folks? And so I think it depends on the audience, and it’s really important to make sure that you’re thinking about who you need to share it with. So if it’s something that is internal, is it a dashboard, or are you creating an executive report or an executive summary for your leadership team to be able to propose something that you’re wanting to put forward? If you’re sharing it out with people who aren’t familiar with residential education language, are you creating an infographic or something that is really easy and visual so that way they can still see the data, but it looks different and it’s more digestible for them? So hopefully, Julie, that answers your question too, and I tried to tie that in there as well.
Kirsten Kennedy:
Julie, I can address your question as well. One of the things at the University of South Carolina we have seven faculty-led living-learning communities. They actually get release time from the provost to work half of their time in the residential learning community. And so, one of the pieces of data we were able to take was… One of the things we wanted to know is how soon does a student feel like they’re a part of the university community, and it’s reflective for sure. We asked them several points, but one of the things we learned was that people in one of those faculty-led living-learning communities feel a part of the university community much sooner than students who aren’t in them.
So, when you give a faculty member release time, what that really means is you have to hire somebody else to teach their class, and that costs the department money, and so, in order to be able to show that there was value to that release, that giving that release time when you tie back when do you feel a part of the community, to belongingness, to retention, and then you look at retention data that says the same thing, that’s a really powerful statement. So, not necessarily about student learning but about outcomes from programs that we have, especially when it goes back to funding. And you hear me say that a lot, but honest to goodness, that right now, when people are given back room refunds and those kinds of things, continuing a program that some might think is nice but not necessary, that’s important stuff right now.
I’ll just keep going because I think one of the things that Courtney alluded to is that we’re really focusing on learning outcomes now. You go to slide eight in what was sent out. That just gives a summary of the different kinds. How can I get my cursor down here? The different types of assessment because learning outcomes are one kind of assessment, but there are other types of assessment that can help you build a case if you need to on program effectiveness. So program effectiveness is one, and we talked about that earlier, but this… I don’t know if you… Karen Inkelas’s book Living-Learning Communities That Work, have the best practices model, and let me see if I can find the pyramid. Here, so there are all these different components and there’s an evaluation sheet in there that lets you decide whether or not you have these elements, and the little black lines in there, the mortar between the bricks, that’s assessment.
And so, you need to know if you have the right elements. It’s like CAS, if you look at CAS and it tells you you need all these things and you only have two of those things, and you’re like, “Hmm, we’re a little short in order to be able to make this work right.” So, usage data is helpful. Who needs data, like I talked about earlier when we did those one-on-one meetings, and we found out what students needed, is helpful. You do environmental assessment if you think about campus climate kinds of things and what do marginalized people feel about being on your campus. That’s important in terms of what you’re going to provide for students. Their comparative assessments. How does this learning community compare to that learning community? How does this building, to that building? Men to women? You can go on and on.
And then, the one that I wrote my very first book chapter on is cost assessment because it’s important to know in terms of your budget if you’re spending it effectively. Is there waste? Are there things that popped up that you didn’t know about? So all of those types of assessment are important, and in fact, if you took usage data together with learning data, you could really talk about the great impact that you have. I’m not saying that four people show up to your program, that’s horrible, but if you have a hundred people show up to your program and they learned a lot, the span of your impact is much greater, so taking those in combination can also paint a more accurate and richer description of what’s happening in your programs. I’ll leave that there.
Aaliyah Deggs:
Perfect. Thank you all. Our last question is going to tie everything back into each other. I know Dr. Kennedy talked about cost-effectiveness and the benefit and different things like that, and so with all of the things happening in the world and trying to get assessment, how do you prioritize assessment of a residential curriculum and a programming model?
Kirsten Kennedy:
If you guys aren’t jumping in, I am because I’m pretty passionate about this. You can’t afford not to do it otherwise you’re wasting your time. If you don’t know that what you’re doing is effective and let’s say it’s not, and you don’t know it, you are now wasting your time, effort, and energy on things that don’t matter. And every job has stuff you don’t like. I don’t like filling out paperwork, but you know what, I have to do it. And I swear around here, I’ve got to sign everything four to five times, right? Not a part of my job I like. Apparently it has a purpose, I haven’t exactly figured out why the duplication is important, but I’m sure that it has a reason. You just can’t afford not to, and at this point in student affairs and where we are in the evolution of assessment, if you don’t have that skill, you kind of become unhireable.
You have to be able to talk about a way that you have assessed a program because John Schuh who writes a lot on this, said he doesn’t spend his time anymore convincing people that you need to do assessment. He talks to them about how you do it. What’s the best way to do it? And so it just has to be a part of how you do your job. You wouldn’t spend money without checking on your bank account, so you want to make sure that you’re spending time wisely and also that you’re not hurting anybody in the process either. Are you doing something that actually is harmful to students, especially in terms of their psyche? So I would say you cannot afford to not do it. And when you talk about, what are some of the setbacks? What are some of the difficulties? Is having a supervisor who doesn’t value it, but you need to be able to influence your own sphere. And so if your boss doesn’t value it, but you do, then do your own assessment projects in your building just so you get some hands-on experience.
When I was at Bloomsburg University, my department didn’t care anything about assessment, and I decided I was going to do a GPA Living-learning Community Assessment with a control group. And I worked on that for a whole year all by myself, going to the library, looking stuff up, calling people up, asking for help, how to do this, and it was one of the best things I did. I learned a lot through it. And also, what I learned is my next institution, I’m not going to a place where they don’t value this.
I’ll get off my soapbox.
Courtney Durbin:
I think as all of us are assessment nerds on here that we’re all pretty passionate about making sure that assessment is prioritized. I think it is also the way that we are able to prioritize it is by figuring out what are our objectives, and what matters to us as a department right now, but also what is going to matter in a year from now, as well as five years from now. And so it’s really easy for us to get caught up in the, we have to focus on the present because, especially with the nature of our work, that’s how it always is we’re just nose down to the ground, making sure that we’re getting whatever we’re working on right now done, but we need to be thinking forward as well, especially, as we think about budget allocations and things of that nature. How do we make sure that we advocate for another faculty and residents that that’s something that we see is really impactful?
And so I think making sure that we’re thinking through the present, the short term, and the long term is a way that we can make sure we prioritize it because we’re not waiting until the last minute to throw together an assessment. By making sure that we’re thinking of a future plan, you’re able to think of more thoughtful questions. You are able to better articulate what you’re trying to understand, and ultimately, you’re more prepared to be able to create that assessment, disseminate it, and really see the project from its start to finish. And so, think about the big term, think about what you want to get out of this in the long run of things, and especially as entry-level professionals, it might be difficult for us to be able to see that, especially when we’re in an entry-level coordinator job. We might be at an institution from two to three years, but even thinking about your community still going to be standing, and there’s a legacy that you’ll leave behind.
And so what do you want that legacy to be? Is it one that you were able to create a tradition in your community that was centered around academic excellence because that’s the need that is with your community. And so thinking about it from that stance… Words are hard today. From that standpoint I think is also really important. And then, again, just making sure that when you’re prioritizing assessment, that you’re also making sure you’re prioritizing the closing the loop aspect of it, and really making sure that you think through, what matters, why does it matter, and now what am I going to do with this information and who do you want to share it with? So that’s how we guide our work and how we make sure we prioritize it here at the University of South Florida and making sure that we’re sharing our information not only amongst our assessment committee but leveraging our weekly departmental meetings, talking with our housing and residential education meetings, and then sharing it with other campus stakeholders that live in the academic side of the house as well.
Paige Hicks:
Courtney, that was great. I would say all that times a hundred. And if I was to add anything, I think it’s important that you’re really able to practice aspects of backwards design thinking, right? Starting with the end in mind. I think it’s going to really help you fortify how to prioritize and what is needed right now, but can maybe wait until later, and if this is even a priority, and so when you start to include some of that into your thinking into what is needed is going to be a little bit more helpful.
If anything else, I would say set deadlines. I think, again, it’s really easy to push off assessment work and think that it is not needed in the moment, but it is, and also get others involved, and so I think that it’s everyone’s responsibility to have a stake in assessment and sharing our story. I do think it is also helpful to have someone there to say, “Hey, I need this. Where are we at with this?” And so someone is able to really weave together and theme everything that is coming in from the department and share it out with others, so that’s my answer.
Aaliyah Deggs:
Thank you, Paige. You did a great job of incorporating an answer to a question that we have in our chat, who do you believe should be the architects and gatekeepers of managing all things assessment? Does anyone have anything else to add to that question?
Kirsten Kennedy:
Yeah, I do. One of the things that happens when you get good data is other people want access to it, and you need to set up some really clear policies internally about who you’ll share that data with, especially if it contains some kind of sensitive information, which here are one-on-one conversations because they’re documented to be able to theme in terms of a qualitative way. And we check on that if we have a missing student, we’re like, “Oh, no, we better go back and check that data and see were there any warning signs that we missed or any clues about where that student might be.” But what if athletics calls and says, “I want to know what’s going on with my football players?” Are you going to give that data up? And so, it’s important to set those guidelines up ahead of time, who can have access to that data, and for what reasons? And I think that speaks to the gatekeeper piece.
And around here, if you complete a survey and they say it’s anonymous, there’s always a way to map that back to the email address and eventually find out who that is. And I don’t like… Well, now that I just told you, all that’s possible, but I don’t like that to be common knowledge because I don’t want students to know that that is possible, but we’ve set up policies about we won’t do that. We’re not going to map anonymously back to an email address, even though we can ethically it’s not the right thing to do.
Aaliyah Deggs:
Thank you. We have another… Oh, so go ahead, Courtney.
Courtney Durbin:
Oh, I was just going to add in terms of architects. I think it’s also important that there is a group of people, and it’s not just one person holding the assessment. I think it is important, and obviously, it depends from institution and people power and all those different things, but I think if you’re able to form a committee, a task force of two to three people even, three is ideal. But if two is all you got, then you’ll make it work with two. But making sure that assessment is a part of the larger conversation and having a committee or a task force that really spends a weekly, a bi-weekly meeting time to have those conversations around assessment and to work on those projects because if you don’t create that space for it in your everyday work or on a weekly basis, then it’s never going to be a priority, and it’s not going to happen.
Aaliyah Deggs:
Thank you. Oh, we have another question in our chat about what are some resources that you all would recommend as far as assessment and different things like that?
Kirsten Kennedy:
All right, well, already put it out there. I have a book, so if you like what I said, I guess, buy the book. It’s 35 bucks from ACUHO-I. If you don’t like what I said, there’s lots of other really good resources too. It’s just that mine focuses particularly on residence-life assessment, and it’s got things in there about data ethics and building an assessment culture, methods, which we haven’t touched at all upon that, but assessment is not research-light. Assessment is for a different purpose. Research is to try to extend the knowledge in the field and the knowledge base, and assessment is try to improve practice and inform policy. And so anyway, what happens is, especially if you’re interacting with faculty is that they don’t like the results, they’re going to attack your methods, so you better be solid on it.
And there’s a chapter in here that Sherry Woosley from Skyfactor wrote along with Matt, I never say his last name. It starts with a B and has lots of A’s in it, Banas in it, I don’t know. Anyway, and they wrote a wonderful chapter on that and how do you select the right method depending on what your question is, and we didn’t get to it yet, but you also thinking about who’s the audience going to be for this if you’re writing a larger report and how do they best receive information? Some people only want to know the numbers and think that qualitative isn’t really research and the words that you gather aren’t data, and other people really respond to student experiences and the stories that qualitative data can tell, and so I recommend that.
There’s also anything by John Schuh, Peggy Maki, and full disclosure, my dad and John Schuh were longtime colleagues, and so the first book that was written about this, Assessment Practice in Student Affairs, my dad wrote, so if you want to know where some of my passion comes about that, it’s generational and hereditary apparently. So really anything that John Schuh wrote, I would go after that, and also… Shoot, I can’t think of her name. She’s at IUPUI. I’ll come up with it. Give me a little bit of time.
Courtney Durbin:
I was going to echo the John Schuh book. This book sits in my office and whenever I’m frequently referring to different assessment strategies or just little tidbits that I might be thinking about, it really allows me… It breaks assessment down, so if you need to try and understand assessment from a foundational level, I would definitely say check out that book, and I put it in the chat, so if you need to refer to it later, feel free. But it gives a lot of really great examples of how you can practically use assessment, what other institutions have done, and then you’re able to navigate it and use it at your own institution with whatever assessment you’re doing there.
And then the other great resource that I think, I know I plug Skyfactor a lot. That’s been my big project since I’ve been here, but I have been able to attend some of their sessions when I attended a ACUHO-I last summer, and they also have some webinars on their Skyfactor website, and so I would definitely recommend if you’re ever able to connect with them. They also break down assessment in simple terms from a practitioner lens, so I would definitely reach out for that as well.
Kirsten Kennedy:
ACUHO-I has a certificate program on assessment. It’s a little pricey, but that’s a good one as well.
Paige Hicks:
I’ll also quickly share that there are also a lot of free resources online as well, and so feel free to Google some of those. I recently just got done taking a curricular assessment, like how do you develop, design, and implement curriculum assessment, and that was a free four to five-week session put on by an individual at the University of Colorado Boulder A&M, and I learned a lot. Also, websites like student affairs, assessment.org, I found some great items off of there as well. So getting connected to colleagues, and also other free resources can be helpful and beneficial as well.
Aaliyah Deggs:
Thank you all for providing those resources for us and giving us a lot of knowledge today. Do you all have any final words before we conclude our webinar today?
Courtney Durbin:
I know someone asked if they could have emails of the panelists, so if you want to reach out or need anything, want to talk about assessment and nerd it up, feel free to reach out, and we’ll drop our emails below. The webinar is also recorded and will be on the website. Thanks, Aaliyah, you rock.
Aaliyah Deggs:
I got you.
Kirsten Kennedy:
Trudy Banta, that’s the other one, I knew it would come to me.
Paige Hicks:
That’s a great one.
Kirsten Kennedy:
I guess my parting words would be don’t be afraid of it. Start out small, do the things you’re comfortable with. For a lot of student affairs people, it’s talking with students, it’s more than talking with students, then it’s making sure that you record it in some way, either in notes afterwards, but then being able to produce some results from that, so it’s not just talking with a few students and say, “I conducted an assessment and here’s what I found.” It goes back to that methods-thing, and then you can grow into the things that you’re not as familiar with. I was on a panel for ACPA many years ago, and one of the statements I made then, I think still stands, which is, “I’ve never reached out to a student affairs professional and asked for help and been told no.” We have a very kind and caring profession, and hopefully that’s extended to our colleagues as well as to students, so if you need something, reach out and ask someone. I think it’s the best. Just start, and the rest of it will come.
