10 points show how the latest RTA plan misses a big opportunity to signal that equity is a priority for NOLA public transit.Read More
No worries and no dummies here!
Remember those "For Dummies" books series? If you ever flip through one, you’ll see that the title is misleading. The books aren’t actually for dummies, they’re for people who don’t know much about their topic of choice and want a primer.
As someone who is new to monitoring and evaluation (brand spanking new! two months new!), I often wonder how other people felt when they first entered the field of evaluation. With so many new acronyms, terms and tools that I don't know how to use, how am I ever going to catch up with my colleagues? It takes a succession of baby steps, and I wrote a list of 5 tips to share for baby evaluators.
1. Learn how to organize. Where would I be without Google Drive, Asana and my own network of personal notes? Having systems for organization is an important step in the ladder to keeping a clear mind so that you can focus on your tasks! Everyone is different of course, but having a reliable system that you and your colleagues can easily navigate will save oodles of time, confusion and frustration in the long run.
2. Take on new responsibilities. Especially the topics that you're unfamiliar with or that scare you the most. This is how you gain confidence. I recently agreed to be a part of a local team for a new project, and I couldn't be more grateful for the opportunity to develop in this way. There is no easy way, just grab the opportunity when you see it.
3. Learn how to simply analyze quantitative data.
If you're qualitatively-oriented like me, don’t fret at “quantitative.” It doesn't have to be boring or difficult. Flux uses pivot tables a lot. It's an Excel tool that allows you to analyze lots of data quickly and easily. Most people have some familiarity with Excel, so the learning curve for this tool is much less steep than with other software programs, and you likely already have access to Microsoft Office. Practice makes perfect.
4. Get involved! I joined NOLA Eval (our local evaluator's collaborative) within my first few weeks at Flux and I'll also be heading to the American Evaluation Association's annual conference in Washington, DC this year, so I can absorb all the experience, knowledge and perspectives I can get. Communing with like-minded others is a great way to get to know people and develop leadership skills.
5. Read. For the first month, I did a lot of reading on the field of evaluation. Just reading up on evaluation will help you to understand the conceptual background of the field. Knowing history, well-known cases, current topics and key people counts for a lot. Save yourself the awkwardness of pretending to know when someone mentions a well-known evaluator.
Those are all the tips I have for you baby birdies today, so feel free to fly away from the nest, but don't forget to stay tuned for more beginner tips!
Understand what the target audience considers relevant and where this overlaps with that which we desire to communicate.Read More
Gulping an early-morning coffee and listening to the clang of aluminum ladders as my neighbor loaded his truck for the day, an old friend was surprising me with a story about a 37.4 MILLION DOLLAR renovation going on in my backyard. The project is called the Bell Artspace Campus, and it’s converting the Bell School into 79 residence units for “low- to moderate-income artists, cultural workers and their families.
But we weren’t talking scale of the project that morning. We were worrying about why it was so HARD to get artists actually from the Tremé to actually apply for the spaces. To me, the issue circles around some keywords often thrown around in proposals and board rooms, but devilishly difficult to implement:
Yup. It’s a big nut to crack. But upon thinking for a minute, I decided that or me, engaging neighborhood residents in a sustainable way boils down to one thing:
If you can get local residents to feel that a neighborhood development project is 'theirs,' that's a HUGE win, and it's something that I haven't seen much of in New Orleans.
I'll always remember a presentation by a Maori group from New Zealand at a conference. As they said - traditional social interventions (a socially-oriented building renovation being one type of intervention) are done 'on' or 'for' people. Then, Robert Chambers (and many others) popularized the idea of participatory approaches, which attempt to do the evaluation 'with' people. The Maori approach, the so-called developmental approach, is about doing evaluation 'as' the program.
Working as Maori. Working as Tremé.
In New Orleans, I think people simultaneously revel in the cultural diversity of the city and sweep it under the rug. Local musicians are wonderful, but including them as key decisionmakers in a room of funders, builders, and NGO leaders, when many may not have any formal knowledge of neighborhood development, is to watch a clash of cultures. And when that happens, usually, the old social structures are reproduced.
Culture clashes often reproduce old social structures.
Those in traditional leadership roles leave frustrated saying 'well at least we tried,' while those typically marginalized leave frustrated saying 'another example of how they never listen to us.'
So here’s my off-the-cuff list of suggestions for doing social intervention AS New Orleans:
1. Take your time:
Start making contacts early, like years in advance. Sometimes, you will literally have to mentor and capacity build a new core of leaders, because they simply don't exist when you start. Nobody around has the right mix of social standing and leadership skill, so they have to be discovered, encouraged, mentored. Look for the liaisons, the translators, that can shift between worlds with diplomacy. Think about living deep in Tremé and only ever hanging out with your close neighbors - would feel like a different country than if you did the same thing in uptown, no?
2. Don’t Expect Pure Progress:
This could mean physically, in terms of the actual building, but really I mean socially. Don't expect 'the community' will be on board at first. Poverty is an embarrassing, shameful emotional construct to be kept secret (see the ever-famous Voices of the Poor report by the World Bank) as much as an economic status. You will need time to really hear a full spread of opinions articulated, and that means that it will take time for the team of liaisons to churn, find those more hidden pockets of dissent (which turn into your 'last mile' problems). Expect to take almost as many steps back and to the side as forwards.
3. Expert Facilitation:
In-person meetings are key to gaining trust and the facilitator of these experiences is that absolute last place to skimp or cut corners. For something on the scale of the Bell School Artspace, you may need several to relate between more than two cultures. Nobody should expect an Atlanta-raised funder representative, a Boston-raised NGO leader, a Plaquimines-raised builder, and a Tremé-raised musician to be on the same page.
4. Evaluation & Feedback:
Finally - I had to say it - it's key to show everyone that their knowledge and opinions are contributing to substantive changes. This means setting up a system that combines the capacity of the stakeholders with data from further afield to produce inputs for decisionmaking -- exactly what evaluators are trained to do.
I believe seeing these four features baked into New Orleans social interventions, whether non-profit projects, government policies, or for-profit social enterprises, is key to a plural future for New Orleans. A New Orleans built as New Orleanians, not ‘for’ them.
How many of you have been personally victimized by your monthly energy bill?! (Entire gym raises hand...even Regina George.)
That is because IT IS SO HOT IN NEW ORLEANS!
How much does that bill run you in the summer? 1 million? 5 million? 12 billion?! That is what it feels like when I check my statement every month, anyway.
We here at Flux definitely feel this heat, and one of our recent projects with Energy Wise Alliance (http://www.energyla.org/) has shown us that most New Orleanians do too! This project is called the Context Blueprint, and it is one of our most exciting products available at Flux!
The Context Blueprint is a conglomeration of GIS mapping and demographic data from large public data sources.
We focus on the main areas of education, energy & environment, transportation, and socio-economics. You choose from these themes, tell us some key things you are interested in finding out, and we search through our diverse datasets to give you the information you need... complete with awesome maps and visuals!
This is great for internal use, reporting cycles, grant proposals, and meetings with potential funders!
For Energy Wise, we took the information and questions they had for us, and created a report complete with GIS mapping that showed them information on energy bills, school locations, and income levels. Energy Wise wanted insight on where to expand their programmatic efforts, and the Context Blueprint gave them both the facts and the maps to visualize the energy use landscape, the populations they were targeting, and areas they could potentially expand towards!
For example, we created maps that showed things such as the percentage energy bills take up of family income per geographical area.
Now while we all feel the heat of these massive energy bills, Energy Wise is here to save the day! (By providing energy saving hacks and tutorials for all New Orleanians -- with a focus on underserved populations.)
Here are some tips to beat the heat and cut down on those energy bills (in summer and winter)!
Use draft blockers to keep temperatures comfy
Create a breeze -- use ceiling fans! Also, fans help evaporate sweat from skin..among other things
Plant trees outside to keep cool inside
Close all blinds during the summer to keep things cool
Open blinds in the winter to heat things up
Adjust the Temperature: When you are home and awake, set your thermostat as low as is comfortable. When you are asleep or out of the house, turn your thermostat back 10° to 15° for eight hours and save around 10 percent a year on your heating and cooling bills. A programmable thermostat can make it easy to set back your temperature.
Find And Seal Leaks: Seal the air leaks around utility cut throughs for pipes ("plumbing penetrations"), gaps around chimneys and recessed lights in insulated ceilings, and unfinished spaces behind cupboards and closets. Add caulk or weather stripping to seal air leaks around leaky doors and windows.
Lower Your Water Heating Costs: Water-heating accounts for about 18 percent of the energy consumed within a person’s home. Turn down the temperature of your water heater to the warm setting (120°F). You'll not only save energy, you'll avoid scalding your hands.
Please enjoy this interlude to our ongoing project series (we will pick back up next week talking about Energy Wise Alliance) for this special bulletin about our NOLA Evaluation Community of Practice!
Evaluation in NOLA
“Ohhhh snap.” Said my sandal strap and my internal dialogue in unison. It was midway through the night and I was high in the hills of Rio de Janeiro, quite a (literal) hike away from my bed. We were at a pre Carnaval dance party in Tabajaras, a favela right above Copacabana. I apparently was Samba-ing too hard and blew a shoe way too early. I was sitting there, looking at my shoe, trying to figure out how to piece it back together so I could get back on the dance floor and eventually back to my apartment. While I was giving up and intending on moving forward barefoot, a man from the community came up to me and gave me the sandals he was wearing so I could make it home. He gave me the ACTUAL SHOES OFF HIS FEET! It was the nicest thing anyone has ever done for me, and I wasn’t even a permanent part of his community. My group in Rio was close with their community, both in proximity and friendships, and we did a lot of things with them, but to be so included in their community when I truly had done nothing to merit it except to show up, was a mind blowing experience for me. It really drove home the importance of our communities and the people we surround ourselves with, and now I feel even more lucky to live in a city like New Orleans, where community is so intrinsic!
Community is a pivotal part of the human experience.
It gives us comfort, validity, and comradery! It helps us battle disasters, hurricanes (I see you Irma), and adversity. It gives us resilience in the face of tragedies, loss, and injustice. Most importantly, it provides us with a larger family and helps us to feel like we have somewhere to belong!
Yesterday, we had our FIRST EVERRRRRRRR community event for evaluation in New Orleans! Dun dun duhhhhh..
Welcome NOLA EVAL!
We were in the company of many interesting individuals -- a few who are already influencing the greater good in the field as well as a few who were just interested in what we do as evaluators! We had local eval representatives from Flux, Learning Design+Strategy, and Louisiana Public Health Institute.
It was fantastic.
Normally, the evaluation landscape in New Orleans is kind of lonely…
People typically are not even aware of what the word “evaluation” means in a professional sense (see first post!), and this isn’t just a New Orleans thing… it’s a worldwide phenomenon. I guess that is the trouble of being a part of a new field! The national association (American Evaluation Association/AEA) does not really have a presence locally, with the closest chapters being in Houston, Atlanta, and the Florida Panhandle (talk about a commute!!) And even with the AEA there is only a conference once a year, so overall very little opportunity to come together with peers to swap best practices, stories, and ideas!
Being surrounded by peers in the profession and those who recognize how cool evaluation is was extremely refreshing! It was inspiring to see what fun and interesting things for the future of this community of practice our small group of people came up with in only an hour and a half.
In the first few minutes we hashed out some misunderstandings regarding the scientific rigor and viability of qualitative analysis and the beauty of mixed methods! We got to discuss the pros and cons of qualitative vs quantitative and how they can best work together, all while gaining ideas about how we could share our specific skills in these areas with the rest of the group!
Community building at its finest...taking differences, discussing them, and finding a way to build ourselves up as a result!
Next we want to start having internal case study exchanges and discussions to educate the group about projects or professional skills that we find particularly informational and interesting.
Overall, we are stoked to report that the New Orleans evaluation community is GROWING!!!
We are currently scheduling the next meeting and have tons of great ideas for our future. We plan on doing two meetings a month, one lunch time meeting and one evening meeting.
Contact us at Flux if you want to jump on board!
My email: email@example.com
NOLA EVAL Facebook Group: https://www.facebook.com/groups/nolaeval/
“I hope you can see how excited I am...”
I said beaming a way-too-wide smile way-too-early in the morning in an echo-y meeting room on Elysian Fields. “There are very few things that I go all-in on, but I really can’t wait to show you this.”
We had been working together for over two years by now, so I saw some sympathetic glimmers in the eyes of the people around the table, but mostly it was more placid “Ok...we’ll see.”
My audience was the core staff of KID smART, a local non-profit focused on arts integration in education. We were gathered for a workshop on, you guessed it (or maybe not), Spreadsheets for Data Analysis, and the flagship feature, the Pivot Table. I’ll write more about the beauty of Pivot Tables in another post, so for now, you’ll just have to trust me. I’ll just entice you with the fact that I later got a call from KIDsmART in which the words “We’re pivoting all around the office,” were spoken. Pivot Tables rock. A lot. Seriously.
Zooming out, this mini-example speaks to a key feature of evaluation, and a core tenet of Flux as a company:
Sustainability through mentorship.
Good evaluation means not only doing high-quality work, but also building the foundational skills of the people implementing a project.
Skill. Teaching to fish.
It’s a joy watching clients pick up the tools they need to do their work better. Tools to collect data and feedback, tools to analyze and systematize that information, and tools to present findings to their audiences (including themselves). It’s even more rewarding watching them wrap their heads around the concepts of evaluation. For example: the meaning of ‘impact,’ the breadth of sources of data, and how it flows through an organization (all to be treated in later posts).
Emotion. Evolving towards self-determination.
A lot of the associations with the word ‘evaluation’ have to do with reporting requirements to funders. Not fun. Sometimes insightful, but usually a huge burden.
With new skills and some orienting concepts, evaluation can become a much more exciting exercise, an internally-driven process. Nonprofits should not be beholden to produce the indicators their funders want to extract but instead respected as the organizations empowered to create the change in the real world that funders want to see.
Nonprofits should not be beholden to produce the indicators their funders want to extract but instead respected as the organizations empowered to create the change in the real world that funders want to see.
But this shift in perspective needs nonprofit staff who have the capacity to weave multiple threads of data into a convincing, communicable story of organizational impact.
A core of such skilled, motivated, and courageously self-critical professionals is exactly what we hope to [help] create.
KIDsmart has shown remarkable hunger to learn and admirable commitment to improvement, not perfection, in terms of telling their story using data (see their latest Milestones Report). It’s inspiring. Flux is proud to be part of their journey.
Evidence is political. Evaluation is an artRead More
“So what do YOU do?”
Welcome to the shiny new Flux blog!
My name is Katie Hillman and I want to open this blog with an awesome, troubling, odd, fascinating, and all-around doozie of a word:
Also, not coincidentally, it’s what we do here at Flux. Evaluate.
THROWING THE BOOK [I.E. DICTIONARY] AT YOU:
“to determine or fix the value of; or to determine the significance, worth, or condition of usually by careful appraisal and study.”
Uhhh... yeah...but what does it mean as a profession? That is a much more elusive question to answer. During my current stint in M&E (Monitoring and Evaluation), fielding the question of “what I do for a living” has been the most common, and difficult, thing to answer. In the definition above, “careful appraisal and study,” most closely rings true to our task. M&E encompasses many things, but focuses on choosing the appropriate methodologies and questions for each project.
In short, Flux’s mission is to MAKE DATA MEANINGFUL.We help social business, non-profit, and government leaders to integrate evidence and evidence-based thinking into decisions. The vision is that data, evidence, and reflection inspire learning, resilience, and dignity in places of need.
STILL NOT ENOUGH?
Never fear. That’s why it’s a blog and, hopefully eventually, a conversation. I am new to the field of evaluation and am also working to get a clear answer to the question of “what do I do?”. But every day I realize that it cannot easily be defined in a sentence and that is part of its beauty. Feel free to check out our first official attempt at defining ourselves on the Flux website.
Over the next few posts, we’ll take you through some example projects to start to give this broad-reaching idea some real, concrete shape.
WE’RE NOT MAKING THIS UP! WE SWEAR!
Finally, we’re not the only people out there doing this. Evaluation is a professional field with its own, complete with practitioners, professional bodies, rules, and a colorful history. Flux is a member of the American Evaluation Association and the European Evaluation Society, and there are a ton more around the world! The United Nations even recognized 2015 as the International Year of Evaluation.
Welcome to our blog and thanks for joining us on our journey of Monitoring and Evaluation in New Orleans!