Evaluator Competencies Reflection

The meaning of evaluation to me has been grounded in my professional and personal experiences. I took job titles like senior quality analyst, credit analyst, quality specialist, and math teacher all having to do with data backing up decision making. Also, when my mom got sick and it took two years to get her a diagnosis, doctors did many evaluations. The commonalities among those experiences brought me to an understanding that it takes time, data, awareness and a lot of research to come to a conclusion that makes sense in the decision making process. On the scale from 1 to 6, I would not rate myself as an expert, it would currently be a solid 4. I know that I have to hone my experiences, new knowledge, and evaluation skills to get to an expert level. 

My experience with data came into play as I drew conclusions on my self-assessment. I took the average of each domain, at the thousandths place value and put them in order making the highest my strength and lowest my weakness. In the competencies within the domain, I chose anything below four on the scale to focus on for professional development. Also, somewhere down the line, I recalled looking at survey results and knowing that the top two highest on the scale was the only focus for positive but the others were negative. I am sure I will learn more about evaluation instruments in the course. At first it was just data values I reviewed, then I got right into the context to make sense of it all. Here are my results based on my own observations:

1. Interpersonal

2. Planning and management

3. Methodology

4. Professional practice

5. Context

I was not surprised that my interpersonal skills along with planning and management were strengths because I have been intentionally working on that during the Learning Design and Technology program at Arizona State University. However,  even as those two were the highest, I looked at the competencies that I scored below five. Interpersonal competencies were attending to power and privilege and facilitating culturally responsive interactions (American Evaluation Association, 2018). I know that I have to check my bias and political viewpoints and remain objective stating facts but also have to research my audience so I know how I can approach the evaluation process. In planning and management, addressing aspects of culture, planning for use and influence, coordinating and supervising, and documenting (2018) were the competencies that I needed to work on. My area of growth in the planning and management domain is to be aware of culture and the influence my report will have on stakeholders and I actually worked on delegating tasks in a team lead situation to help me focus on higher priority tasks.

I was not surprised about the last three: methodology, professional practice and context. The methodology, I need to work on making sense of research studies and translating the findings, however, it will take practice and repetition to get to understand it. Professional practice is understandably low because evaluation as a profession is not something that I really know about, I have used different competencies in previous lines of work when I was involved in decision making. Finally, the context domain where the competency that stuck out the most was clarifying diverse perspectives, stakeholder interests, and cultural assumptions (2018). In this domain, it will take courage to back up the data and options for the stakeholders. The skill of taking the evaluator language and process and translating it to a point where stakeholders will understand is a challenge. Also, knowing the possible outcomes of each option could affect a lot of people in a negative or positive way. 

I remember a situation where my humanity would get too involved but knew that the facts backed up my recommendations. During the mortgage crisis I was a senior credit analyst in small business loans and licensed to lend up to a million dollars. I really enjoyed the passion and grit small business owners possessed. A file came to me where I had to close the whole banking relationship, not just small business but any personal accounts they were named on. This client was with the bank for a very long time and very profitable through the years, with those characteristics it was not going to be an automatic close of the account by our computer system. I reviewed all the data we used like credit reports, financial documents, notes from interactions, calls to the personal banker, researched current trends and so on. I came to the conclusion that it would be too high risk for the bank. I dreaded that call, my humanity knew that this was pure circumstance but my job was to protect bank assets. I called the owner and tried to see if there was anything in the conversation that would give a lifeline, so I can keep the accounts open but the responses were red flags we were trained to spot. The conversation remained professional and the client even had plans for reapplying when the situation changed. Before I got off the phone, the client asked me if I could keep the account open long enough to get food and gas for the family but due to the risk I had to immediately close the account. Although, I may have taken my time. This story sticks with me and one of the reasons I left banking. In banking, context was painting a picture while using all the resources and data we had while keeping our main goals in mind. I had to check my bias in a lot of circumstances but also balance the influence I had in decision making, while being culturally aware of my audience on both sides.

The American Evaluation Association did a good job of creating competencies for the evaluation profession.  If I were to put myself in the shoes of a stakeholder, these competencies are what I would look for in picking the evaluator and entrusting them to make sound recommendations.

I found in self-assessment one has to be honest with themselves in order to actually pinpoint further development and create a plan to be better. Beyond the class for evaluation of learning systems, I would get an informational interview with top evaluators and ask them more about what their likes and dislikes are about the profession, how the day to day looks like, and what they think it will look like in the future. The qualitative data is just as important as the quantitative to make sense of it all.

References

American Evaluation Association. (2018). The 2018 AEA evaluator competencies.