_edited.jpg
Vibrant Hawai‘i Evaluation Methodology

 

FRAMEWORK

Participatory Action Research (PAR) is both a research design and method (Baum, MacDougall, & Smith, 2006). It requires all involved to share their experience of the problem and contribute their knowledge and expertise to change it for the better. This iterative cycle of reflection-action-reflection provides participants a holistic understanding of the situation in order to contribute individual and collective action toward desired outcomes.

 

The iterative reflection and use of data to inform actions embedded within Participatory Action Research closely aligns with the practices of Hikaloi, Nalu, and Kilo. These practices have sustained communities, established resilience, and endowed abundance in Hawaiʻi for centuries. Used together, they cultivate an ability to ʻauamo kuleana - engage in citizenship and relationship with the system/context to lead and contribute toward a different outcome. 

 

3 LEVELS OF REFLECTION

Hikaloi/Hikaloiloi - the consistent practice of self-reflection that supports personal growth and improvement towards excellence. Personal reflection allows us to make meaning of our observations and experiences. Vibrant Hawai‘i also employs the actions of hikaloi at the end of any project at the close of every quarter with a formal After Action Review or “Mo Bettah Next Time” process. This encourages open and honest discussion, focuses on outcomes and results, and sustains methods that work.

 

Nalu - the observation, reflection, and recognition of the desired learning outcome. It is further defined in this way: to ponder, meditate, reflect, and speculate.  The word nalu in the Hawaiian language also means wave. Just as a surfer intently observes waves in order to survive, knowledge or meaning-making must also include keen observations of the world around us. I ka nānā nō a ʻike. By observing, one learns. ʻŌlelo Noʻeau #1186 (Pukuʻi,1983).

 

Kilo - the acute, sustained observation and examination of a place, individuals, or systems, with a focus on gaining an understanding of the reciprocal relationships. Kilo encompasses all the repeated actions of nalu above and takes these data points one step further to string together a holistic view of the topic/information. The observations are shared and discussed to gain knowledge and insights that enable approaches to problem-solving, stewardship, and systemic change. 

 

COMMUNITY-DRIVEN DESIGN

In 2018, informed by the Aloha United Way ALICE (Asset Limited, Income Constrained, Employed) Report (United Way, 2018), a Small and Mighty (SAM) group of Hawaiʻi Island community change leaders came together with a common why, a belief that increasing wealth and wellbeing on Hawai‘i requires ʻauamo kuleana, or the active agency to create collective impact. 

 

For the first half of 2019, the SAM group practiced kilo by participating in networking groups focused on social determinants of health, building relationships with community leaders, and learning about the unique missions and capacities that each of their organizations held. Kilo provided an opportunity to understand what factors might inhibit or improve those priority objectives, and gain nuanced information about communities and organizations. 

 

In the summer of 2019, the SAM group held a series of 5 regional forums facilitated by the Vibrant Hawaiʻi Community Lead in Hilo, Kona, Waimea, Puna, and Kaʻū. The objective outcome was to understand where stakeholders positioned their desired futurity. The SAM group aimed to have equal representation of diverse community roles with participation from individuals whose kuleana is primarily based in business, philanthropy, government, education, social services, faith communities, and community-based organizations. 

 

BASELINE DATA

At each forum, participants broke into small groups and were asked to draw their vision of a vibrant Hawaiʻi and share the stories behind their drawing. Each idea was written on a half sheet of paper and forum participants grouped similar ideas together in a waterfall. Each waterfall was given a title and participants scored the current state of each waterfall on a 4-point scale (1= Terrible; 2= Bad; 3= Good; 4= Vibrant). These activities are grounded in the work of Dr. Barbara Holtmann (2011). 

 

An average score for each waterfall was calculated, and each waterfall from the four regions was analyzed and categorized by the Vibrant Hawaiʻi Leadership Council - a group of representatives from each forum. Through consensus, a final list of vibrant Hawaiʻi Indicators emerged. 

Indicator
Definition
Baseline
Living Aloha
Being in harmony with our ʻāina, community, and ourselves. He ʻike ʻana ia i ka pono. ʻŌlelo Noʻeau 620. One has seen the right thing to do and has done it. (Pukuʻi, 1983)
2.39
Belonging
The result of building relationships to place and people so that we can care for and be cared for by them. ʻIke aku, ʻike mai, kōkua aku, kōkua mai; pēlā iho la ka nohona ʻohana. ʻŌlelo Noʻeau 1200. (Family) life requires an exchange of mutual help and recognition. (Pukuʻi, 1983)
2.36
Get Chance
The establishment of equitable systems that promote choice and prosperity. Kau i Kāpua ka poʻe polohuku ʻole. ʻŌlelo Noʻeau 1608. Without resources, one gets nowhere. (Pukuʻi, 1983)
1.67
Get Choice
The determination within a person to take ownership of their future and contribute to collective abundance and wellbeing. "Ua hele aku au me ka manaʻo paʻa. Ua paio aku au me ka manaʻo koa. Ua lanakila au me ka manaʻo pono. Ua mākaukau au me ka manaʻo wiwoʻole." Iosepa Nāwahīokalaniʻōpuʻu (Sheldon,1908/1999)
1.56
Resilience
The ability of individuals to adapt to change and adversity without being uprooted from their source(s) of wellbeing. E Lēkia e, ʻonia i paʻa. ʻŌlelo Noʻeau 334. Make a move to give yourself a secure hold. (Pukuʻi, 1983)
2.63
Strong ʻOhana
Strong ʻOhana: As a result of aloha, belonging, having chance, choice, and developing resilience, ʻohana are able to serve as a pillar of support. Hilinaʻi Puna, kālele ia Kaʻū. ʻŌlelo Noʻeau 994. Said of one who leans or depends on another. (Pukuʻi, 1983)
2.13
Thriving Community
The result of creating opportunities that encourage individual and ʻohana contribution towards community-driven solutions. Mōhala i ka wai ka maka o ka pua. ʻŌlelo Noʻeau 1232. Flowers thrive where there is water, as thriving people are found where living conditions are good. (Pukuʻi, 1983)
2.17

In addition to the seven Vibrant Hawaiʻi Indicators, community forum participants identified Waypoints - conditions or results that inform the strategies and actions of each of our Streams. These are included in Table 2 along with the average scores measured in 2019. 

Stream
Waypoint
Baseline
Housing Coalition
Access to Safe and Affordable Housing
1.0
Health & Wellbeing
Access to Health Resources
1.67
Health & Wellbeing
Health & Wellbeing
1.86
Economy
Sustainable Industries
2.0
Economy
Diversified Economic Sectors
1.63
Economy
Access to Meaningful Work Opportunities
1.44
Education
Access to Equitable and Diverse Education Opportunities
1.33
Education
Access to Quality Education Opportunities
1.50

HIKALOI AND NALU

Vibrant Hawaiʻi will measure these Indicators annually through an electronic survey disseminated to community members during Makahiki: a season of peace, celebration, and reflection signaled by the rise of the constellation Makaliʻi (Pleiades) in late October-early November to sync with our traditional season of reflection. Survey results are shared on our website and used to adjust our courses of action and advocacy to move closer to “Vibrant” or a score of 4.0. By disseminating our survey during Makahiki, we provide our community a tool to annually assess the Vibrant Hawaiʻi Indicators (nalu), while prompting participants to reflect on how they are contributing to the current state of those indicators (hikaloi). 

 

KILO: DATA INFORMED APPROACH TO SYSTEMS CHANGE

Vibrant Hawaiʻi Indicator scores are collected annually and recorded in tandem with other community and population-level indicators to guide and measure our collective impact. We use Clear Impact software to visually organize the data and follow the five steps in Clear Impact’s (2021) Turn the Curve Action Plan: record the data points, analyze the story behind the data, identify existing and new partners whose kulena can improve the data, brainstorm what works, and develop and implement a comprehensive action plan for Core Teams guided by 3 core questions of Asset Based Community Development created by the Collaborative for Neighborhood Transformation:

  • What can we (community members) do on our own?

  • What can we do in partnership with and with the support of organizations?

  • What is outside our agency to change?

VIBRANT HAWAIʻI’S APPROACH TO SYSTEMS CHANGE

The Water of System Change (Kania, Kramer, & Senge, 2018) describes six interdependent conditions that hold social problems in place: policies, practices, resource flows, relationships and connections, power dynamics, and mental models. Oftentimes, efforts to transform systems are focused on explicit indicators of change: shifting policies, practices, and resource flows. 

 

The Water of System Change warns changemakers that if they fail to invest in implicit indicators of change: building trust among stakeholders and shifting mental models, any explicit changes made will likely revert to its original broken state because of the implicit actions fueled by poor relationships, unbalanced power dynamics, and unhealthy mindsets. 

 

Vibrant Hawaiʻi’s kuleana (mission and approach to system change) is to convene conversations so that all waʻa (explicit actions) can travel toward a common goal, build community awareness, will, and action from the foundation of our shared values, shift deficit narratives, systems, and policy that perpetuate poverty and inequity, and implement strategies that are developed and resourced by the community and reflects native intelligence. 

 

We invest in sustained system transformation that is achieved through healing relationships and transformation of mental models. We do this by convening and kūkākākā: a unique dialogue application that fosters metanoia: suspending (kū) judgement, cynicism, and fear, and carving (kā) new paths forward with curiosity, compassion, and courage.

References

Aloha United Way (2018). ALICE: A study of financial hardships in Hawaiʻi. Retrieved from: www.auw.org/sites/default/filespictures/alohauinitedwayALICE-report-Hawai ́i-2017.pdf

Baum, F., MacDougall, C., & Smith, D. (2006). Participatory action research. Journal of epidemiology and community health, 60(10), 854.

Clear Impact (2021). Turn the curve thinking. Retrieved from: https://clearimpact.com/results-based-accountability/turn-the-curve-thinking/

Collaborative for Neighborhood Transformation. (n.d.). What is asset based community development (ABCD). Retrieved from: https://www.neighborhoodtransformation.net/pdfs/What_%20is_Asset_Based_Community_Development.pdf

Holtmann, B. (2011). What it Looks Like when it’s Fixed. A case study in developing a systemic model to transform a fragile social system. Johannesburg: PWC.

Kania, J., Kramer, M. and Senge, P. 2018. The Water of Systems Change. FSG. Retrieved from https://www.fsg.org/publications/water_of_systems_change

Pukuʻi, M. K., & Varez, D. (1983). 'Olelo No'eau: Hawaiian proverbs & poetical sayings. Honolulu, Hawai'i: Bishop Museum Press.

Sheldon, J. G., Like, E. L., & Prendergast, J. K. (1999). Ka Buke Moolelo o Hon. Joseph K. Nawahi. (2nd edition). Hilo, Hawaiʻi: Hale Kuamʻo, Ka Haka 'Ula o Ke'elikōlani. (original work published 1908)