Prop 90: Community Health research, analytics, and benchmarking

Summary

If Web3 is all about communities, how do we know if we're heading in the right direction? We've been advancing a research project to:

• develop a framework for Community Health with actionable metrics
• create an open source data collection tool
• implement the tool in Cosmos Hub and others
• analyse the interactions and perceptions data to validate the framework and provide insights to advance the Cosmos community. We're seeking $30,000 or equivalent as grant match-funding to complete the project. For clarity, the output of this grant is a community health assessment (including conceptual framework, designing KPIs, building the data collection tool, etc). And a range of more specific, actionable insights on how to improve community health. We cant say what the specific insights are before doing the research, though.

About RnDAO

We're an R&D DAO with a mission to empower humane collaboration. Having already delivered research projects on sub-DAOs, DAO Conceptual Foundations, Decentralised Leadership, DAO Compensation and more: http:// rndao.mirror.xyz

Abstract

Like many others, Cosmos depends on the health of its community and its vibez—yet understanding and measuring these factors (what we call Community Health) is challenging. Today, Web3 communities are left to rely on infrequent transactions data (on-chain records), while data from the significantly more frequent social interactions is limited to basic indicators of Discord or Discourse, Web2-oriented bots like Statsbot that give simplistic measurements, or a patchwork of “homemade” surveys to fill in the gaps. These solutions are ill-suited for DAO communities, surveys are time-consuming for community managers and contributors to use, and the results of these approaches, hampered by poor indicators and/or poor sampling, are unreliable. Perhaps even direr, the lack of real-time analytics leaves DAO community leaders and members without established baselines to measure against to understand the impact of community-focused initiatives, identify best practices, monitor shocks to the system, or rapidly gauge the effects of system-wide changes (such as market crashes, protocol migrations, etc.).

Specification

For the conceptual framework, we have assembled a team including two PhDs in network science and an organisation designer with significant DAO and community building experience to bridge both theory and practice. We've reviewed over 50 papers on the topics of Community, Social Network Analysis, Resilience, Trust, Engagement, and more. And are synthesising all of these findings to define the key indicators that have high validity and high predictive capacity for community health, while also taking a holistic perspective that accounts for member's wellbeing and planning to collect data across communities to offer an ecosystem health scrore too. For the data collection tool, we're going beyond traditional surveys. This research proposal focuses on the use of two critical techniques as a starting point: Organizational Network Analysis (ONA) is a structured way to visualize how communications, information, and value creation occur through an organization based on interaction graphs. ONA has been shown to provide a wide range of insights to improve contributor retention, avoid member burnout, predict team performance and community resilience, identify key contributors, enable decentralization, and improve coordination. Although relatively new, ONA is gaining in popularity over traditional survey tools. Pulse Surveys are frequent and automated micro-surveys that provide qualitative and quantitative insights. In traditional organizations, they have been shown to increase employee response rate and employee engagement with related initiatives. They're also used as a tool for culture design and implementing culture change. Lastly, Pulse Surveys significantly reduce admin work for community managers and related roles. Importantly, these surveys offer an opt-out option from future prompts and are also framed to leverage coaching techniques to offer a prompt for reflection, thus offering value to participants as a recentering exercise on the value of community. The data is anonymised and collected in a central repository for this first phase (we're exploring decentralised hosting) and managed by a team having received ethics training and at risk of losing their credentials should it be misused.

Rationale

The techniques used (ONA based on communication and Pulse Surveys) provide maximum insights on Community Health compared with exclusively on-chain data analysis. And provide minimum disruption for community members and minimal admin compared to long-form surveys and user interviews. Crucially, the techniques selected and the usage of Discord messages increase participation by those less likely to respond to long surveys e.g. those less engaged and thus likely with the most valuable feedback to give. Although our initial focus is Community Health metrics, the proposed approach sets the foundation for further applications. Some of the potential applications and insights for Cosmos Hub and Cosmos-Chains-based DAOs are:

• Generate baseline metrics for Community Health / vibez to quantify and better understand the impact that a specific event is having on a community and/or sub-groups within the community
• Predict which contributors are likely to leave the DAO and take preventive action (without breaching privacy
• Build funnels to track member onboarding and identify areas for improvement
• Identify measurements of decentralization to serve as KPIs or Insights metrics
• Monitor specific topics like contributor wellbeing, alignment, community experience, etc. in near real-time
• Attract talent and investment with objective Community Health metrics instead of proxy metrics like member count or proposal count, or financial metrics such as TVL In addition to the initial research on Community Health, the potential applications mentioned above can enable more effective and targeted efforts to build healthier DAO communities.

Why our Team

Team Leads Katerinabc Ph.D. in Team Dynamics using Social Network Analysis, Teaching Collaboration, and Organizational Performance at Northwestern University (since 2016). Co-organized Learning in Networks sessions at the International Conference of Social Network Analysis (2018 - 2020), and previously advised a people analytics company on social network metrics. Thegadget.eth Software Engineer. Previously, Product Manager at Neolyze (Business Intelligence Dashboard for Instagram). Danielo Previously, Head of Governance at Aragon, 8 years experience in Organization Design consulting (clients include Google, BCG, Daymler, The UN, and multiple startups), and visiting lecturer at Oxford University. Note: Other RnDAO members will participate throughout the process in user research, literature review, tool development, and workshop facilitation. Team Advisor Sam Previously, Head of Technical Research at Aragon. Previously, Lead Developer of the official JavaScript API for the Ethereum blockchain.

Budget & Timeline

Budget $30,000 in USDC (equivalent to 3,165 ATOM using lower threshold for today dec 12)* DAO Health Conceptual Framework and translation into actionable metrics: $8,000 Data Collection Tool Development (discord bot for interactions data collection, config front-end, and results dashboard): $17,000 Data Analysis, Insights Report, and Community Workshop: $5,000

• the requested budget forms part of a total $90,000 target. Total project costs will be split across multiple, participating DAOs (grants already received from Aragon and Aave). Estimated Timeline taking into account a 1 weeks delay due to the end of year holidays

0. Kickoff (grant approval)
1. update of the conceptual framework and publication - estimated week 2-3 from kickoff
2. implementation of the tool in Cosmoss discord - estimated week 3-4 from kickoff
3. delivery of the Community Health report & workshop - estimated week 4-7 from kickoff

FAQ

What kind of insights can this proposal deliver? Although part of the work is precisely figuring out what metrics to use and which to discard, weve already spent significant time developing a framework to define community and community health and will use this as our starting point, further refining with feedback from the community. How will the insights add value to Cosmos? A weak community will fall apart in the face of difficulties while a strong community will come together stronger. The insights enable the Cosmos community to understand which areas to target with initiatives so to improve community health. Instead of shooting in the dark, the insights provide strategic focus and clarity. Will this proposal have a continuation? This is very much the start of what could become an ongoing workstream of research and action. Were hoping the community is satisfied with this pilot so that we can then make subsequent proposals to both deepen the research further and action insights to make the community stronger.

What reassurance can you provide for accountability? Were part of a DAO thats been around for over a year, and both the DAO and we, the leads of this proposal, have invested a hundred plus hours building up our reputation in the space with the names and social profiles we have disclosed. The reputational damage would cost multiple times the 30k being requested (not even counting the lost opportunity of building trust with the Cosmos community so we may make subsequent proposals). Of course, theres an uncertainty component, as with any R&D initiative, but you can count on us to do everything in our power to make this a success.