# Methodology and Data Provenance This report makes quantitative claims about a project that merges code daily. This section states, for every class of claim, exactly where the data comes from, how fresh it is, what its error bars are, and how to regenerate it. Nothing in this report should be trusted *because the report says it*; everything should be re-derivable from the listed primary sources using the scripts under `tools/`. ## 1. Primary sources | Source | What it provides | Access method | |---|---|---| | `github.com/home-assistant/core` | Integration manifests (`manifest.json`), per-integration `quality_scale.yaml` rule files, code owners (`CODEOWNERS`), strict-typing roster (`.strict-typing`), test layout | Git clone; parsed by `tools/fetch_quality_scale.py` | | `analytics.home-assistant.io` | Opt-in installation telemetry: total reporting installations, per-integration install counts/percentages | Public JSON endpoints; pulled by `tools/fetch_install_base.py` | | GitHub Issues API on `home-assistant/core` | Open-issue counts per integration (the repo labels issues `integration: ` via an issue-form bot), issue ages, titles for thematic clustering | REST search API; pulled by `tools/fetch_issue_counts.py` | | `github.com/home-assistant/architecture` | Architecture Decision Records (ADRs) and architecture discussions | Read directly | | Home Assistant blog + developer blog (`developers.home-assistant.io/blog`) | Roadmap posts (published roughly twice yearly since mid-2024), Quality Scale relaunch announcement (October 2024), release notes | Read directly | | `developers.home-assistant.io` docs | Quality Scale rule definitions per tier, integration development model | Read directly | ## 2. Snapshot basis and drift The narrative tables in `data/` are a **snapshot constructed at authoring time** from the sources above. Three classes of numbers appear in this report, with different reliability: - **Class A — structural facts** (e.g., "the Quality Scale has four tiers plus legacy/internal/custom designations", "Silver requires a reauthentication flow", "integration issues are labeled `integration: `"). These change only via deliberate project decisions and are reliable as stated. - **Class B — distribution shapes** (e.g., "the large majority of integrations carry no tier", "issues cluster heavily in cloud-polling integrations"). These drift slowly; conclusions drawn from them are robust over a 6–12 month horizon. - **Class C — exact counts** (e.g., a specific integration's open-issue count or install percentage). These drift weekly. Every Class C number in this report is marked `≈` and must be regenerated via `tools/` before being used for anything operational (e.g., selecting the final integration list for the uplift program in Milestone 2). ## 3. Install-base methodology Home Assistant analytics are **opt-in** (Settings → System → Analytics). Historically, roughly a quarter to a third of the real installed base opts in; the published absolute counts therefore *understate* reality, but the **relative ranking** of integrations is the best available estimator of real-world popularity and is what this report uses. Known biases: - Power users (more integrations, more custom components) opt in at higher rates — long-tail integrations are likely *over*-represented in percentage terms. - Default/onboarding integrations (`sun`, `met`, `radio_browser`, `shopping_list`, `google_translate`, `mobile_app`, `backup`) appear at the top of any raw ranking because they ship enabled or near-enabled. The "top ~150" analysis in report 04 **excludes pure-default and helper/system domains** and ranks user-chosen integrations, since those are where contribution effort pays off. The exclusion list is in `tools/build_tables.py` and documented in `data/top_integrations.csv`. ## 4. Issue-triage methodology For each integration domain in the top-150 list: 1. **Count:** open issues matching `repo:home-assistant/core is:issue is:open label:"integration: "`. (The label is applied automatically by the issue-form bot based on the reporter's selection, so coverage is good but not perfect; issues filed before the form existed or mislabeled by reporters are missed. Estimated label coverage: high-80s to 90s percent.) 2. **Sample:** the 30 most recent and 15 oldest open issues per integration are read and hand-clustered into the theme taxonomy defined in report 04 §2 (auth/reauth, connection recovery, upstream API drift, entity correctness, setup/config-flow, performance, feature request misfiled as bug, unreproducible/stale). 3. **Severity mix:** issues are bucketed *breaks-setup* / *breaks-runtime* / *degrades-data* / *cosmetic* from the sample, and the bucket proportions are extrapolated to the count. This is an estimate, marked as such. 4. **Fixability:** each integration gets a fixability bucket — `local` (fixable entirely in `home-assistant/core`), `lib` (requires changes to the third-party client library, which may or may not be maintained), `vendor` (blocked on the device vendor's cloud/firmware), `arch` (requires a core architecture change). This bucket drives the effort scoring in report 06 more than the raw count does. ## 5. Quality-scale methodology Since the October 2024 relaunch, an integration's tier is declared in its `manifest.json` (`"quality_scale": "bronze" | "silver" | "gold" | "platinum" | "internal" | "legacy"`) and substantiated by a per-rule `quality_scale.yaml` checklist in the integration directory, validated by `hassfest` in CI. `tools/fetch_quality_scale.py` parses every `homeassistant/components/*/manifest.json` in a core checkout and tabulates the distribution; report 03 analyses the snapshot and — more importantly — analyses *which rules* most commonly separate each tier from the next, because that gap analysis is what sizes the uplift effort. ## 6. Roadmap-signal methodology Report 05 is qualitative. It synthesizes: the published roadmap blog posts (the project began publishing formal roadmap updates in 2024), the Open Home Foundation's stated priorities, the ADR corpus in `home-assistant/architecture`, recurring themes in release blog posts (2024.x–2025.x cycle: voice, AI, dashboards, backups, Matter, Z-Wave long-range, energy), and the "Working Group"/"Month of What the Heck" style community initiatives. Where this report attributes intent to the core team, it cites the public artifact; where it infers intent, it says "inferred." ## 7. Scoring methodology (preview) Report 06 scores each candidate path on five axes — **user impact breadth, user impact depth, durability, mergeability/friction, and strategic alignment** — each 0–10 with documented anchors, weighted, and divided by an effort estimate in engineer-months with a risk multiplier. The full model, the weights, the rationale for the weights, and a sensitivity analysis (does the ranking survive ±2 on any single score?) are in report 06. The ranking is stable under sensitivity testing; the #1 and #2 paths swap only if one assumes bug-fix durability equal to test-backed uplift durability, which §06.5 argues is implausible.