Back to all articlesCustomer feedback

Update feedback codes without breaking comparisons with earlier findings

Alactic Editorial9 min read

Prepared with AI assistance by Alactic Editorial. Examples identified as hypothetical are illustrative, not customer results.

Hypothetical example: Your latest feedback report shows fewer onboarding complaints. But last month, onboarding included account access problems; this month, those problems have their own code. The apparent improvement may be a classification change rather than a customer experience change.

To maintain a customer feedback codebook over time, preserve published definitions, record changes against stable code identifiers, and decide how historical material will be treated before comparing periods. Add, split, merge or retire codes when the analytical purpose requires it—not merely to tidy labels.

The practical control is a code-change register linked to versioned codebooks and coded material. It should let another analyst reconstruct what a finding meant, which records contributed to it, and whether a later finding measures the same concept.

1. Separate a label edit from a change in meaning

A code means more than its name. Its definition, inclusion and exclusion rules, relationship to other codes, and coding unit determine which material qualifies.

Before editing, ask: Could the same feedback receive a different coding decision after this change? If yes, treat it as a substantive revision.

  • Label-only edit: Correct a spelling error without changing scope. Preserve the former label as an alias.
  • Definition clarification: Make an existing boundary explicit. Test previously coded material; a clarification that changes decisions is substantive in practice.
  • Scope change: Add an inclusion, remove an exclusion, or change the evidence required. This needs a comparability decision.
  • Structural change: Split, merge, add or retire codes. Record predecessor and successor relationships, not just new names.

Keep identifiers distinct from labels. A familiar identifier should not conceal a new concept. When a code is divided into genuinely different concepts, create new identifiers and retain the original as historical.

2. Choose the change that serves the analysis

A more detailed codebook is not automatically better. Distinctions that analysts cannot apply consistently create fragile findings.

  • Add when a relevant concept falls outside existing definitions. Check that it is not simply unfamiliar wording for an existing theme.
  • Split when a broad code combines issues requiring different interpretation or action. Include both an other-specified category and an unspecified category where needed to preserve the parent's full scope.
  • Merge when separate codes no longer serve a useful distinction. Preserve the former distinctions in historical records.
  • Retire when a code leaves the current analytical scope or is replaced. Retirement stops future assignment; it does not erase historical meaning.

Hold uncertain themes in candidate notes containing the excerpt, proposed definition and analytical purpose. Review them at a defined checkpoint rather than creating a permanent code for every surprising comment.

Frequency alone is not a sufficient reason to split or retire a code. A rare theme may matter to the decision, while a frequent theme may not need subdivision.

3. Preserve the evidence behind each version

Keep a read-only snapshot of every codebook version used in a published analysis. Each entry should contain its identifier, label, definition, inclusion and exclusion rules, boundary examples, status, and predecessor or successor relationships.

For each assignment, retain the source reference, coded excerpt or location, code, codebook version and coding date. Document whether multiple codes can apply and whether the coding unit is a passage, conversation or another unit.

For each report, retain the selected corpus, period, counting unit, denominator and codebook version. Stable definitions cannot rescue a comparison that switches from counting conversations to counting excerpts.

Store revised assignments separately from original assignments. Label a revised historical series as restated rather than silently replacing the values behind an earlier finding. Distinguish a proposed definition, an approved definition and a version actually applied to the material: approval does not complete historical recoding.

4. Copy this filled code-change register

Fictional worked example: All identifiers, quantities, dates and decisions below are hypothetical. A feedback team introduces V2 at the start of April. These filled entries provide a reusable register format.

Change A: Split onboarding difficulty

  • ID, status and owner: CH-01; approved for V2, effective 1 April; feedback analysis lead. Historical comparison pending coverage review.
  • Old definition: C10, Onboarding difficulty: any explicit obstacle to completing initial setup. Includes access, configuration and uncertainty about the next step; these examples are not exhaustive. Excludes post-setup problems and general dissatisfaction without an explicit setup obstacle.
  • Successor definitions: C11, Initial account access: inability to enter the account during initial setup. C12, Setup configuration: difficulty completing required settings. C13, Next-step uncertainty: not knowing the next setup action. C14, Unspecified onboarding obstacle: an explicit setup obstacle with insufficient detail to classify its nature. C15, Other specified onboarding obstacle: a clearly described initial-setup obstacle outside C11–C13.
  • Shared boundaries: All children inherit C10's initial-setup requirement and exclusions. C14 is not a holding category for clearly described but unfamiliar obstacles. C15 records those obstacles with a descriptive note.
  • Mapping: C10 becomes historical-only. No automatic mapping to an individual child. Multiple specific children may apply to one conversation.
  • Affected material: January–March eligible corpus, including 84 conversations previously carrying C10.
  • Recoding choice: Re-read all 84 and review the remaining eligible corpus for missed evidence. Preserve original assignments.
  • Coverage gate: Check every old inclusion and exclusion against the successor family. Confirm that every qualifying old case fits at least one child and every child-qualified case satisfies the old definition. Investigate mismatches before approving a comparison.
  • Permitted comparison: Historical child trends require completed recoding. Parent-versus-family totals require completed recoding, a passed coverage check and consistent corpus and counting rules. Deduplicating children alone does not establish equivalent coverage.

Change B: Merge waiting codes

  • ID, status and owner: CH-02; approved for V2, effective 1 April; support insight lead.
  • Old definitions: C20, Slow first response: explicit dissatisfaction with waiting for an initial reply. C21, Slow follow-up: explicit dissatisfaction with waiting for a later reply on the same issue.
  • New definition: C22, Reply waiting dissatisfaction: explicit dissatisfaction with time awaiting an initial or subsequent reply. Excludes resolution duration unless reply waiting is stated.
  • Mapping and affected material: C20 or C21 maps to C22 only after confirming that the new definition is their union. Historical assignments cover 61 distinct conversations, with some overlap.
  • Recoding choice: Derive the combined historical series from the union of conversation identifiers after checking inclusion and exclusion rules.
  • Warning: Adding old totals double-counts overlap. V2 alone cannot support a first-response-versus-follow-up breakdown.

Change C: Add setup ownership transfer

  • ID, status and owner: CH-03; approved for V2, effective 1 April; feedback analysis lead.
  • Old definition: No dedicated code; related material might appear in notes but was not systematically classified.
  • New definition: C30, Setup ownership transfer: an explicit request or obstacle involving handing setup responsibility to another person. Excludes general requests for help without a transfer of responsibility.
  • Mapping and affected material: No reliable historical mapping. Applies from April and may coexist with onboarding obstacle codes.
  • Recoding choice: No historical review this cycle; earlier eligible material remains unassessed for C30.
  • Warning: Earlier periods are not coded for this theme, not zero. Do not call April an increase or claim the theme newly emerged.

Change D: Retire a catch-all

  • ID, status and owner: CH-04; approved for V2, effective 1 April; research reviewer.
  • Old definition: C40, Poor experience: any broadly negative account, without requiring a specific obstacle.
  • Successor and mapping: No replacement. Assign specific problem codes only when their evidence requirements are met; retain unclassified material for review.
  • Affected material and recoding: Preserve historical C40 assignments and stop future use. Re-read source material if a later analysis needs specific historical themes.
  • Warning: C40 ends as a series. Its disappearance does not mean poor experiences disappeared.

5. Work through coverage before counting

Continuing the hypothetical example: Reviewers test Change A with a passage describing a required setup item that the customer cannot obtain. The obstacle is clearly specified but is neither account access, configuration nor next-step uncertainty. It belongs in C15, not C14. Without C15, the successor family would omit a case that C10 included.

They also test a configuration complaint arising after setup. It fails the old definition and must fail the successor definition too. A family that captures more than its predecessor is no more equivalent than one that captures less.

After reviewing the full historical corpus, suppose all 84 old C10 conversations qualify for the new family, no additional qualifying conversations are found, and the definition review identifies no remaining coverage gaps. Hypothetical child counts are C11: 30, C12: 25, C13: 20, C14: 12 and C15: 7.

Those counts sum to 94 assignments, but their union contains 84 distinct conversations because some conversations have multiple children. The restated family total is therefore 84, not 94.

The count reconciliation supports the review; it does not replace it. Equal totals can conceal different included conversations. Record unmatched cases, their source references and the resolution. If the team intentionally excludes other specified obstacles instead, document a narrower successor scope and mark a series break—even after recoding.

6. Choose recoding or a limited comparison

Recode history when the decision requires a trend under the new definition and the original material supports it. Review a consistent historical scope, not only records already carrying the predecessor code. Otherwise, old omissions can survive inside an apparently complete restatement.

A limited comparison is preferable when earlier evidence is unavailable, too vague, or not worth reviewing for the current decision. Options include:

  • Compare an unchanged broad concept only after establishing equivalent coverage; report new children separately from their supported start date.
  • Start a new series with a visible definition break.
  • Report a new theme only for periods systematically reviewed for it.
  • Recode a defined subset and restrict conclusions to that subset.

A pilot can reveal whether definitions work; it does not establish full-period counts. Likewise, successful recoding cannot by itself make narrowed definitions equivalent to broader historical ones.

7. Test boundaries and repair overwritten history

Before release, assemble clear positives, exclusions, ambiguous passages and cases near changed boundaries. For a split, include multiple-child cases, unspecified obstacles and clearly specified obstacles outside the named categories. Test old negatives as well as old positives.

Have reviewers code independently, then discuss differences at passage level. Record whether disagreements arise from wording, insufficient detail, different coding units or interpretation. Revise rules and repeat the check rather than resolving every disagreement by adding a code.

If labels have already been overwritten, pause affected trend claims and preserve the current state. Seek earlier codebooks, exports, report notes and source-level assignments. A chart label alone cannot establish old inclusion rules.

Where source material survives, recode a bounded period and label it restated. Where historical meaning cannot be recovered, mark uncertainty or a series break. If a circulated conclusion changes, explain the correction rather than merely replacing its chart.

8. Apply a release check to the next report

Before changing the live codebook, complete a register entry covering predecessor and successor definitions, affected material, mapping, coverage checks, recoding and permitted comparisons. Assign an owner to unfinished historical work.

Before publishing, check that every changed code has an approved entry and every affected trend has either a supported common-definition analysis or a visible limitation. Monitor unresolved mappings and accumulation in other-specified or unspecified categories; these are review signals, not automatic evidence of a defective codebook.

The completion test is practical: another analyst can identify the definition behind a finding, reproduce its counting rule, and explain why the earlier period is—or is not—comparable.

Customer feedbackQualitative analysisCodebook management

Put the next conversation to work.

Explore Alactic Communications for WhatsApp conversations, campaigns and AI-assisted workflows. Choose the product and plan that fit your team.

Explore Communications