Map missing, blank, and null values without erasing useful data
Prepared with AI assistance by Alactic Editorial. Examples identified as hypothetical are illustrative, not customer results.
In a hypothetical integration, a contact update arrives without a job title. The mapper turns the missing property into an empty string, sends it to the destination, and erases a title that was already correct. The API request succeeds; the business record becomes less useful.
The fix is to map intent before value. For each field, decide whether an incoming representation means keep the destination, set a value, clear it, reject the change, or apply a default. Missing, blank, and null are inputs to that decision—not interchangeable instructions.
This is a contract for ongoing updates, not a data-cleanup exercise. Its purpose is to distinguish an intentional clear from an answer that was unavailable when the source sent its message.
1. Separate representations from actions
Preserve these source states until the field-specific rule has evaluated them:
- Missing: the property is absent from the payload.
- Explicit null: the property is present with the JSON value
null. - Empty string: the property is present as
"". - Whitespace-only string: the property contains only whitespace.
- Concrete value: the property contains a candidate value, including potentially valid
falseor0. - Explicit clear: a separate operation or marker requests removal.
Translate these representations into KEEP, SET(value), CLEAR, or REJECT(reason). Defaulting is a conditional way to produce SET(value), not a universal response to empty input.
A truthiness expression such as incomingValue || defaultValue collapses meaningful distinctions. It can replace zero, ignore false, and conceal the difference between an absent property and an empty string. Check property presence separately from value. If an earlier transformation has already replaced missing properties with nulls, the mapper may lack the information needed to choose correctly.
2. Establish the source's authority
Name the authority for each field before writing transformation rules. A source might be permitted to propose a website but not remove it. Another might own an operational status but have no authority over account-manager notes.
Establish whether messages are partial updates or complete snapshots. In a partial update, omission can mean the field was not part of the change. A snapshot contract could define omission differently. Neither interpretation should be guessed from the payload's appearance.
For each field, record who may set or clear it, what null means, whether whitespace is trimmed, where defaults apply, and how invalid inputs are handled.
Permission to clear deserves the same scrutiny as permission to set. OWASP authorization guidance recommends least privilege, deny-by-default behavior, and permission checks on every request. Applied here as a design recommendation, a clear marker expresses intent but does not grant authority. The integration must still authorize the requested operation.
3. Reuse this filled hypothetical mapping table
Fictional worked example: A service application sends partial account and contact updates to a CRM. The destination hypothetically supports a distinct clearing operation. These are proposed contract rules, not claims about an Alactic feature or any particular API.
The optional inbound properties below have field-specific meanings. Account review status is optional in the incoming message, but the destination workflow requires a stored status. Its creation default is therefore a contrasting case, not a reason to default other optional fields.
The hypothetical clearFields list is a proposed integration convention. A payload supplying both a value and a clear instruction for the same field is rejected before writing. Tests assume an authorized source unless stated otherwise.
| Field | Source representation | Destination action | Update implication | Hypothetical test with expected result |
|---|---|---|---|---|
| Job title | Missing, null, empty, or whitespace-only string | KEEP; omit from partial write. | Unavailable answers cannot erase an existing title. | Existing Operations Lead plus null remains Operations Lead. |
| Job title | Nonblank string; or authorized explicit clear | Trim and validate, then SET; explicit clear becomes CLEAR. | No default. Clearing requires a separate instruction. | " Team Lead " stores Team Lead; a separate authorized clear removes it. |
| Company website | Missing or null; empty or whitespace-only string | Missing/null: KEEP. Blank: REJECT(blank_not_allowed). | Blank is a visible source-quality problem, not removal. | Three spaces leave the existing website unchanged and record the rejection reason. |
| Company website | Nonblank value; or authorized explicit clear | Trim, validate against agreed URL rules, then SET; authorized clear becomes CLEAR. | Invalid values do not replace a valid website. No invented default. | not a URL is rejected; an authorized clear removes the existing website. |
| Preferred contact time | Missing; explicit null | Missing: KEEP. Null: CLEAR only from the designated scheduling source. | Null deliberately means something different here than for job title. | Existing morning becomes unset after authorized null; another source's null is rejected. |
| Preferred contact time | Recognized value; empty, whitespace-only, or unrecognized string | Recognized value: SET. Otherwise REJECT. | No preference is inferred from a blank answer. | afternoon is stored; "" leaves the current value unchanged with a rejection reason. |
| Estimated monthly order count | Missing or null; integer zero or positive integer | Missing/null: KEEP. Accepted integer: SET. | Zero is supplied information, not missing data. | Existing 12 plus numeric 0 becomes zero. |
| Estimated monthly order count | Blank, numeric text, negative number, or fraction; authorized explicit clear | Invalid representation: REJECT. Authorized clear: CLEAR. | Strict typing prevents silent coercion; no zero default. | "0" is rejected; a separate authorized clear unsets the estimate. |
| Account review status | Missing on confirmed creation; missing on update | Create: SET(unreviewed). Update: KEEP. | Default only initializes the required destination status. | A new record becomes unreviewed; an existing reviewed record stays reviewed. |
| Account review status | Allowed status; null, blank, or clear request | Allowed status: SET. Null, blank, or clear: REJECT. | The destination workflow never intentionally loses its required status. | Authorized reviewed is stored; null leaves the current status unchanged and records rejection. |
To reuse the artifact, replace the fictional fields, authority rules, accepted values, and destination encodings. Keep separate rows wherever representations have materially different consequences.
4. Work through one update end to end
Continue the fictional example with an existing contact whose title is Operations Lead, website is populated, order estimate is 12, and review status is reviewed. An authorized service source sends:
{
"jobTitle": null,
"companyWebsite": " ",
"estimatedMonthlyOrderCount": 0
}The mapper produces four decisions: keep the title, reject the blank website, set the order estimate to zero, and keep the omitted review status. Under this example's field-level rejection rule, valid independent changes may proceed. The outbound partial update contains only the order estimate.
The expected result is an unchanged title, website, and review status, with the estimate now zero. The exception record identifies companyWebsite and blank_not_allowed. This is not a wholly successful event: one field needs correction even though another changed correctly.
If those fields instead formed one indivisible business decision, the contract should reject the whole update. Choose rejection scope explicitly rather than letting an implementation accident decide it.
5. Translate actions into destination semantics
The mapping decision is not the outbound payload. An adapter must translate it into the destination's documented behavior.
KEEP may mean omission for a partial update, but omission in a replacement operation might not preserve data. Confirm operation semantics and inspect serialization: a serializer that emits null for omitted fields can undo a correct mapping decision. An HTTP method alone does not settle those details.
CLEAR might require null, an empty string, or a dedicated removal operation. If the destination cannot express the intended clear, reject or hold it for review rather than substituting a syntactically accepted blank.
Preserve property presence and raw value
Check source authority and field rules
Produce KEEP, SET, CLEAR, or REJECT
Encode accepted actions for the destination
Validate the outbound request
Record acknowledgment and reconcile uncertaintyAvoid reading a whole record and writing it back merely to preserve untouched fields. That can overwrite intervening changes. Use documented conditional updates or version checks where available; otherwise define how concurrent changes will be detected and reviewed.
6. Keep defaults out of ordinary updates
A default is useful when it establishes an initial workflow state. It becomes dangerous when it pretends to be source knowledge. In the fictional contract, unreviewed initializes a workflow; defaulting an unavailable order estimate to zero would assert a business fact that was never supplied.
Determine creation versus update before applying defaults. Inserting defaults during initial normalization could reset an established status during an unrelated contact update.
A rule that defaults only when the destination is empty also requires reliable current-state knowledge. A read followed by a write can race with another change. Prefer documented destination-side conditional behavior where available; otherwise treat that sequence as a concurrency risk, not an atomic guarantee.
Preserve the distinction between an explicitly supplied default value and an omitted field. They may produce the same initial stored value but represent different decisions.
7. Test transitions and serialized requests
API acceptance does not prove mapping correctness. Start with a known destination state, apply the input, and inspect both the outbound request and resulting record.
- Populated destination: test missing, null, empty string, and whitespace separately.
- Explicit clear: test authorized and unauthorized sources, plus a conflicting value-and-clear payload.
- Creation versus update: confirm that defaults initialize only the permitted case.
- Edge values: test zero and false wherever those types are valid.
- Retries and delayed events: confirm they do not reapply defaults or silently reverse newer changes.
A repeated clear may appear harmless until a newer value has been entered between attempts. Define stale-event handling using supported versioning or ordering mechanisms; where those are unavailable, route ambiguous destructive changes for review.
Acceptance criteria should be observable: keep cases preserve values, authorized clears affect only intended fields, invalid inputs yield visible reasons, and serialized requests retain the intended distinctions.
8. Make failures recoverable
Record the field, action, reason, source identity, event reference, contract version, and destination acknowledgment. An intentional keep and a validation rejection both may omit an outbound field, but they require different operational responses.
A timeout leaves the write outcome uncertain. Reconcile it through supported destination mechanisms before replaying a destructive operation.
If a defect starts erasing values, pause the affected field or route. Identify affected writes using event references and contract versions. Restore values only from trustworthy prior-state records after checking for legitimate intervening changes; otherwise request review. Replaying corrected events cannot reconstruct a lost value that the source never contained.
Begin rollout with one optional field and controlled records. Compare decisions with resulting state, tracking unexpected clears, rejection reasons, defaults applied on updates, and unauthorized clear attempts. Expand only after unexplained differences have been resolved.
Put the next conversation to work.
Explore Alactic Communications for WhatsApp conversations, campaigns and AI-assisted workflows. Choose the product and plan that fit your team.
Explore Communications