Flattening Nested Arrays in DataWeave

Nested arrays appear frequently in integration payloads: accounts contain contacts, orders contain lines, batches contain records, and API responses contain pages of results.

Sometimes the target needs that hierarchy. Other times you need one flat collection.

DataWeave provides flatten, and in transformations that map and flatten in one operation, flatMap can express the intent more directly.

Basic `flatten` Example

Input:

[
  ["A", "B"],
  ["C"],
  ["D", "E"]
]

Transformation:

%dw 2.0
output application/json
---
flatten(payload)

Output:

["A", "B", "C", "D", "E"]

The nested arrays are combined into one array.

A Common Integration Example

Suppose accounts contain contacts:

[
  {
    "Id": "A1",
    "Name": "Acme",
    "Contacts": [
      { "Id": "C1", "Name": "Ravi" },
      { "Id": "C2", "Name": "Maria" }
    ]
  },
  {
    "Id": "A2",
    "Name": "Global Media",
    "Contacts": [
      { "Id": "C3", "Name": "James" }
    ]
  }
]

If you map each account to its contacts:

payload map (account) -> account.Contacts default []

The result is an array of arrays:

[
  [
    { "Id": "C1", "Name": "Ravi" },
    { "Id": "C2", "Name": "Maria" }
  ],
  [
    { "Id": "C3", "Name": "James" }
  ]
]

If the target expects one contact collection:

flatten(
    payload map (account) -> account.Contacts default []
)

Now all contacts are in one array.

Preserve Parent Context Before Flattening

A more realistic requirement is to flatten the contacts while retaining information about their parent account.

%dw 2.0
output application/json
---
flatten(
    payload map (account) ->
        (account.Contacts default []) map (contact) -> {
            accountId: account.Id,
            accountName: account.Name,
            contactId: contact.Id,
            contactName: contact.Name
        }
)

Output:

[
  {
    "accountId": "A1",
    "accountName": "Acme",
    "contactId": "C1",
    "contactName": "Ravi"
  },
  {
    "accountId": "A1",
    "accountName": "Acme",
    "contactId": "C2",
    "contactName": "Maria"
  },
  {
    "accountId": "A2",
    "accountName": "Global Media",
    "contactId": "C3",
    "contactName": "James"
  }
]

This is a useful integration pattern: enrich child records with required parent context before removing the hierarchy.

Once the structure is flattened, relying on positional relationships to reconstruct the parent later is fragile.

Using `flatMap`

When your transformation naturally means "map each parent to zero or more child records and return one flat array," flatMap can make that intention clearer.

The previous example can be expressed as:

%dw 2.0
output application/json
---
payload flatMap (account) ->
    (account.Contacts default []) map (contact) -> {
        accountId: account.Id,
        accountName: account.Name,
        contactId: contact.Id,
        contactName: contact.Name
    }

Conceptually:

map + flatten → flatMap

That does not mean flatMap should replace every use of flatten. Use it when mapping and flattening are one logical operation.

Optional Child Arrays

Enterprise payloads often omit a child collection entirely or provide null.

This pattern is safe and expressive:

payload flatMap (account) ->
    (account.Contacts default []) map (contact) -> {
        accountId: account.Id,
        contactId: contact.Id
    }

An account with no contacts contributes zero items to the final array.

Whether that is acceptable depends on the requirement. If every account is expected to have at least one contact, silently producing no child records may hide a data-quality problem.

Filter Children Before Flattening

Suppose only contacts with an email address should be included:

payload flatMap (account) ->
    (account.Contacts default [])
        filter (contact) -> contact.Email != null
        map (contact) -> {
            accountId: account.Id,
            contactId: contact.Id,
            email: contact.Email
        }

This combines several operations while keeping their responsibilities visible:

parent iteration → child selection → child transformation → flat result

Do Not Flatten Just Because You Can

Hierarchy often carries business meaning.

For example:

{
  "orderId": "O1",
  "lines": [ ... ]
}

may be exactly the contract expected by a downstream order API.

Flattening order lines into unrelated records can make it harder to preserve transaction boundaries, parent-level attributes, or ordering.

Before flattening, ask what the next system needs.

Nested structure required downstream? → preserve it
Independent child records required?   → flatten may help

Flattening Is Not Deduplication

If the same child record appears in multiple nested arrays, flatten does not remove duplicates.

For example:

[
  ["A", "B"],
  ["A", "C"]
]

becomes:

["A", "B", "A", "C"]

If uniqueness is required, that is a separate transformation and should be based on a meaningful key or contract rule.

In enterprise integrations, two records that look identical are not necessarily duplicates.

Flattening Is Not Pagination Handling

An API may return multiple pages, each containing an array of records. Combining those page arrays with flatten can be part of the final transformation, but it does not solve pagination itself.

You still need correct logic for:

detecting additional pages;
fetching them;
handling partial failures;
respecting API limits;
deciding how much data should be held in memory.

For very large datasets, collecting every page and flattening everything in memory may not be the right architecture.

Quick Decision Guide

Situation	Typical approach
Already have an array of arrays	`flatten`
Map parents to child arrays, then combine	`flatMap` or `map` + `flatten`
Need parent information in flat child records	enrich children before flattening
Need to remove unwanted children	`filter` before flattening/result construction
Need unique records	separate deduplication rule
Downstream requires hierarchy	do not flatten

Practical Rule

Flatten only after deciding that the nested relationship is no longer required by the next processing step.

And when child records still need parent information, copy that context into each child before flattening the hierarchy.

Basic flatten Example