Resilient and Nested Iteration

In Lesson 3 you ran a function across each item with map and got back a tidy, predictable result every time. That works beautifully when the data is clean and every piece is a single value.

Real data is rarely that polite. Picture six new Rosewood Bakery branches texting in their daily takings as hand-typed strings: a couple are mistyped, your parser chokes on them, and that one failure mid-map throws away every good result it computed before. Other times the natural unit is not a single value at all but a whole group, a table you want to run code on, group by group. This lesson gives you the two tools that handle exactly those cases: resilient iteration that survives failures (safely, possibly), and nested iteration over whole tables tucked inside a column (nest, unnest).

By the end of this lesson you will be able to:

• Explain why one broken element aborts an entire map, and what that costs you
• Run a function over every element and capture each result-or-error with safely, never stopping
• Reach for possibly when you just want a sensible fallback value on failure
• Read a list-column (a column whose cells are whole tables) and move between flat and nested with nest and unnest
• Run a real calculation on each group by mapping over a list-column, then tidy the result back

Prerequisites: Lesson 3 (the purrr map family), how to write a function and pass a \(x) ... lambda, and that a column of a data frame or tibble holds equal-length values.

One broken element stops the whole run

Rosewood Bakery is opening six new branches, and head office texts in each branch's recent daily takings as a hand-typed string like "120,135,150". You want each branch's mean daily takings. So you write a small parser, then map it over the branches. Build this week's batch once (run it), and notice that two branches were typed badly:

One branch at a time is fine. But the moment you map over all six, the run hits airport (the third element), mean_takings calls stop(...), and that error does not just skip that branch, it aborts the entire call. You get nothing back, not even the three good branches that came before it:

map_dbl(takings, mean_takings)
#> Error in mean_takings(txt): non-numeric takings: two hundred

safely(): catch every failure, keep going

safely() is an adverb: hand it a function and it returns a new function that does the same job but can never throw. Instead of returning a bare value (or crashing), the wrapped function always returns a small two-slot list: result and error. Exactly one slot is filled.

Because the wrapped function never throws, map runs to the very end no matter how many elements break. Here is the precise idea. safely turns a function \(f\) into a new function \(f^{\star}\). For any input \(x\), \(f^{\star}(x)\) is the pair

\[ f^{\star}(x) = \begin{cases} (\,\text{result}=f(x),\ \text{error}=\texttt{NULL}\,) & \text{if } f(x) \text{ succeeds} \\ (\,\text{result}=\texttt{NULL},\ \text{error}=e\,) & \text{if } f(x) \text{ raises error } e \end{cases} \]

and crucially \(f^{\star}\) itself never raises. Wrap mean_takings and map the wrapped version over all six branches:

Now nothing is lost. Every branch is accounted for, and you can pull out just the numbers you wanted, using NA for the ones that broke:

What does safely hand back?

You wrap a parsing function with safely() and map it over 100 files, 3 of which are corrupt. What do you get back?

possibly(): just give me a fallback

The result/error pairs are perfect when you want to inspect why things failed. Often you do not care: you just want a plain answer per element, with a sensible stand-in where it broke. That is possibly(). You give it your function and an otherwise value to return on any failure, and it hands back the value directly, no wrapper to unpack.

Same answer as the safely version above, but in one step and ready for arithmetic. The rule of thumb:

• Want to diagnose failures (log them, retry them)? Use safely and read the error slots.
• Want a clean result with a fallback and do not care about the error detail? Use possibly(otherwise = ...).

Make it resilient, your way

Head office wants each branch's highest single day's takings as a plain numeric vector, with NA where a branch was typed badly, ready to drop into a report. You already have takings and this strict parser:

safe_max <- possibly(max_takings, otherwise = NA_real_)
map_dbl(takings, safe_max)
#>   downtown    harbour    airport university     suburb     market
#>        150        121         NA         NA         70         99

List-columns: a whole table inside one cell

Now the second half. So far each element you iterated over was a single value or a string. But the natural unit is often a group: all of croissant's daily sales, all of muffin's, all of bagel's. R lets a single column hold a whole tibble in each cell, one per group. That is a list-column, and nest() builds it.

Take Rosewood's six-day sales in long form (one row per item per day). nest() collapses it to one row per item, packing each item's days-and-units into its own mini-tibble in a list-column. unnest() reverses it, spreading the tables back out into ordinary rows. Toggle between the two views below, then press Run to do it for real:

The nested cell does not say 40, 45, 50; it says tibble [3 x 2], a promise that a whole 3-row, 2-column table is folded up in there. The flat frame and the nested frame hold the same data, shaped for two different jobs: flat for filtering and joining, nested for "do something to each group as a unit".

Map a calculation over each group

Here is why list-columns matter for iteration. Once each group is a single cell, you can map a function over that list-column and get one result per group, no loop, no copy-paste, no splitting the data into pieces by hand. The pattern is always the same three beats:

Build the full six-day sales inline, nest by item, then map a few summaries over each item's table:

Each map call walked the three mini-tables and returned one number per group, lined up beside its item. When you want the per-row detail back, unnest() the list-column and the new summary columns ride along, repeated down each group's rows:

What did nest produce?

You start with a 200-row sales table covering 8 products and run nest(sales = c(day, units)) grouped by product. What does the result look like?

One summary per group, the tidy way

Put it together. Using sales_long from above, nest each item's rows into a sales list-column, then add a column avg holding each item's mean daily units by mapping over that list-column. The result should have one row per item. Fill in the blank with the calculation that runs on each group's mini-table d.

avg_by_item <- sales_long |>
  nest(sales = c(day, units)) |>
  mutate(avg = map_dbl(sales, \(d) mean(d$units)))

avg_by_item |> select(item, avg)
#> # A tibble: 3 x 2
#>   item        avg
#>   <chr>     <dbl>
#> 1 croissant    50
#> 2 muffin       30
#> 3 bagel        40

References

A few authoritative, free places to take this further:

• purrr: safely(), possibly(), quietly() - the official reference for the three adverbs and exactly what each one captures.
• tidyr: nest() and unnest() - how list-columns are built and spread back, with the current argument syntax.
• R for Data Science (1e): Many models - the canonical worked example of nest plus map plus list-columns, fitting one model per group.
• R for Data Science (2e): Iteration - the map family in context, the foundation everything here builds on.

Lesson 4 complete

You can now make iteration survive the real world. safely runs your function over every element and hands back a result/error pair so a single failure can never abort the run; possibly does the lighter job of returning a clean value with a fallback. And with nest you fold each group into a list-column of whole tables, map a calculation, even a model, over them, and unnest to tidy the answers back out.

That is the end of R Foundations: Iteration. You started by replacing loops with vectorization, moved to the apply family and then purrr's typed map, and you finish able to iterate over messy, grouped, real data without it falling over. Next up, the Debugging course: when a function does misbehave, how to find out exactly why.