{
  "$type": "site.standard.document",
  "canonicalUrl": "https://rednafi.com/python/modify-iterables-while-iterating/",
  "description": "Safely modify lists, sets, and dictionaries while iterating using list comprehensions, filters, or copying to avoid skipping elements.",
  "path": "/python/modify-iterables-while-iterating/",
  "publishedAt": "2022-03-04T00:00:00.000Z",
  "site": "at://did:plc:fgtm2c26vfcj74rfmeggbyqj/site.standard.publication/3mnl6f7ob462z",
  "tags": [
    "Python"
  ],
  "textContent": "If you try to mutate a sequence while traversing through it, Python usually doesn't\ncomplain. For example:\n\nThe above snippet iterates through a list of numbers and modifies the list l in-place to\nremove any even number. However, running the script prints out this:\n\nWait a minute! The output doesn't look correct. The final list still contains 56 which is\nan even number. Why did it get skipped? Printing the members of the list while the for-loop\nadvances reveal what's happening inside:\n\nFrom the output, it seems like the for-loop doesn't even visit all the elements of the\nsequence. However, trying to emulate what happens inside the for-loop with iter and next\nmakes the situation clearer. Notice the following example. I'm using ipython shell to\nexplore:\n\nThe REPL experiment reveals that:\n\n> Whenever you remove an element of an iterable that's already been visited by the iterator,\n> in the next iteration, the iterator will jump right by 1 element. This can make the\n> iterator skip a value. The opposite is also true if you prepend some value to a sequence\n> after the iterator has started iterating. In that case, in the next iteration, the\n> iterator will jump left by 1 element and may visit the same value again.\n\nHere's what happens when you prepend values after the iteration has started:\n\nNotice how the element 4 is being visited twice after prepending a value to the list l.\n\nSolution\n\nTo solve this, you'll have to make sure the target elements don't get removed after the\niterator has already visited them. You can iterate in the reverse order and remove elements\nmaintaining the original order. The first snippet can be rewritten as follows:\n\nRunning the script prints:\n\nNotice, how the iterator now visits all the elements and the final list contains the odd\nelements as expected.\n\nAnother way you can solve this is - by copying the list l before iterating. But this can\nbe expensive if l is large:\n\nThis time, the order of the iteration and element removal is the same, but that isn't a\nproblem since these two operations occur on two different lists. Running the snippet\nproduces the following output:\n\nWhat about dictionaries\n\nDictionaries don't even allow you to change their sizes while iterating. The following\nsnippet raises a RuntimeError:\n\nYou can solve this by making a copy of the keys of the dictionary and iterating through it\nwhile removing the elements from the dictionary. Here's one way to do it:\n\nRunning the snippet prints:\n\nVoila, the key-value pairs of the even numbers have been removed successfully!\n\nFurther reading\n\n- [How to modify a list while iterating - Anthony Sottile]\n\n\n\n\n[how to modify a list while iterating - anthony sottile]:\n    https://www.youtube.com/watch?v=JXis-BKRDFY",
  "title": "Modify iterables while iterating in Python"
}