## 8.3. Periodic Execution

This section describes procedures that can be used to batch transactions when executing large write operations.

Cypher is great for querying graphs and importing and updating graph structures. While during imports you can use `PERIODIC COMMIT` to control transaction sizes in memory, but for other graph refactorings it’s not that easy to commit transactions regularly to free memory for new update state.

`CALL apoc.periodic.commit(statement, params)` | repeats an batch update statement until it returns 0, this procedure is blocking
`CALL apoc.periodic.rock_n_roll(statementIteration, statementAction, batchSize) YIELD batches, total` | iterate over first statement and apply action statement with given transaction batch size. Returns to numeric values holding the number of batches and the number of total processed rows. E.g.

This section includes:

- Section 8.3.1, “apoc.periodic.iterate”
- Section 8.3.2, “apoc.periodic.commit”
- Section 8.3.3, “apoc.periodic.rock_n_roll”

### 8.3.1. apoc.periodic.iterate

With `apoc.periodic.iterate` you provide 2 statements, the first outer statement is providing a stream of values to be processed. The second, inner statement processes one element at a time or with `iterateList:true` the whole batch at a time.

The results of the outer statement are passed into the inner statement as parameters, they are automatically made available with their names.

Table 8.9. configuration options
param | default | description
--- | --- | ---
batchSize | 1000 | that many inner statements are run within a single tx params: {_count, _batch}
parallel | false | run inner statement in parallel, note that statements might deadlock
retries | 0 | if the inner statement fails with an error, sleep 100ms and retry until retries-count is reached, param {_retry}
iterateList | false | the inner statement is only executed once but the whole batchSize list is passed in as parameter {_batch}
params | {} | externally passed in map of params
concurrency | 50 | How many concurrent tasks are generate when using `parallel:true`
failedParams | -1 | If set to a non-negative value, for each failed batch up to `failedParams` parameter sets are returned in in `yield failedParams`.

| We plan to make `iterateList:true` the default in upcoming releases, due to the automatic UNWINDing and providing of nested results as variables, most queries should continue work.

So if you were to add an `:Actor` label to several million `:Person` nodes, you would run:

    CALL apoc.periodic.iterate(
      "MATCH (p:Person) WHERE (p)-[:ACTED_IN]->() RETURN p",
      "SET p:Actor",
      {batchSize:10000, parallel:true})

Which would take 10k people from the stream and update them in a single transaction, executing the second statement for each person.

Those executions can happen in parallel as updating node-labels or properties doesn’t conflict.

If you do more complex operations like updating or removing relationships, either don’t use parallel OR make sure that you batch the work in a way that each subgraph of data is updated in one operation, e.g. by transferring the root objects. If you attempt complex operations, try to use e.g. `retries:3` to retry failed operations.

    CALL apoc.periodic.iterate(
      "MATCH (o:Order) WHERE o.date > '2016-10-13' RETURN o",
      "MATCH (o)-[:HAS_ITEM]->(i) WITH o, sum(i.value) as value SET o.value = value",
      {batchSize:100, parallel:true})

iterating over the whole batch (more efficient).

    CALL apoc.periodic.iterate(
      "MATCH (o:Order) WHERE o.date > '2016-10-13' RETURN o",
      "MATCH (o)-[:HAS_ITEM]->(i) WITH o, sum(i.value) as value SET o.value = value",
      {batchSize:100, iterateList:true, parallel:true})

The stream of other data can also come from another source, like a different database, CSV or JSON file.

### 8.3.2. apoc.periodic.commit

Especially for graph processing it is useful to run a query repeatedly in separate transactions until it doesn’t process and generates any results anymore. So you can iterate in batches over elements that don’t fulfill a condition and update them so that they do afterwards.

| as a safety net your statement used inside `apoc.periodic.commit` must contain a `LIMIT` clause.

The query is executed repeatedly in separate transactions until it returns 0.

    call apoc.periodic.commit(
      "match (user:User) WHERE exists( user.city )
       with user limit {limit}
       MERGE (city:City {name:user.city})
       MERGE (user)-[:LIVES_IN]->(city)
       REMOVE user.city
       RETURN count(*)",
      {limit:10000})

Table 8.10. Results
Updates | Executions
--- | ---
2000000 | 200

### 8.3.3. apoc.periodic.rock_n_roll

copies over the `name` property of each person to `lastname`.

    CALL apoc.periodic.rock_n_roll('match (p:Person) return id(p) as id_p', 'MATCH (p) where id(p)={id_p} SET p.lastname =p.name', 20000)