## Data and Dependencies

## Learning Objectives * Learn about how to create dependencies between kernel functions * Learn about how to move data between the host and device(s) * Learn how to represent basic data flow graphs

#### Moving data with USM

With USM you move data and create dependencies between kernel functions explicitly

* Manual chaining of dependencies * Explicit data movement

#### Creating dependencies

![SYCL](../../Static/images/data_dependency.svg "SYCL")

* Kernel A first writes to the data * Kernel B then reads from and writes to the data * This creates a read-after-write (RAW) relationship * There must be a dependency created between Kernel A and Kernel B

#### Moving data

![SYCL](../../Static/images/data_movement.svg "SYCL")

* Here both kernel functions are enqueued to the same device, in this case a GPU * The data must be copied to the GPU before the Kernel A is executed * The data must remain on the GPU for Kernel B to be executed * The data must be copied back to the host after Kernel B has executed

#### Data flow

![SYCL](../../Static/images/data_flow.svg "SYCL")

* Combining kernel function dependencies and the data movement dependencies we have a final data flow graph * This graph defines the order in which all commands must execute in order to maintain consistency * In more complex data flow graphs there may be multiple orderings which can achieve the same consistency

#### Data flow with USM


#include <sycl/sycl.hpp>
#include <vector>

int main() {
  constexpr int N = 1024;
  std::vector<int> data(N, 1);
  sycl::queue q;
  int *dev = sycl::malloc_device<int>(N, q);

  sycl::event e1 = q.copy(data.data(), dev, N);

  sycl::event e2 = q.parallel_for(N, e1, [=](sycl::id<1> i) {
    dev[i] *= 2;
  });

  sycl::event e3 = q.parallel_for(N, e2, [=](sycl::id<1> i) {
    dev[i] += 1;
  });

  q.copy(dev, data.data(), N, e3).wait();
  assert(data == std::vector<int>(N, 3));

  sycl::free(dev, q);
}

* With USM you define dependencies explicitly by passing around `event` objects. * Data movement is performed explicitly with `copy`.

* Every command enqueued to the `queue` returns an `event` representing its completion.

* Passing an `event` when enqueuing another command makes it wait on that command first. * Kernel B waits on Kernel A, and the copy-back waits on Kernel B, giving the read-after-write chain.

#### Concurrent data flow

![SYCL](../../Static/images/concurrent_data_flow.svg "SYCL")

* If two kernels are accessing different data then there is no dependency between them * In this case the two kernels and their respective data movement are independent * By default `queue`s are out-of-order which means that these commands can execute in any order * They could also execute concurrently if the target device is able to do so

#### Concurrent data flow with USM


#include <sycl/sycl.hpp>
#include <vector>

int main() {
  constexpr int N = 1024;
  std::vector<int> dataA(N, 1), dataB(N);
  sycl::queue q;
  int *devA = sycl::malloc_device<int>(N, q);
  int *devB = sycl::malloc_device<int>(N, q);

  sycl::event e1 = q.copy(dataA.data(), devA, N);
  sycl::event e2 = q.copy(dataB.data(), devB, N);

  sycl::event e3 = q.parallel_for(N, e1, [=](sycl::id<1> i) { devA[i] *= 2; });
  sycl::event e4 = q.parallel_for(N, e2, [=](sycl::id<1> i) { devB[i] += 1; });

  q.copy(devA, dataA.data(), N, e3).wait();
  q.copy(devB, dataB.data(), N, e4).wait();

  assert(dataA == std::vector<int>(N, 2));
  assert(dataB == std::vector<int>(N, 2));

  sycl::free(devA, q);
  sycl::free(devB, q);
}

* Two independent chains never share an `event`, so the runtime is free to overlap them.

* Each kernel depends only on its own copy: Kernel A on `e1`, Kernel B on `e2`.

* Each copy-back depends only on its own kernel, so the two chains never meet. * The two chains can execute in any order, or concurrently if the device allows.

## Questions

#### Exercise

Code_Exercises/Data_and_Dependencies/source

![SYCL](../../Static/images/diamond_data_flow.svg "SYCL")

Put together what you've seen here to create the above diamond data flow graph using the USM data management model