## Asynchronous Execution

## Learning Objectives * Learn about how commands are enqueued asynchronously * Learn about the different reasons for synchronization * Learn about the different ways to perform synchronization

#### Asynchronous execution

* All command submitted to a `queue` are done so asynchronously. * The functions return immediately and the command is run in a background thread. * This includes individual commands like `memcpy` and collections of commands derived from a command group. * This means you have to synchronize with those commands.

#### Synchronization

There are a number of reasons why you need to synchronize with commands

* Await completion of a kernel function. * Await the results of a computation. * Await error conditions which come from a failure to execute any of the commands.

#### Synchronization with kernel functions

There are two ways ways to synchronize with kernel functions.

* Calling `wait` on an `event` object returned from enqueuing a kernel function command, either via a command group or a shortcut function. * Calling `wait` or `wait_and_throw` on the `queue` itself.

#### Synchronizing with kernel functions


#include<sycl/sycl.hpp>
#include<vector>

using namespace sycl;

int main(){
  std::vector<int> hostA = { 1, 5, 66, 14, 55, 11, 12 }; 
  queue gpuQueue(gpu_selector_v);

  int* devA = malloc_device<int>(hostA.size(), gpuQueue);

  auto e = gpuQueue.memcpy(devA, hostA.data(), hostA.size());
  e.wait();

  gpuQueue.parallel_for(hostA.size(), [=](id<1> idx){
    devA[idx]++;
  }).wait();
}

* Calling `wait` on an `event` object returned from enqueuing a command group will wait for the commands from that command group to complete. * This is how we have synchronized in our examples so far. * This effectively creates a blocking operations that will complete in place by immediately synchronizing.

#### Synchronizing with kernel functions


#include<sycl/sycl.hpp>
#include<vector>

using namespace sycl;

int main(){ 
  std::vector<int> hostA = { 1, 5, 66, 14, 55, 11, 12 };
  queue gpuQueue(gpu_selector_v);

  int* devA = malloc_device<int>(hostA.size(), gpuQueue));

  gpuQueue.memcpy(devA, hostA.data(), hostA.size());

  gpuQueue.wait();

  gpuQueue.parallel_for(hostA.size(), [=](id<1> idx){
    devA[idx]++;
  });

  gpuQueue.wait();
}

* Again calling `wait` or `wait_and_throw` on a `queue` will wait for all commands enqueued to it to complete. * Note you generally don't want to call `wait` on the `queue` after every command, instead you want to create dependencies between commands, which we cover in the next lecture.

#### Synchronizing with data

There are multiple ways ways to synchronize with data, but it differs depending on the data management model you are using.

* you can synchronize the same way you would for kernel functions, calling `wait` on an `event` or the `queue`.

#### Synchronizing with data


gpuQueue.memcpy(data, devicePtr, sizeof(int)).wait();


gpuQueue.memcpy(data, devicePtr, sizeof(int));
gpuQueue.wait();

* Simply call `wait` on the `event` returned from `memcpy`. * Alternatively call `wait` on the `queue`.

#### Synchronizing with errors

* Errors are handled by a `queue` and any asynchronous errors can be produced during any of the synchronization methods we've looked at. * The best way to ensure all errors are caught is to synchronize by calling `wait` or `wait_and_throw` on the the `queue`.

## Questions

#### Exercise

Code_Exercises/Asynchronous_Execution/source

Try out the different methods of synchronizing with a kernel function and the resulting data from the computation.