# StreamOchestrator

| Consideration           | Status                            |
| ----------------------- | --------------------------------- |
| Preferred import style  | Directory-only _[1]_              |
| Multithread environment | Should be runnable in a WebWorker |

_[1]_ Only the `orchestrator` directory itself should be imported and relied on by the
rest of the code, not its inner files (thus `./index.ts` should export everything that may
be imported by outside code).

## Overview

To be able to play a content, the player has to be able to download chunks of media data -
called segments - and has to push them to media buffers, called `SegmentSinks` in the
RxPlayer code.

In the RxPlayer, the _StreamOrchestrator_ is the entry point for performing all those
tasks.

Basically, the _StreamOrchestrator_:

- dynamically creates various `SegmentSinks` depending on the needs of the given content

- orchestrates segment downloading and "pushing" to allow the content to play in the best
  conditions possible.

## Multiple types handling

More often than not, content are divised into multiple "types": "audio", "video" and
"text" segments, for example. They are often completely distinct in a Manifest and as
such, have to be downloaded and decoded separately.

Each type has its own media buffer. For "audio"/"video" contents, we use regular
browser-defined _MSE_ SourceBuffers. For any other types, such as "text", those buffers
are entirely defined in the code of the RxPlayer.

We then create a different Stream for each type. Each will progressively download and push
content of their respective type to their respective media buffer:

```
- AUDIO STREAM -
|======================    |

- VIDEO STREAM -
|==================        |

- TEXT STREAM -
|========================  |
```

_(the `|` sign delimits the temporal start and end of the buffer linked to a given Stream,
the `=` sign represent a pushed segment in the corresponding buffer)_

## A special case: SourceBuffers

Media buffers created for the audio and/or video types rely on a _native_ (implemented by
the browser) `SourceBuffer` Object implementation.

Media buffers relying on native SourceBuffers have several differences with the other
"custom" (entirely defined in the RxPlayer) types of media buffers:

- They are managed by the browser where custom ones are implemented in JS. As such, they
  must obey to various browser rules, among which:
  1. They cannot be lazily created as the content plays. We have to initialize all of them
     beforehand.

  2. They have to be linked to a specific codec.

  3. The `SourceBuffer` has to be linked to the MediaSource, and they have to the
     MediaSource after the media HTMLElement has been linked to the MediaSource

- They are in a way more "important" than custom ones. If a problem happens with a
  SourceBuffer, we interrupt playback. For a custom media buffer, we can just deactivate
  it for the rest of the content.

  For example, a problem with subtitles would just disable those with a warning. For a
  video problem however, we fail immediately.

- They affect buffering when custom media buffers do not (no text segment for a part of
  the content means we will just not have subtitles, no audio segment means we will
  completely stop, to await them)

- They affect various API linked to the media element in the DOM. Such as
  `HTMLMediaElement.prototype.buffered`. Custom media buffers do not.

Due to these differences, media buffers relying on SourceBuffers are often managed in a
less permissive way than custom ones:

- They will be created at the very start of the content

- An error coming from one of them will lead us to completely stop the content on a fatal
  error

## PeriodStreams

The _DASH_ streaming technology has a concept called _Period_. Simply put, it allows to
set various types of content successively in the same manifest.

For example, let's take a manifest describing a live content with chronologically:

1.  an english TV Show
2.  an old italian film with subtitles
3.  an American film with closed captions.

Example:

```
08h05              09h00                       10h30                now
  |==================|===========================|===================|
        TV Show               Italian Film            American film
```

Those contents are drastically different (they have different languages, the american film
might have more available bitrates than the old italian one).

Moreover, even a library user might want to be able to know when the italian film is
finished, to report about it immediately in a graphical interface.

As such, they have to be considered separately - in a different Period:

```
        Period 1                Period 2                Period 3
08h05              09h00                       10h30                now
  |==================|===========================|===================|
        TV Show               Italian Film            American film
```

In the RxPlayer, we create one _PeriodStream_ per Period **and** per type.

_PeriodStreams_ are automatically created/destroyed during playback. The job of a single
_PeriodStream_ is to process and download optimally the content linked to a single
_Period_ and to a single type:

```
- VIDEO BUFFER -

     PERIOD STREAM
08h05              09h00
  |=========         |
        TV Show


- AUDIO BUFFER -

     PERIOD STREAM
08h05              09h00
  |=============     |
        TV Show


- TEXT BUFFER -

     PERIOD STREAM
08h05              09h00
  |======            |
        TV Show
```

To allow smooth transitions between them, we also might want to preload content defined by
a subsequent _Period_ once we lean towards the end of the content described by the
previous one. Thus, multiple _PeriodStreams_ might be active at the same time:

```
+----------------------------   AUDIO   ----------------------------------+
|                                                                         |
|      PERIOD STREAM 1        PERIOD STREAM 2         PERIOD STREAM 3     |
| 08h05              09h00                       10h30                now |
|   |=============     |=================          |================   |  |
|         TV Show               Italian Film            American film     |
+-------------------------------------------------------------------------+

+------------------------------   VIDEO   --------------------------------+
|                                                                         |
|      PERIOD STREAM 1        PERIOD STREAM 2         PERIOD STREAM 3     |
| 08h05              09h00                       10h30                now |
|   |=====             |===                        |===                |  |
|         TV Show               Italian Film            American film     |
+-------------------------------------------------------------------------+

+------------------------------   TEXT   ---------------------------------+
|                                                                         |
|      PERIOD STREAM 1        PERIOD STREAM 2         PERIOD STREAM 3     |
| 08h05              09h00                       10h30                now |
|     (NO SUBTITLES)   |=========================  |=================  |  |
|         TV Show               Italian Film            American film     |
+-------------------------------------------------------------------------+
```

### Multi-Period management

The creation/destruction of _PeriodStreams_ is actually done in a very precize and optimal
way, which gives a higher priority to immediate content.

To better grasp how it works, let's imagine a regular use-case, with two periods for a
single type of buffer:

---

Let's say that the _PeriodStream_ for the first _Period_ (named P1) is currently actively
downloading segments (the "^" sign is the current position):

```
   P1
|====  |
   ^
```

Once P1 is full (it has no segment left to download):

```
   P1
|======|
   ^
```

We will be able to create a new _PeriodStream_, P2, for the second _Period_:

```
   P1     P2
|======|      |
   ^
```

Which will then also download segments:

```
   P1     P2
|======|==    |
   ^
```

If P1 needs segments again however (e.g. when the bitrate or the language is changed):

```
   P1     P2
|===   |==    |
   ^
```

Then we will destroy P2, to keep it from downloading segments:

```
   P1
|===   |
   ^
```

---

Once P1, goes full again, we re-create P2:

```
   P1     P2
|======|==    |
   ^
```

_Note that we still have the segment pushed to P2 available in the corresponding media
buffer_

When the current position go ahead of a _PeriodStream_ (here ahead of P1):

```
   P1     P2
|======|===   |
        ^
```

This _PeriodStream_ is destroyed to free up ressources:

```
          P2
       |===   |
        ^
```

---

When the current position goes behind the first currently defined _PeriodStream_:

```
          P2
       |===   |
    ^
```

Then we destroy all previous _PeriodStreams_ and [re-]create the one needed:

```
   P1
|======|
    ^
```

In this example, P1 is full (as we already downloaded its segments) so we also can
re-create P2, which will also keep its already-pushed segments:

```
   P1     P2
|======|===   |
    ^
```

---

For multiple types of buffers (example: _audio_ and _video_) the same logic is repeated
(and separated) as many times. An _audio_ _PeriodStream_ will not influence a _video_ one:

```
---------------------------   AUDIO   --------------------------------
     Period 1         Period 2
|================|=============   |
     ^

---------------------------   VIDEO   --------------------------------
     Period 1
|=======         |
     ^

----------------------------   TEXT   --------------------------------
     Period 1         Period 2          Period 3
|================| (NO SUBTITLES) |================   |
     ^
```

At the end, we should only have _PeriodStream[s]_ for consecutive Period[s]:

- The first chronological one is the one currently seen by the user.
- The last chronological one is the only one downloading content.
- In between, we only have full consecutive _PeriodStreams_.

### Communication with the API

Any "Stream" communicates to the API about creations and destructions of _PeriodStreams_
respectively through `"periodStreamReady"` and `"periodStreamCleared"` callbacks.
