Various improvements to both Gstreamer and clutter-gst 3.0 have been incorporated in Apertis and upstream.

Multimedia buffering strategies

For playback of network streams typically some buffering has to be used to ensure playback without any pauses, stuttering or other glitches. Broadly speaking two types of buffering strategies are used, both of which serve slightly different use-cases:

  • Stream buffering
  • Progressive buffering

Progressive buffering

Progressive buffering (download buffering in the clutter API) is the buffering strategy most people are used to from e.g. youtube. Using such a strategy the network media file is temporarily stored locally and playback starts when it is expected the media can be played back without a need to pause for further buffering. Or in other words, playback starts when the remaining time to finish the download is less then the playback time of the media.

To enable progressive buffering using clutter-gst one needs to set the buffering mode to download mode:

clutter_gst_playback_set_buffering_mode (player, CLUTTER_GST_BUFFERING_MODE_DOWNLOAD);

In case the uri isn’t suitable for download buffering Gstreamer will automatically fall back to stream buffering

Stream buffering

In the case of live streams (e.g. radio streams, live video feeds etc), progressive buffering can’t be used as there simply no one media file which can be partially downloaded. In this case, buffering is purely done to smooth out jitter in the download speed, however on average the users bandwidth must be enough to download in real time.

For stream buffering adapting to current bandwidth situation of mobile users can be done by using an adaptive bitrate streaming technoglody such as HLS or MPEG-DASH.

Downloading based on available bandwidth

By default when doing buffering, Gstreamer bases the available bandwidth measurement purely on the observed incoming rate. The clutter-gst implementation of download buffering uses this remaining time estimate to decides its policy. In case expected bandwidth can be determined from other sources (e.g. the navigation system knows the bandwidth estimates along the planned route), it’s possible for either an application or an specialized subclass of ClutterGstPlayback to override this behaviour.

For overriding the default policy an application can either use a subclass which overrides the should_buffer class method from ClutterGstPlayback or the application can listen to the should-buffer signal and return whether buffering should continue or not. To do so the application should first connect to the signal (playbing being a ClutterGstPlayback):

g_signal_connect (player, "should-buffer", G_CALLBACK (player_should_buffer), app);

And implement the callback using the applications intended buffering policy (the example below stops buffering when the remaining playback time is less then 110% of the remaining download time):

static gboolean
player_should_buffer (ClutterGstPlayback *self, GstQuery *query)
{
  ClutterGstPlaybackPrivate *priv = self->priv;
  gdouble position;
  gdouble duration;
  gboolean ret = FALSE;
  gint64 left;
  gdouble time_left;
  gboolean busy;


  /* Use the estimated total duration left as estimated by queue2 based on the
   * averge incoming bitrate, we can stop buffering once the remaining download
   * takes less time then the remaining play time (with a 10% safety margin).
   * However regardless of that keep buffering as long as queue2 indicates that
   * buffering should happen (based on its high water marks */
  gst_query_parse_buffering_range (query, NULL, NULL, NULL, &left);
  gst_query_parse_buffering_percent (query, &busy, NULL);

  position = clutter_gst_playback_get_position (self);
  duration = clutter_gst_playback_get_duration (self);
  if (duration)
    time_left = duration - position;
  else
    time_left = 0;

  if (left == -1 || (!busy && (((gdouble)left * 1.1) / 1000) <= time_left)) {
    ret = FALSE;
  } else {
    ret = TRUE;
  }
  g_debug ("Application should buffer says: %d", ret);

  return ret;