Due to how aggregator works, it depends on how buffers are pulled
whether aggregate() is called again or it is waiting for a timeout or EOS:
works:
- pad 1: 4 buffers, pad 2: 4 buffers
- aggregate ready: take all 4/4 buffers
- pad 1: 1 buffers, pad 2: 1 buffer
- aggregate ready: take all 1/1 buffers
waits:
- pad 1: 5 buffers, pad 2: 4 buffers
- aggregate ready: take all 5/4 buffers
- pad 1: 0 buffers, pad 2: 1 buffer
- aggregate not ready: waiting for timeout or EOS
Also don't manually set the clock time as that's unnecessary.
Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/274
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/998>