Otherwise there is a mismatch between the QoS values and what upstream
would expect, leading to too much buffer dropping in video decoders in
case rate < 1.0 or not enough buffer dropping in case rate > 1.0
Adding validate tests with and without decoders.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/679>
