Some code incorrectly relied on GstVideoDecoder/Encoder base class implementation details. Follow-up to !3695. Fixes #1742. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3783>