Make sure that we a) don't free structures of caps as we don't own them
and b) make sure to create a copy of all still referenced structures
before it gets removed, the caps are unreffed or something else happens.
This still needs to be fixed for gst_caps_do_simplify()!
This workaround is inspired by the one in gst-python.