pnmenc: Fix wrong logic leading to memory mishandling

While encoding the frame in ASCII mode, per component four bytes are needed
and after every 20 bytes, a \n will be added. So the calculation should be
size = size * (4 + 1 / 20). This should exclude the header being written.
Since header is also being included in the calculations, memory mishandlings
are happening.

https://bugzilla.gnome.org/show_bug.cgi?id=759520
This commit is contained in:
Vineeth TM 2015-12-17 08:51:48 +09:00 committed by Sebastian Dröge
parent 4d88848fae
commit 10ed707b60

View file

@ -202,10 +202,10 @@ gst_pnmenc_handle_frame (GstVideoEncoder * encoder, GstVideoCodecFrame * frame)
if (pnmenc->info.encoding == GST_PNM_ENCODING_ASCII) {
/* Per component 4 bytes are used in case of ASCII encoding */
size = size * 4;
size = size * 4 + size / 20;
size += strlen (header);
frame->output_buffer =
gst_video_encoder_allocate_output_buffer (encoder, (size + size / 20));
gst_video_encoder_allocate_output_buffer (encoder, (size));
} else {
size += strlen (header);
frame->output_buffer =