The position in the segment is relative to the start but the offset isn't, so subtract the start from the position when setting the offset. Add unit test for this as well.