Another thing that works in every other browser but does not work in Internet Explorer.
So my solution is to set the MIME type for the .vtt files. This can be done under the Site administration / Server / File types (Moodle >= 2.9). Click on “Add a new file type” button. Set Extension to vtt, set MIME type to text/vtt then click “Save changes”.
You have to re-upload your .vtt files to have the changes take effect.