Plex does not support the use of ISO, IMG, Video_TS, BDMV, or other "disk image" formats. If you wish to use those with Plex, you should convert them to a compatible format.
There are two broad strategies for converting disk image formats into something much more compatible:
- Remux the content. Remuxing is quick and retains the full quality of the original source, but the resulting files can be quite large (anywhere from 2GB to 50GB, depending on the source).
- Transcode the content. Transcoding can take quite a bit of time to process (anywhere up to several times the source content duration), but should result in a much smaller file with virtually no visually-noticeable loss of quality if done well. Resulting files might be anywhere from 500MB to 15GB.
There is no single correct choice here. You'll need to use what makes sense for you in your particular situation. If you have a lot of storage space, remuxing is probably easiest. If you're tight on space but have a powerful computer and some time, transcoding can be great.
Remux the Content
One of the easiest ways to remux your content is to use the MakeMKV application. You can use this to extract the audio, video, and subtitle tracks from your source and them remux them into a standard MKV container. MakeMKV is available for Windows and OS X.
Transcode the Content
There are lots of applications available to convert and transcode content. Handbrake is a free tool that works well in this situation. Handbrake is available for Windows, OS X, and Linux.
You can also see the ffmpeg info below for additional suggestions that can be used.
Don Melton's Transcode Tools
Don Melton has some useful tools available, which are based on other popular open source tools. This is generally better for more advanced users, as running scripts like this can be a bit complicated sometimes. It can be used to transcode "into a smaller, more portable format while remaining high enough quality to be mistaken for the originals".
Related Page: Video Transcoding Tools by Don Melton
You can use FFmpeg from the command line to encode content. Some useful output arguments to use:
-preset veryslow(unless you simply aren't willing to wait longer)
-crf 18(18 is a good default to use; you can use 16 for slightly higher quality at the expense of time/size)
Example command line:
ffmpeg -i inputfile.mkv -crf 18 -map 0 -acodec copy -scodec copy -c:v libx264 -threads 0 -preset veryslow outputfile.mkv
This would preserve audio and subtitle tracks as well as use libx264 to encode with a CRF of 18. The encoding may take a while to complete, but it should give you very good quality at reasonable filesize.