When you initially access a media item, your server will do a quick "on-demand analysis" of the item if it hasn't already been done recently. That's done to make sure streams inside the file haven't changed, to pick up sidecar subtitles, to pull in new media information that might not have been collected with an earlier PMS version, etc. The analysis should generally only take a few seconds itself.
If your content is on a hard drive that's gone to sleep, you'll also have to wait for the drive to spin up and be accessible before the analysis can take place.
So, in most cases: HDD spinup + media analysis = delay
Plex Pass subscribers can make use of the Scheduled Tasks feature to have analysis run during maintenance periods to help alleviate the need for on-demand analysis in most cases.
Related Page: Plex Media Server > Server Settings > Scheduled Tasks