Does the computer have to stay running?

Yes.

In general, for your Plex Apps to be able to access your content, they have to be able to connect with your Plex Media Server. That means that the computer or device with your Server needs to be powered on and that the Server needs to be running.

If you're trying to access your content remotely (e.g. using your phone when you're not at home), the Server will need to be connected to the internet and successfully signed in and connected to Plex.

Related Page: Plex Media Server > Server Settings > Connect