The server is getting very laggy. Yesterday was painful when trying to build, just kept clicking and nothing happened. The Ping time was well up to 20+ seconds as well and it certainly was not from my connection (I checked).
If the server is hosted on a VM or as some kind of Cloud service in Azure, GCP or AWS then I would advice looking at chucking some extra resource at it to avoid the lag. As it's lagging when only I have been on the server i doubt it's a bandwidth issue (unless there is one at your host providers side) and so that would suggest it's a processing issue, so worth checking if your spiking your CPU or Memory on the box.
Hope this is helpful for your trouble shooting. As a Senior Solutions Architect and a Cloud and Integrations industry expert I am happy to provide you with infrastructure (virtual or physical) advice to reduce these issues if you require. Feel free to ask.
Adam AKA TecSatan