By default, the SQL server Manager Studio(SSMS) uses the following way to connect to a report server instance:
Parse the server name(we typed) to get the ReportServer URL. This step uses the sql
Server Reporting Services(SSRS) WMI provider to get the URL.
Use the got URL to connect to the report server.
There is a little different between the returned URL and the URL you used(http://localhost/ReportServer). The returned URL should be:
http://<machinename>/ReportServer. This URL is same to the URL we can get from the "Web Service URL" in the Reporting Services configuration Manager.
In this case, in order to troublehshoot the issue, please make sure we can open the report server by using
http://<machinename>/ReportServer (from Internet Explorer). If so, we may be able to connect to the report server by using the server name too.
However, if we are not able to connect to the report server, please check the authentication configuration settings. If we are using Kerberos authentication, please make sure the Service Principal Name(SPN)s are registered correctly.
If you have any more questions, please feel free to ask.
Jin Chen - MSFT