To remove an agency from your organization's Community simply take the following steps:

You will be able to find the community agencies that have submitted candidates in the last 180 days on the Your Community page under the agencies section of the site. From there simply find the agency you want to upgrade and click on the action menu to the right of that agency recruiter and select Block from Community.

Note: Blocking an agency not only removes the individual recruiter but their whole team