You can remove an agency recruiter from a JobCast, by going to the "Invited Agencies" page of the JobCast you wish to remove them from.
Then find the agency recruiter you want to remove and click on the action menu of that record, then select "Remove from JobCast".
Additionally, you can search for the agency recruiter and remove them from the JobCasts tab on their agency card.
And finally, if you want to prevent an agency from receiving your JobCasts in the future you can Remove the Agency from your community, for more information on that check out this FAQ article.