First I will note that running multiple cassandra servers on a single host is not recommended. If it is necessary it is often easier to use virtualization to simulate multiple servers. What you are looking to do is possible though.
You will need to run multiple agents on the Cassandra servers, yes. You can install multiple agents when using the manual agent installation. You should follow the process described here:
You will need to install each agent to a different location on the machine. You will also need to make a manual change to the conf/address.yaml file. You should add the following lines to each file:
The <listen_address> above should be replaced with the configured listen_address from your cassandra.yaml depending on which cassandra instance the agent should monitor. Additionally each cassandra will need to use a different jmx port in order to run on the same host. You should use the correct port accordingly.