You can certainly ask them to leave...I've done so when a couple of bow hunters were crossing behind my home to track a coyote...but your options are limited if they turn a deaf ear. (Calling the sheriff's office would have been what I'd have had to do had they not complied.)
The danger is allowing things to escalate...and appearing to be the one responsible for things spiraling out of control. Nobody wants to feel powerless over their lives. But these days it seems that you're always within spitting distance of a court appearance.