Under SDA, yes. But under trespass, no. Does a farmer have to ask you to leave if your hunting or fishing without permission? no, he just calls the cops or the game warden.
I think it just depends under which law they want to use to prosecute with.
Along with Mr. Brown says about trespass, the issue on hunting and fishing is only if it is posted. You can hunt and fish on any non posted land except primary residences and land with farm or ranch structures. Up until August of 09 it was any unposted land. The owner of said un posted land has the right to ask you to leave. If he does so in the presence of law enforcement any return by you will result in a trespassing citation.