Rolandthemad, mybe you brought a hostile into town and tipped the scales with the guards when you attacked first.?
Making the first move in town is the quickest way to get into trouble..
Who is he and why is he following you? Retrace your steps and reflect on your adventure- between your journal, map, and experience an answer may be found.
Good luck!