Do the character's base stats ever change? I have 2 characters at lvl 3 and 4 at lvl 2 and none of them had any stat changes at level up. How does this work?

Baldur's Gate is not Diablo: You do not get to raise your character's stats each time you Level Up. Instead, you get more hitpoints, weapon skills, access to spells, increased Thieving abilities, etc.

Your stats are not meant to be fluid and ever-changing: They are pretty much set in stone, as they describe the strengths and limitations that we all have. There ARE ways to change your stats, but they are few and far between, so count yourself lucky if you find something that proves beneficial. (Notice that I didn't say the changes HAVE to be beneficial.)