Sounds all good to me! Especially with the filter. You don't have to worry about his bubble nests unless you want to breed him with some girl
The plants are actually bothering him, not sure how much though, but it's been proven (by a member on the forum) that fish prefer natural environments. But you have to consider the fact that you barely introduced him, it's very traumatic for fish to move, especially if the water quality is different besides the temperature.
You are adding water conditioners to the water, right? Since you kept goldfish before, you probably know how important that is
Aside from that, I have nothing much to add, feed him once a day, and add some treats every once in a while! Bettas love blood worms, I've got freeze-dred and frozen ones, and Bruce goes nuts when I give him some
Remember to feed a tiny, tiny amount! Bettas are a lot smaller than goldfish and won't eat as much, and you don't want any left over foods on the bottom