the song sounds very nice but as far as the drumming goes i have to agree with all guys that have posted in here. your ideas and musical sentences are promising but there is a very obvious problem with the metronome.
i would humbly suggest starting to work with a click track immediately and lay down the the same groove in different tempos. playing along with favorite songs is another option to make your practice a lot more fun than just playing to a click. i don't consider myself a very rudimental guy, i am an ear taught drummer that never had any courses; but there's something i would like to call shadow drumming (just like in shadow boxing). you basically air drum, pillow drum, x drum with every song you hear even if you don't have sticks with you and you have to pretend that you're holding them and sit behind your setup. that helped me a great deal. drumming starts basically by imitating a pro drummer's playing then you start developing your own style. while doing so, do not neglect the fact that rhythm is based on your solid understanding of this repetitive clicks so called metronome.
whatever the reason may be for your shaky performance(the band, you experience level etc.) some musicians could be oversensitive and over-demanding when it comes down to laying down a decent groove. so it could become a real pain in the butt for a drummer. some of the people i have come across to work with, even hear and complain about 64th note anomaly(a rush or a pull or a slight blunder) in the rhythm (how hard to hear or sense it, basically depends on the tempo of the track) especially during recording sessions. so for instance, one time after several takes and painful hours i could get it right how he wanted, in my first time recording in a studio. made me feel like crap.
if you could have the chance to use an e-drum or triggers with your acoustic set, i would also recommend recording midi tracks and checking out how your notes fit into the desired grid when you examine the midi track. keep on working until every note (lengths, ends etc.) is in it's place as they are desired to be. not before not after.
another tip for using a click track is from a Simon Phillips instructional video. he says if you don't hear the click while playing, everything is fine, but if you start hearing the clicks that has only one meaning, you're out of sync or tempo. practice with a click track until you don't hear there is one. and of course with different tempos.
cheers!
p.s. a sample recording. play along with jewel
Jewel - Only One Too play along