Wait, you are bolting aluminum to carbon with stainless steel and no one has a problem with that?
Jake has made it pretty clear that stainless and carbon react. In spite of that, it is a common way to add fittings to a carbon mast or whatever. Lightly rusted stainless seems to retain most of it's strength.
Adding aluminum to the equation makes a bomb of corrosion in my experience, and should be avoided. perhaps i have mis-read the OP.
Mainly, since i don't speak Australian, this post is to find out what a ring spanner is?
Also if you glue a washer down with epoxy it won't be stuck very well and it would be hard to keep the glue out of the hole in the washer. After it dries you won't be able to slide the rivet in. Trying to drill the dry glue out of the hole will prolly pop the washer loose.