I know someone out there has done this - you've got your primer on the cinder block wall - it's white, which is all you want - so you don't bother with paint. Any regrets?
I used Olympic's pore-filling masonry primer, mainly because it was only $10/gal. I know primer isn't as tough as paint...