It still shouldn't be that hard to do, as once the holesaw has 2 points to drill between it should stay straight enough to continue. I did mine freehand, with the pilot bit protruding only 3/8 - 1/4" past the end of the holesaw itself and it stayed true. Just take your time and remember, a holesaw can jerk the drill right out of your hand (if you're using a hand drill) if it binds, so keep it as straight as possible. It did, however, take quite some time to drill through it.
Everyone says that the exhaust manifold is cast iron, but I believe it's properties to be more along the lines of cast steel. If you grind on it, it turns shiny like steel, not black like iron. If you weld on it, it takes the weld just like steel. I'm not totally convinced that it is in fact iron, but that's just from my experiences. Your (and others') mileage may vary.