You are liable. He had the right of way and you have to yield, and make a turn when it is safe to do so. Even if he was speeding you are not going to be able to prove it. He reacted to your "unsafe" turn.
Ex. You are driving and someone swerves into your lane and forces you off to the side of the road where you proceed to crash. The offending persons actions caused you to swerve right? Basically if he responded the way a "normal" driver would have responded, you will be found liable do to a failure to yield to oncoming traffic...
Remember if he were to file a civil action he would only have to prove the preponderance ex. 51% to 49% that what took place was what he described, which sounds logical...and he may ask for filing and court fees as well...