K(ret)probe for TCP time to connect?

I'm working on instrumenting a TCP connections to assist in making load balancing decisions elsewhere. Specifically, I'm looking to get statistics for the time to connect, and notification of failed connects. I initially tried to put kretprobes around inet_stream_connect, but if the socket is in non-blocking mode that notification happens asynchronously. I can't figure out a (good) way to keep track of this data. Any suggestions?