high-level tcp bindings for std

[olsonjeffery]

quick summary

This pull req includes high-level TCP/IP bindings for the rust stdlib (both server and client API), under the std::net::tcp module. I also did some reshuffling of the IP stuff and pushed it into the net::ip module.

also: adding result::unwrap (from a patch from @nmatsakis) and ignore'd a test on std::timer that is starting to fail more and more, as perf chokes due to valgrind and threading behavior (the test is time-sensitive, so probably poorly conceived to begin with)

only slightly more in-depth

An example demonstrating the TCP client/request API can be found here.

Interestingly, because of API-style friction between libuv and rust, I found it neccesary to expose two separate UIs to the TCP server API. Quickly, with examples, they are:

• the net::tcp::listen_for_conn entry point, an example of which is here. This call is blocking for the lifetime of the TCP server connection and uses a callback running on the libuv loop (!).
• and the net::tcp::new_listener entry point, an example of which can be found here. This bit of API, on successful setup, returns a (vaguely) comm::port-like object that uses can peek/recv on (using custom API), as needed, to get new connections.

The new_listener API varies, subtly, from the listen_for_conn entry point because it accepts new connections immediately (with the overhead that this entails), while listen_for_conn gives a user the chance to manually accept a connection, or just drop it, in line with the default behavior of libuv.

To more fully articulate this, I'm just going to paste my commit msg from 95f0b90:

• we now have two interfaces for the TCP/IP server/listener workflow,
  based on different user approaches surrounding how to deal with the
  flow of accept a new tcp connection:

1. the "original" API closely mimics the low-level libuv API, in that we
   have an on_connect_cb that the user provides that is ran on the libuv
   thread. In this callback, the user can accept() a connection, turning it
   into a tcp_socket.. of course, before accepting, they have the option
   of passing it to a new task, provided they make the cb block until
   the accept is done .. this is because, in libuv, you have to do the
   uv_accept call in the span of that on_connect_cb callback that gets fired
   when a new connection comes in. thems the breaks..

I wanted to just get rid of this API, because the general proposition of
users always running code on the libuv thread sounds like an invitation
for many future headaches. the API restriction to have to choose to
immediately accept a connection (and allow the user to block libuv as
needed) isn't too bad for power users who could conceive of circumstances
where they would drop an incoming TCP connection and know what they're
doing, in general.

but as a general API, I thought this was a bit cumbersome, so I ended up
devising..

1. an API that is initiated with a call to net::tcp::new_listener() ..
   has a similar signature to net::tcp::listen(), except that is just
   returns an object that sort of behaves like a comm::port. Users can
   block on the tcp_conn_port to receive new connections, either in the
   current task or in a new task, depending on which API route they take
   (net::tcp::conn_recv or net::tcp::conn_recv_spawn respectively).. there
   is also a net::tcp::conn_peek function that will do a peek on the
   underlying port to see if there are pending connections.

The main difference, with this API, is that the low-level libuv glue is
going to accept every connection attempt, along with the overhead that
that brings. But, this is a much more hassle-free API for 95% of use
cases and will probably be the one that most users will want to reach for.

what's needed to fill this out

1. IPv6! I stubbed out the data structure, but parsing IPv6 addr strings is non-trivial, so I'm not sure how to tackle this from rust, just yet. If anyone wants to pitch in, that'd be swell.
2. The other bits of the tcp API. I only added connection-establishment for clients/server and read/write over TCP streams.. so stuff like setting keep-alive, etc needs to be added.

[pcwalton]

How about if here?

[brson]

How safe is conn_recv_spawn? Do you have to ensure the server_port outlives the task handling the connection?

[olsonjeffery]

@brson ok, so once the new_conn_po in conn_recv_spawn recvs, we have a valid stream for the client. We then pass that valid stream into a new task, where a tcp_socket is created and is then passed to the user-supplied callback. At this point, conn_recv_spawn returns.. So if the server_port goes out of scope and tears down during/before the body of the callback is ran, then operations against the client stream (contained in the tcp_socket) will error out.

Ideally, the user would keep the server_port bottled up in a loop or somesuch, so this wouldn't be an issue. But if it does happen, it won't be a catostrophic failure/segfault, since the stream contained in the tcp_socket is a different one from the stream contained in the server_port/tcp_conn_port (the client stream is created in tcp_nl_on_connection_cb L884, with the call to malloc_uv_tcp_t)

[brson]

Merged!

Thanks for this huge amount of work and for writing such of thorough description in the pull request.

I've read a fair bit of the code now and I am happy with the direction we're going. The overall design seems to get a little clearer every iteration. I have yet to take a close look at the final tcp interface, though I do intend to. Integrating it with servo might help me get a better feel for where we are - this week I will try to do some experiments with it.

Some miscellaneous things we should consider for the future:

• We need to figure out the right abstractions to make the high level APIs memory-safe. Atomic ref counted types will help.
• The safe APIs need clearly defined failure modes with accompanying tests. Using the high-level API it ultimately shouldn't be possible to break the uv loop, even when any client tasks fail.
• At some point we need to think about how rust's I/O API's (streams in particular) should look and how uv fits in.

I am going to leave this pull request open a while longer because I want to come back to it again.

[olsonjeffery]

thanks for fixing timer, with the send/copy kind change. I did a fix last night, but was running tests overnight, so you beat me to it.

100% agreement on all points. we're getting there.