Skip to content

Add IPv4 and IPv6 multicast implementation for UDPSocket #5196

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 20, 2017

Conversation

kegilbert
Copy link
Contributor

Continuation of initial pull request that only added IPv4 support. Extensive rework largely took place in lwip_stack.c to add IPv4/IPv6 logic.

Tested by enabling IPv6 in mbed_lib and mbed_app.json using the following test script on two K64Fs hooked up to a common router, one acting as a broadcast node and the other as a receiver.

#include "mbed.h"
#include "EthernetInterface.h"

const char* MCAST_GRP = "FF02::1";
//const char* MCAST_GRP = "229.0.0.1";
const int MCAST_PORT = 5007;

#ifdef SEND
int main() {
    DigitalOut led(LED2);
    EthernetInterface eth;
    eth.connect();
    
    UDPSocket sock;
    sock.open(&eth);
    
    SocketAddress multicast_group;
    multicast_group.set_ip_address(MCAST_GRP);
    multicast_group.set_port(MCAST_PORT);
    
    char out_buffer[] = "very important data";
    while (true) {
        printf("Multicast to group: %s\r\n", MCAST_GRP);
        sock.sendto(multicast_group, out_buffer, sizeof(out_buffer));
        led = !led;
        Thread::wait(1000);
    }
}
#else
DigitalOut led(LED3);

void beat(void) {
    led = !led;
}

int main() {
    printf("Running recv...\r\n");
    Ticker heart;
    heart.attach(&beat, 0.25f);
    EthernetInterface eth;
    eth.connect();
    
    UDPSocket server;
    server.open(&eth);
    server.bind(MCAST_PORT);
    int ret = server.join_multicast_group(MCAST_GRP);
    if (ret != 0) {
        printf("Error joining the multicast group, error: %d\r\n", ret);
        while (true) {}
    }
   
    SocketAddress client;
    char buffer[256];
    while (true) {
        printf("\r\nWait for packet...\r\n");
        int n = server.recvfrom(&client, buffer, sizeof(buffer));
        printf("Packet from \"%s\": %s\r\n", client.get_ip_address(), buffer);
    }
}
#endif

ipv4-multicast-1
ipv6-multicast-1

@kjbracey-arm Thoughts on this implementation?

@theotherjimmy @geky

@tommikas
Copy link
Contributor

Failing to compile with IAR:

23:49:50 [K64F_6LOWPAN IAR] [Warning] lwip_stack.c@298,18: [Pe111]: statement is unreachable
23:49:50 [K64F_6LOWPAN IAR] [Warning] lwip_stack.c@663,18: [Pe174]: expression has no effect
23:49:50 [K64F_6LOWPAN IAR] [Warning] lwip_stack.c@691,37: [Pe174]: expression has no effect
23:49:50 [K64F_6LOWPAN IAR] [Error] lwip_stack.c@1134,25: [Pe144]: a value of type "void const *" cannot be used to initialize an entity of type "nsapi_ip_mreq_t *"
23:49:50 [K64F_6LOWPAN IAR] [Warning] lwip_stack.c@1158,0: [Pe069]: integer conversion resulted in truncation
```

Copy link
Contributor

@kjbracey kjbracey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty good now. My only remaining concern is: what happens when the socket is closed?

In a conventional stack, one of the main reasons for making this a socket option, as opposed to some non-socket API, is to track ownership of the joins, giving a clean-up guarantee.

When the application closes the socket, or the application dies, the joins made on that socket are cancelled.

So, the question is - do we define that as part of NSAPI, in which case this LWIP adaptation layer will need to keep its own socket membership lists for cleanup on close, as lwIP doesn't?

Or do we specify that behaviour is undefined if a socket is closed with group joins in force, and the application is required to leave before closing?

My preference would be to define cleanup - the undefined behaviour we already have between stacks causes enough problems, and here we have it in our power to implement it in the adaptation layer.

On the other hand - how much do applications come and go on mbed systems - is it worth devoting RAM+ROM space to clean-up that may never happen?

(Nanostack does implement this as a true socket option internally, so does its own clean-up on close)

return NSAPI_ERROR_UNSUPPORTED;
}
err_t igmp_err;
nsapi_ip_mreq_t *imr = optval;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing const on imr.

}
#endif

return igmp_err;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to do error conversion from lwip to NSAPI here. I guess you could then initialise igmp_err to ERR_VAL to get NSAPI_ERROR_PARAMETER - no err_t currently maps to NSAPI_ERROR_UNSUPPORTED.

Seems reasonable - I kind of like distinguishing "I don't know this option" from "I don't like what you specified for this option". I'd argue all existing size checks returning UNSUPOPRTED should be PARAMETER too.

* @param address Multicast group IP address
* @return Negative error code on failure
*/
int join_multicast_group(const SocketAddress &address);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wondering if this should be in Socket. This is falling into a trap much of the rest of the class does in that stuff ends up in UDPSocket only because it's "not TCP". Other types of socket like raw or ICMP will want this API, and it's calling an IP-level socket option.

(And Socket is currently "IP socket", rather than generic socket)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved it over to Socket. Added an overload in TCPSocket to return an error for TCPSocket objects, let me know if you'd rather remove that altogether or had a better implementation.

Copy link
Contributor

@kjbracey kjbracey Sep 27, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it is particularly important to block it on a TCP socket - it's weird but harmless. On some stacks it does work - it does make you join the group regardless. The TCP socket wouldn't process those packets itself, as it never processes multicasts, but other sockets and pings would start responding.

But if you do want to block it, it logically needs to be in setsockopt, looking at s->conn->type like the TCP-specific options do.

I guess blocking it might help consistency - if some stacks block it maybe it's good if all stacks do.

@kjbracey
Copy link
Contributor

That FF02::1 test isn't very good - that's "all hosts", so any IPv6 node should already be a member of it without you adding it.

May be worth checking you're seeing the correct IGMP/MLD messages being sent.

@kegilbert
Copy link
Contributor Author

Updated the branch to fix the IAR issue and move the implementation to the Socket class. Also successfully retested on IPv6 with FF05::208 as an unassigned address.

return NSAPI_ERROR_PARAMETER;
}
err_t igmp_err;
const nsapi_ip_mreq_t *imr = (const nsapi_ip_mreq_t*) optval;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unnecessary C-style cast is unhelpful - would have hidden the const-correctness error previously reported.

ip_addr_set_any(IP_IS_V6(&if_addr), &if_addr);
}

igmp_err = NSAPI_ERROR_UNSUPPORTED;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be an lwIP error value, as you're going to convert it.

@kegilbert
Copy link
Contributor Author

Addressed @kjbracey-arm's comments and added some basic tracking of joined multicast addresses in the Socket class. On Socket close, all joined multicast addresses are left before closing to create a defined behavior.

@@ -83,11 +94,39 @@ int Socket::modify_multicast_group(const SocketAddress &address, nsapi_socket_op

int Socket::join_multicast_group(const SocketAddress &address)
{
if(_multicast_addr_count == SOCKET_MAX_MULTICAST_MEMBERSHIPS) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the wrong layer - it's in the "friendly veneer" rather than the core socket option. It means direct use of the socket options won't track.

while((_multicast_addr_registry & (0x0001 << index))) { index++; }

// Create copy of socketaddress to prevent dead pointers when reading back during leave group
_socket_multicast_register[index] = SocketAddress(address);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be a copy-assignment anyway with just = address, wouldn't it?

Regardless, I think you would need to be storing the actual multicast request structures, not just the socket address, as you want to track memberships per-interface.

You might want to check for existing matches - add group normally refuses if you've already joined on the same socket. And cancel the entry if the underlying join fails?

@@ -225,6 +228,9 @@ class Socket {
mbed::Callback<void()> _event;
mbed::Callback<void()> _callback;
rtos::Mutex _lock;
uint32_t _multicast_addr_count;
uint32_t _multicast_addr_registry;
SocketAddress _socket_multicast_register[SOCKET_MAX_MULTICAST_MEMBERSHIPS];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An array feels a little memory inefficient, especially if it was actually the multicast request with both group and interface address. Most sockets won't be doing any joins at all.

Maybe have these fields in a malloced sublock you allocate on first join? Although I know some disfavour dynamic allocation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, I wasn't sure what the stance on dynamic memory allocation was compared to static allocation in mbed generally so went with this option. Completely open to changing it.

@kegilbert
Copy link
Contributor Author

Moved the socket membership logging to the lwip_stack layer. Switched to dynamic memory allocation on first multicast address add to avoid extra memory on every Socket object that does not use multicast.
@kjbracey-arm

Copy link
Contributor

@kjbracey kjbracey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you need to update your array first, then back out in the case of an error. In the case of an add error, you just clear your bit, the add didn't stick.

In the case of a remove error, you just remove anyway. I think if you get an error on leave, that probably means you somehow weren't a member anyway.

You should precheck if the address is in the array whether or not its an add or remove. Prechecking for add avoids pointless duplicates, and checking for remove stops you removing another socket's membership.

Something like

    index = find_multicast_member()
    if (add) {
        if (index != -1) {
             return error
        }
        copy to table, set bit
        do lwip join
        if lwip call failed, clear bit
    } else {
       if (index == -1) {
           return error;
       }
       clear bit regardless
       do lwip leave
    }

The error on the add should be the equivalent of EADDRINUSE, but doesn't seem to exist in nsapi_types.h. Maybe it should be added. It's the sort of thing we might want to return from bind...

.imr_multiaddr = s->multicast_memberships[index].imr_multiaddr,
.imr_interface = s->multicast_memberships[index].imr_interface
};
mbed_lwip_setsockopt(0, s, 0, NSAPI_DROP_MEMBERSHIP, &address, sizeof(address));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could just pass &s->multicast_memberships[index], no need for the temporary.

if(igmp_err == ERR_OK) {
if(!s->multicast_memberships) {
// First multicast join on this socket, allocate space for membership tracking
s->multicast_memberships = malloc(sizeof(nsapi_ip_mreq_t) * LWIP_SOCKET_MAX_MEMBERSHIPS);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to check for memory allocation failure here.

next_free_multicast_member(s->multicast_memberships_registery, index);

s->multicast_memberships[index].imr_multiaddr = imr->imr_multiaddr;
s->multicast_memberships[index].imr_interface = imr->imr_interface;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s->multicast_memberships[index] = *imr; would do.

next_registered_multicast_member(s->multicast_memberships_registery, index);

bool address_matched = true;
for(int i = 0; i < NSAPI_IP_BYTES; i++) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd be a bit less literal about this; rather than a manual loop, address_matched is just

memcmp(&s->multicast_memberships[index].imr_multiaddr, &imr->imr_multiaddr, sizeof(nsapi_addr_t)) == 0 &&
memcmp(&s->multicast_memberships[index].imr_interface, &imr->imr_interface, sizeof(nsapi_addr_t)) == 0

You could just memcmp the whole structure, if you assumed no padding, which I think is probably fine.

// First multicast join on this socket, allocate space for membership tracking
s->multicast_memberships = malloc(sizeof(nsapi_ip_mreq_t) * LWIP_SOCKET_MAX_MEMBERSHIPS);
} else if(s->multicast_memberships_count == LWIP_SOCKET_MAX_MEMBERSHIPS) {
return ERR_BUF;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If failing here, you've already joined, and the reference counting is borked. Maybe you should check for space first and fill it in.

Then clear the bit if the actual call to lwIP fails.

@kegilbert
Copy link
Contributor Author

Reordered the registry updates and added in the additional allocation checks along with some minor cleanup.

Copy link
Contributor

@kjbracey kjbracey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is basically there. Just style nits now.

return NSAPI_ERROR_NO_ADDRESS;
}

clear_multicast_member_registry_bit(s->multicast_memberships_registery, find_multicast_member(s, imr));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

member_pair_index - don't need to find again.

// First multicast join on this socket, allocate space for membership tracking
s->multicast_memberships = malloc(sizeof(nsapi_ip_mreq_t) * LWIP_SOCKET_MAX_MEMBERSHIPS);
if(!s->multicast_memberships) {
return ERR_BUF;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're mixing error codes types again - you should be returning NSAPI errors

igmp_err = ERR_USE; // Maps to NSAPI_ERROR_UNSUPPORTED
int32_t member_pair_index = find_multicast_member(s, imr);

if(optname == NSAPI_ADD_MEMBERSHIP) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style - if (. Applies in a lot of the most recent change

// Track multicast addresses subscribed to by this socket
nsapi_ip_mreq_t *multicast_memberships;
uint32_t multicast_memberships_count;
uint32_t multicast_memberships_registery;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spelling - registry

nsapi_ip_mreq_t *multicast_memberships;
uint32_t multicast_memberships_count;
uint32_t multicast_memberships_registery;
} lwip_arena[MEMP_NUM_NETCONN] = {0};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

={0} is meaningless. Static data is always default-initialised to zero.

static void mbed_lwip_arena_dealloc(struct lwip_socket *s)
{
s->in_use = false;

while(s->multicast_memberships_count > 0) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style - while (

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tidy still needed

@@ -1080,6 +1108,24 @@ static nsapi_size_or_error_t mbed_lwip_socket_recvfrom(nsapi_stack_t *stack, nsa
return recv;
}

int32_t find_multicast_member(struct lwip_socket *s, const nsapi_ip_mreq_t *imr) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be static. s could be const.

#define LWIP_SOCKET_MAX_MEMBERSHIPS 4
#endif

#define next_registered_multicast_member(registry, index) while(!(registry & (0x0001 << index))) { index++; }
Copy link
Contributor

@kjbracey kjbracey Oct 5, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Magic macros seem a bit tricksy, and not gaining much here.

Maybe avoid confusion by making them static inline functions that return an updated index, rather than modifying index by reference?

We assume C99 or later, so static inline is portable (and works the same as in C++03).

@kjbracey
Copy link
Contributor

kjbracey commented Oct 5, 2017

As it's nearly there I'd like more review from @geky, @c1728p9, @SeppoTakalo and @mikaleppanen.

@mikaleppanen
Copy link

Looks good to me.

@kegilbert kegilbert force-pushed the multicast-ipv6-2 branch 2 times, most recently from 46e02df to 0c48e8b Compare October 9, 2017 23:10
@kegilbert
Copy link
Contributor Author

Updated style and moved from macros to static inline functions

@mbed-ci
Copy link

mbed-ci commented Oct 9, 2017

Build : SUCCESS

Build number : 31
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/5196/

Triggering tests

/test mbed-os

@mbed-ci
Copy link

mbed-ci commented Oct 10, 2017

@kegilbert
Copy link
Contributor Author

/morph test

@theotherjimmy
Copy link
Contributor

@kegilbert That was not a morph test. :D

@theotherjimmy
Copy link
Contributor

/test mbed-os

@mbed-ci
Copy link

mbed-ci commented Oct 10, 2017

Copy link
Contributor

@kjbracey kjbracey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nearly, nearly there. Should be good to go into 5.7, I think.

}

member_pair_index = 0;
member_pair_index = next_free_multicast_member(s, member_pair_index);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

simplify to member_pair_index = next_free_multicast_member(s, 0).

@@ -84,6 +116,18 @@ static struct lwip_socket *mbed_lwip_arena_alloc(void)
static void mbed_lwip_arena_dealloc(struct lwip_socket *s)
{
s->in_use = false;

while(s->multicast_memberships_count > 0) {
static uint32_t index = 0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This static is broken - will fail on second dealloc - make it non-static, declared & initialised outside the loop.

static void mbed_lwip_arena_dealloc(struct lwip_socket *s)
{
s->in_use = false;

while(s->multicast_memberships_count > 0) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tidy still needed


s->multicast_memberships[member_pair_index] = *imr;

set_multicast_member_registry_bit(s, member_pair_index);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess you could reduce code size by deferring the member table update to line 1255, making the condition "if err == OK", rather than doing it now then backing out. As long as you've already chosen the member index and know there's room.

But while pondering that, it occurs to me that there's no multithreading protection here. igmp_joingroup is not one of the thread-safe APIs in lwIP.

Now, calls to your API are protected by a mutex on your Socket, so you don't need to worry about your own membership list protection. But you will need a sys_arch_protect() around just the lwIP calls, for both join and leave, to protect its per-interface list.

mreq.imr_multiaddr = address.get_addr();
mreq.imr_interface = nsapi_addr_t(); // Default address, NSAPI_UNSPEC

return this->setsockopt(0, socketopt, &mreq, sizeof(mreq));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First parameter here should be NSAPI_SOCKET

static uint32_t index = 0;
index = next_registered_multicast_member(s, index);

mbed_lwip_setsockopt(0, s, 0, NSAPI_DROP_MEMBERSHIP, &s->multicast_memberships[index],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Third parameter should technically be NSAPI_SOCKET - at some point mbed_lwip_setsockopt should enforce that.

First parameter should be written as NULL.

@mbed-bot
Copy link

Result: FAILURE

Your command has finished executing! Here's what you wrote!

/morph test

Output

mbed Build Number: 1559

Example Build failed!

@mbed-ci
Copy link

mbed-ci commented Oct 10, 2017

@mbed-ci
Copy link

mbed-ci commented Oct 11, 2017

@kegilbert
Copy link
Contributor Author

This still has the Needs Review tag, should I add someone else to the review list?

@geky
Copy link
Contributor

geky commented Oct 18, 2017

I think this is good to go then, @theotherjimmy, @0xc0170, should this be "needs: preceding PR"?

Unfortunately this will have to wait until we get the network tests working again: #5335

@geky geky added the needs: CI label Oct 18, 2017
@0xc0170
Copy link
Contributor

0xc0170 commented Oct 19, 2017

I think this is good to go then, @theotherjimmy, @0xc0170, should this be "needs: preceding PR"?

I am not certain now, @kegilbert tests are green, so all should be good now?

@geky
Copy link
Contributor

geky commented Oct 20, 2017

Although this needs #5335 before all tests are ran

@0xc0170
Copy link
Contributor

0xc0170 commented Nov 9, 2017

Although this needs #5335 before all tests are ran

@kegilbert Can you help @yennster with the 5335 as this depends on it? Or anything else can be done for this PR to make it ready for intergration?

@kegilbert
Copy link
Contributor Author

I can take a look at it when I get back next week from GGC, don't have access to a Nucleo board at the moment.

@kegilbert
Copy link
Contributor Author

#5335 was merged, rekicking off CI

@kegilbert
Copy link
Contributor Author

/morph build

@mbed-ci
Copy link

mbed-ci commented Nov 16, 2017

Build : SUCCESS

Build number : 537
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/5196/

Triggering tests

/morph test
/morph uvisor-test
/morph export-build

@mbed-ci
Copy link

mbed-ci commented Nov 16, 2017

@mbed-ci
Copy link

mbed-ci commented Nov 16, 2017

@kegilbert
Copy link
Contributor Author

@0xc0170 @geky Tests passed with the new network test patches.

@geky
Copy link
Contributor

geky commented Nov 16, 2017

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants