Logs of ssh commands larger than pool size get lost

[rymndhng]

As mentioned in the docs, ParallelSSHClient.run_command blocks until all the commands have started. I've noticed that this means if I have more than 10 hosts I want to run the command on -- only the last 10 hosts are accessible. Is there a way for me to keep the pool size at 10 while getting the logs of all the remote hosts?

[pkittenis]

Should be able to get logs for all hosts by calling ParallelSSHClient.get_output() multiple times.

[rymndhng]

Here's what I gathered from the docs of self.pool.spawn:

The Pool which a subclass of Group provides a way to limit concurrency: its spawn method blocks if the number of greenlets in the pool has already reached the limit, until there is a free slot.

So, let's say I have 50 hosts I want to run pssh on and I use the default pool size 10. Then, in get_output(), self.pool.greenlets only contain the last 10 operations.

Is this understanding correct?

[pkittenis]

Yes, that is correct. Calls to get_output() will return logs from pool_size number of hosts.

Additional calls should return logs from another batch of pool_size number of hosts, until there are no more hosts at which point get_output() will return None.

[rymndhng]

I don't get the expected behavior, perhaps there's something obvious I'm missing out on. I'm running against 4 hosts, and I found that the 2nd invocation to get_output() is an empty dictionary.

            client = ParallelSSHClient(hosts=hosts, user=user, pool_size=1)
            output = client.run_command(command)

            while True:
                if len(output) == 0:
                    break

                for host in output:
                    print ">> Performed command \033[93m'{cmd}'\033[0m on \033[92m{name} - {dns}\033[0m".format(
                        cmd=command, name=user, dns=host)

                    for line in output[host]['stdout']:
                        pass

                output = client.get_output()

Where in the code does "additional calls should return logs from another batch" happen? My understanding is that spawn with a fixed pool size will block until all the operations are complete. In which case we would have already discarded all the previous greenlets and lose access to their logs.

[pkittenis]

Thanks, you are correct - that looks like a bug.

[pkittenis]

Have what I believe is a fix for this - will post a bug fix release once I have a test to replicate the failing behaviour.

You can fall back to the - deprecated but still working - ParallelSSHClient.exec_command and ParallelSSHClient.get_stdout if this is a blocker for you.

[pkittenis]

This is good now as of version 0.70.2