Skip to content

Allow output gathering for working hosts when one or more hosts have failures #32

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jasonfharris opened this issue Apr 10, 2015 · 3 comments · Fixed by #33
Closed

Allow output gathering for working hosts when one or more hosts have failures #32

jasonfharris opened this issue Apr 10, 2015 · 3 comments · Fixed by #33

Comments

@jasonfharris
Copy link

When executing say client.run_command(...) over many hosts there is the possibility that a single host has problems. Maybe there is a nice way to recover when a single hosts fails? but I didn't see how to so this easily at present. Right now an exception is thrown and things stop. A nice way to recover would be good. In fact our example actually tests some of the hosts and sometimes they don't respond or they are switched off, etc. etc.

@pkittenis
Copy link
Member

Hi there, thank you for the feedback.

The issue is that if an exception occurs at run_command when getting output the function stops and its return value is lost.

Command execution does not stop, it's just that output is not retrieved due to the exception.

Will have to think about how best to handle this gracefully while still allowing user to get output for the remainder of the host list.

Exceptions will need to be caught regardless.

@pkittenis pkittenis self-assigned this Apr 13, 2015
@pkittenis pkittenis changed the title Allow recovery for an unreachable host Allow output gathering for working hosts when one or more hosts have failures Apr 21, 2015
pkittenis pushed a commit that referenced this issue Apr 21, 2015
…case where the host lists contains one or more failing hosts. Refactored run_command to allow raising an exception on errors or not
@pkittenis
Copy link
Member

Have started adding a keyword argument to turn on/off exception raising in run_command.

This will allow you to make a decision on whether or not you want exceptions raised on errors or placeholder output containing exception object returned instead, along output for the remainder of the host list.

As this is new behaviour and is backwards compatible with the existing API it will be released in a new point release.

NB - return and raise are mutually exclusive in Python. A function that raises an exception cannot return and vice versa. It is not possible to simultaneously raise an exception on error while also returning output.

@pkittenis pkittenis added this to the Release 0.71 milestone Apr 21, 2015
@pkittenis pkittenis modified the milestones: Release 0.71, Release 0.8 Jun 24, 2015
pkittenis pushed a commit that referenced this issue Jun 24, 2015
… as argument to exceptions so that output can be updated correctly for the host the exception occured for. Updated test_pssh_client_run_command_get_output_explicit for changed get_output interface - Resolves #32
@pkittenis pkittenis modified the milestones: Release 0.8, Release 0.80 Jun 24, 2015
@pkittenis
Copy link
Member

Finally got some time to finish this at the output branch.

There is a back-wards incompatible change to a public API function, get_output, so this change has been moved back to the 0.80 release which will be out once documentation has been updated to reflect this and other changes.

Feel free to install the output branch in the meantime but be aware that not all exceptions are yet being tested for partial failures and looking at the code it is likely that UnknownHostException and ConnectionErrorException will cause host output for the failed host to be lost - but not output for rest of host list.

Exhaustive tests for this new behaviour will be in before output branch is merged.

With the changes in that branch, to cause ParallelSSHClient to not stop on errors, call run_command like:

output = client.run_command('cmd', stop_on_errors=False)

If a host in the host list has an exception, its output will contain an exception key with the exception object as its value.

If you have been using get_output directly, interface has changed to:

output = {}
for cmd in cmds:
    # output variable gets updated in-place for every cmd's output
    client.get_output(cmd, output)

from

output = client.get_output(commands=cmds)

pkittenis pushed a commit that referenced this issue Sep 25, 2015
…case where the host lists contains one or more failing hosts. Refactored run_command to allow raising an exception on errors or not
pkittenis pushed a commit that referenced this issue Sep 25, 2015
… as argument to exceptions so that output can be updated correctly for the host the exception occured for. Updated test_pssh_client_run_command_get_output_explicit for changed get_output interface - Resolves #32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants