Skip to content

Documentation: Host sample updated for IPv4+IPv6. #5372

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

Documentation: Host sample updated for IPv4+IPv6. #5372

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion examples/server/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Command line options:
- `--lora FNAME`: Apply a LoRA (Low-Rank Adaptation) adapter to the model (implies --no-mmap). This allows you to adapt the pretrained model to specific tasks or domains.
- `--lora-base FNAME`: Optional model to use as a base for the layers modified by the LoRA adapter. This flag is used in conjunction with the `--lora` flag, and specifies the base model for the adaptation.
- `-to N`, `--timeout N`: Server read/write timeout in seconds. Default `600`.
- `--host`: Set the hostname or ip address to listen. Default `127.0.0.1`.
- `--host`: Set the hostname or ip address to listen. Default `127.0.0.1`. Use `::` for any IPv4+IPv6.
Copy link
Collaborator

@cebtenzzre cebtenzzre Feb 6, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this addition would need to provide at least a little more context.

  • 127.0.0.1 is the IPv4 localhost address, 0.0.0.0 is the IPv4 wildcard address.
  • ::1 is the IPv6 localhost address.
  • 'localhost' should resolve to both IPv4 and IPv6 addresses, on a properly configured system.
  • :: is the IPv6 wildcard address, and its meaning depends on the OS - on Windows it binds to IPv6 only. On Linux it will also bind to IPv4 if sysctl net.ipv6.bindv6only is zero, which is usually the case. I'm not sure about macOS.

I think there would need to be a code change to reliably listen on both stacks on any OS. If configured correctly, providing the hostname of the local host instead of :: is probably sufficient - but I don't know how e.g. a Windows system will default to resolving its own hostname (e.g. it could give 127.0.0.1 instead of the local IPv4 address).

Copy link
Contributor Author

@jboero jboero Feb 6, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What change? Try it. It works on both stacks already. At least for me in Linux it works for both IPv4 and IPv6 as standard. That works for Go apps also. Oh I see what you mean now.

I agree it could use more context maybe but the functionality is there.

- `--port`: Set the port to listen. Default: `8080`.
- `--path`: path from which to serve static files (default examples/server/public)
- `--api-key`: Set an api key for request authorization. By default the server responds to every request. With an api key set, the requests must have the Authorization header set with the api key as Bearer token. May be used multiple times to enable multiple valid keys.
Expand Down