TLS in Gopher

[Christoph Lohmann]

Greetings comrades.

The idea of TLS for gopher has come up again. With rising popularity
people seem to want to push all HTTP principles into gopher. Since I
have control of geomyidae and some gopher clients for tryouts I want to
propose the following to end the whole discussion:

Ideal way:

C->S: STARTTLS\r\n
S->C: <TLS begins on both sides>
C->S(in TLS): selector[\tsearch]\r\n
S->C(in TLS): answer

Compatibility:

C->S: STARTTLS\r\n
S->C: 3Some error .... (TLS on client side fails)
# New connection
C->S: selector\r\n
S->C: answer

I proposed to make every selector beginning with a char of uppercase
ASCII (A‐Z) to be a special case. This will permit following historical
compatibility:

* Servers allowing HTTP transparent serving.
* Maybe allow things like haproxy to work for gopher servers.


## Why no separate port?
* We need to define some way in menus for TLS entries.
* It will require not just an upgrade of clients and servers.
* It will require gophers:// to be defined and introduced.

## What other proposals were there to add TLS support?
* Using some separate tab parameter.
* Will get in the way with gopher+.
* Gopher+ is already a big enough burden.
* Using some caps file.
* Reimplementation of gopher+.
* Adds metadata for fingerprinting.
* Think of tor usage.
* Adds roundtrips.
* Using some special path like »/.starttls«.
* Can overlap with real files.
* My proposal of first checking for TLS data bytes.
* Does not add the other compatibility I mention above.


What do you people think of my proposal?

Depending on the input I will start implementing this in geomyidae and
sacc and write the RFC.


Sincerely,

Christoph Lohmann

[Alex Schröder]

On Tue, Feb 27, 2018 at 7:08 PM, Christoph Lohmann <***@r-36.net> wrote:

> Ideal way:
>
> C->S: STARTTLS\r\n
> S->C: <TLS begins on both sides>
> C->S(in TLS): selector[\tsearch]\r\n
> S->C(in TLS): answer
>
>
The only drawback I see is that this complicates connecting somewhat. How
good will library support be for this? Looking at Python 3.6, for example,
it's trivial to use SSL but needs some more coding to get STARTTLS right.
https://docs.python.org/3.6/library/ssl.html
https://docs.python.org/3.6/library/ssl.html

gnutls-cli has a --starttls option, but I'd have to see it in action.
http://man7.org/linux/man-pages/man1/gnutls-cli.1.html

Sadly, faulty implementations have led to protocol downgrade attacks.
https://crypto.stackexchange.com/questions/10493/why-is-tls-susceptible-to-protocol-downgrade-attacks#10495

I proposed to make every selector beginning with a char of uppercase
> ASCII (A‐Z) to be a special case. This will permit following historical
> compatibility:
>
> * Servers allowing HTTP transparent serving.
> * Maybe allow things like haproxy to work for gopher servers.
>
>
I don't quite understand this. Is this because you assume that all
selectors start with a slash?
What about the following? Would it no longer work?

echo Alex_Schroeder | nc alexschroeder.ch 70



>
> ## Why no separate port?
> * We need to define some way in menus for TLS entries.
> * It will require not just an upgrade of clients and servers.
> * It will require gophers:// to be defined and introduced.
>
>
I personally find this easy to deal with. The default is simply that old
clients cannot follow links to TLS enabled servers and new clients need a
way to enable some sort of TLS mode in which they expect all menus to be
encrypted. As long as you read documents on the same server, no problem.
When you follow links, clients will get errors when they're in the wrong
mode and can try and recover on their own, if they want to, hopefully only
upgrading to TLS mode and never downgrading. When I added this to VF1, I
noticed no significant usability degradation.

Depending on the input I will start implementing this in geomyidae and
> sacc and write the RFC.
>
>
More code is always good!

Cheers
Alex

[Cameron Kaiser]

> The idea of TLS for gopher has come up again. With rising popularity
> people seem to want to push all HTTP principles into gopher. Since I
> have control of geomyidae and some gopher clients for tryouts I want to
> propose the following to end the whole discussion:
>
> Ideal way:
>
> C->S: STARTTLS\r\n
> S->C: <TLS begins on both sides>
> C->S(in TLS): selector[\tsearch]\r\n
> S->C(in TLS): answer
>
> Compatibility:
>
> C->S: STARTTLS\r\n
> S->C: 3Some error .... (TLS on client side fails)
> # New connection
> C->S: selector\r\n
> S->C: answer

Although I would prefer gopher+TLS have a categorical port and only be
accessible that way, I don't object to probing/upgrading, as long as the
client is smart about it.

My personal preference is that the client also cache once a STARTTLS probe
has succeeded, and then start using port 7443 directly when it encounters
a menu item referencing that server again so that downgrade attacks are
minimized. (As a consequence, every gopher server implementing gopher+TLS
should support both STARTTLS upgrading on port 70, as well as always
offering always-encrypted connections on the so-far de facto port 7443.)

A CAPS key could also aid in service discovery, but that can be optional.

To be explicit, once the client knows the server is capable of STARTTLS,
then if it sees port 70 for that server in a menu item, it should
automatically substitute 7443 and immediately speak TLS on that port in
future connections. Other port numbers should not be assumed. I'm not
picky about the timeout. There could be a CAPS key for this too which
would be analogous to HSTS, if people desire.

Similarly, the client should stop probing a server after a certain number
of tries. This number should be greater than 1 in case a downgrade attack
succeeds, but failure should be cached relatively quickly to avoid spamming
logs on currently existing non-TLS servers. Again, I'm not picky about
this number, but the behaviour should have an upper limit.

This adds complexity to gopher+TLS, but since gopher+TLS servers and clients
are likely to be operating on more modern systems, I don't feel it is an
undue burden.

--
------------------------------------ personal: http://www.cameronkaiser.com/ --
Cameron Kaiser * Floodgap Systems * www.floodgap.com * ***@floodgap.com
-- It is the business of little minds to shrink. -- Carl Sandburg -------------

[Iain R. Learmonth]

Hi,

On 28/02/18 02:05, Cameron Kaiser wrote:
> Although I would prefer gopher+TLS have a categorical port and only be
> accessible that way, I don't object to probing/upgrading, as long as the
> client is smart about it.

The idea of requiring a specific port number sounds like a complete hack.

Instead, can you not just cache that STARTTLS was successful and then
immediately attempt TLS handshake when you connect next time instead of
doing the STARTTLS upgrade?

Even better, how about you just always try to do a TLS handshake and
cache the failures instead? STARTTLS in reverse, where if the handshake
fails you reconnect without TLS and then don't attempt a handshake again
for a while.

Thanks,
Iain.

[Nuno Silva]

On 2018-02-28 07:53 +0000, Iain R. Learmonth wrote:
> On 28/02/18 02:05, Cameron Kaiser wrote:
> > Although I would prefer gopher+TLS have a categorical port and only be
> > accessible that way, I don't object to probing/upgrading, as long as the
> > client is smart about it.
>
> The idea of requiring a specific port number sounds like a complete
> hack.

Doesn't look hackish to me -- after all, isn't this how internet
services, including HTTPS, work? They have a default port number
assigned, which is used unless you specify a port number.

>
> Instead, can you not just cache that STARTTLS was successful and then
> immediately attempt TLS handshake when you connect next time instead of
> doing the STARTTLS upgrade?
>
> Even better, how about you just always try to do a TLS handshake and
> cache the failures instead? STARTTLS in reverse, where if the handshake
> fails you reconnect without TLS and then don't attempt a handshake again
> for a while.
>

--
Nuno Silva

[Iain R. Learmonth]

Hi,

On 28/02/18 14:20, Nuno Silva wrote:
> On 2018-02-28 07:53 +0000, Iain R. Learmonth wrote:
>> The idea of requiring a specific port number sounds like a complete
>> hack.
>
> Doesn't look hackish to me -- after all, isn't this how internet
> services, including HTTPS, work? They have a default port number
> assigned, which is used unless you specify a port number.

This solution would preclude the use of any port number except for the
"assigned" port number, as other ports would not be expected to use TLS.
For this reason alone this idea should be considered a poor solution.

Thanks,
Iain.

[Christoph Lohmann]

Greetings.

Thanks for all the input. I rethought my proposel. For finding a solu‐
tion we can all agree on RFC 1436, which says:

> Equally important, clients do not
> have to be modified to take advantage of a new resource.

This gave me the idea that we should see this from the user/client point
of view, which begins at the URI. With gopher:// the one writing the URI
defines the menu type the way the client should interpret the content
given back. The same could apply for gophers://.

To see my new proposal below keep in mind the two main goals we should
have in gopher:

* Keep the compatibility to the old gopher.
* Allow a simple implementation on really old hardware, which is not able
to run TLS.


## Proposal

I propose that we should introduce »gophers://« schema which tells the
client to use »secure mode« for digging through gopherspace. Clients
might introduce some parameter to first try secure mode, even if »go‐
pher://« is specified as URI schema.

## What does the »secure mode« mean?

We do not need the »STARTTLS« I proposed. The simpler way is like some‐
one said to just connect using TLS. The TLS header is an invalid ASCII
selector and so any compliant server will give back an error. If this
does not succeed the client should / must not try to connect us‐
ing plain gopher. If a server supports TLS should be cached by the
client.


## Examples Of Daily Usage

# Plaintext Gopher
% sacc gopher://bitreich.org

# Secure Mode by default. [not implemented yet.]
# Sacc will try first TLS before plain gopher.
# -s can be easily set as an alias in the shell.
% sacc -s gopher://bitreich.org

# The user tells sacc to use secure mode. [not implemented yet.]
% sacc gophers://bitreich.org

For the ultra secure PKI fighters some option to enforce TLS and exit if
it is not available could be implemented too (Example):

% sacc -fs gopher://r-36.net
sacc: Server does not support TLS. Aborting.

% sacc -fs gopher://bitreich.org
sacc: bitreich.org is giving back a self-signed certificate.
[... cert details printed ...]
Do you want to accept and proceed?
[y/n] >

# After a while gophering and someone tries to attack.
% sacc -fs gopher://bitreich.org
sacc: Certificate for bitreich.org changed.
[... cert details printed ...]
Do you want to accept and proceed?
[y/n] >


## Easy Migration (Practical Example)

For bitreich.org I setup all of this without modifying geomyidae at all.

1.) Setup geomyidae to listen on port 72.
2.) Setup stunnel to listen on port 7443 and point to port 72.

With stunnel you get all options for verification and certificate han‐
dling without doubling the code in geomyidae itself.

3.) Setup sslh on port 72 to point to either port 7443 or port 72.

Here is my sslh configuration:

protocols:
(
{ name: "regex"; host: "127.0.0.1"; port: "72"; regex_patterns: [ "^\r\n$", "^\n$", "^.\n$" ]; },
{ name: "tls"; host: "127.0.0.1"; port: "7443"; },
{ name: "anyprot"; host: "127.0.0.1"; port: "72"; },
{ name: "timeout"; host: "127.0.0.1"; port: "72"; }
);

on-timeout: "timeout";

Why the regex? Sslh has some case where if something less than three
bytes is sent tls comes true. We have to handle those cases manually.

Sslh has other options like running vpn, ssh etc. on the same port to
circumvent filtering of censoring networks.

Of course all of this can be implemented in the geomyidae too, but it
should be easy, since only a check for the TLS packet like in sslh[0] is
needed.


## Conclusion

The above proposal should satisfy all use cases. There is a simple up‐
grade route and it will give the user more freedom in choosing how to
walk through gopherspace.

Enforcing pure TLS on all gopher servers is not an option, which means
fallback attacks are possible. Here the client needs to warn users, be‐
cause PKI is about user decision. You cannot trust central certificate
authorities, you only trust the certificates you know, like in SSH.


## RFC

What do you people think about this proposal?


Sincerely,

Christoph Lohmann

[0] https://github.com/yrutschle/sslh.git

[Iain R. Learmonth]

Hi,

On 28/02/18 05:48, Christoph Lohmann wrote:
> I propose that we should introduce »gophers://« schema which tells the
> client to use »secure mode« for digging through gopherspace.

Gopher menus do not use URLs so it's impossible to link to such a
resource. It also creates a division between "secure" and "insecure"
gopherspace where resources are not the same as they have different URLs.

This is, specifically, the thing that I would least like to see in the
implementation of TLS for Gopher. Please stop looking at things that
already exist that have known problems and instead think about solving
those problems. These problems would be a lot more problematic in Gopher
due to the lack of use of actual URLs.

Thanks,
Iain.

[Alex Schroeder]

"Iain R. Learmonth" <***@fsfe.org> writes:

> Gopher menus do not use URLs so it's impossible to link to such a
> resource. It also creates a division between "secure" and "insecure"
> gopherspace where resources are not the same as they have different
> URLs.

Personally, that has two benefits:

First, old clients can use the old resources and follow the old links
without a problem. 100% backwards compatibility.

Second, implementation of clients and servers is simple because
programming languages provide libraries that offer secured or unsecured
connections.

There are two drawbacks:

The first is that Gopher clients will now either have to offer users an
explicit "mode" (will attempt all connections using either TLS or no
TLS, which is what I chose to implement for the client VF-1), or do
their own "sniffing" by attempting to connect using either TLS and then
trying no TLS after informing the user or checking their cache. Granted,
that means we don't have 100% interoperability as old clients cannot
follow links to secured resources.

The second is that secure and insecure sites live in separate spaces,
secure content and insecure content have different URLs. To me, this is
simply a minor annoyance. I haven't run into an actual problem with
this. When I share the link to an insecure resource with somebody who
has a newer Gopher client that can use TLS, it will use an insecure
connection because that's what the URL says. This appears to be
drawback? I can live with it.

Server side sniffing is technically interesting, but not well supported
by the sort of scripting languages that make implementing Gopher servers
so trivially easy. I'm familiar with Emacs Lisp, Python and Perl because
I worked on Gopher clients and servers using those three and I used
gnutls-cli for testing on the command line, and none of these offered
libraries with the kind of abilities described. Writing a Gopher server
would end up being non-trivial. This I find much harder to accept.

Perhaps I'm wrong and it is in fact trivially easy? Can somebody post
some sample code, a few lines maybe, illustrating how it would work in a
simple Gopher server?

Cheers
Alex
--
Public Key Fingerprint = DF94 46EB 7B78 4638 7CCC 018B C78C A29B ACEC FEAE

[Iain R. Learmonth]

Hi,

On 01/03/18 07:18, Alex Schroeder wrote:
> Perhaps I'm wrong and it is in fact trivially easy? Can somebody post
> some sample code, a few lines maybe, illustrating how it would work in a
> simple Gopher server?

Accept connection
Read a few bytes, enough to know if it's a TLS client hello
If it's TLS:
Hand over to TLS library
If it's not TLS:
Handle the query directly

This is not that hard.

In Python you would use the socket.MSG_PEEK option on a recv() call to
take a look at those bytes before passing it off to a TLS library or
handling the request directly.

Thanks,
Iain.

[Mateusz Viste]

On Thu, 01 Mar 2018 08:50:40 +0000, Iain R. Learmonth wrote:
> Accept connection Read a few bytes, enough to know if it's a TLS client
> (...)
> This is not that hard.

What if the only byte received is a single 0x0A ? Is this enough to tell
"not TLS"? How long would the server need to wait for more bytes?

Mateusz

--
mailing lists suck, right? come see us at comp.infosystems.gopher
why? gopher://gopher.viste.fr/0whyusenet.txt

[Iain R. Learmonth]

Hi,

On 01/03/18 09:57, Mateusz Viste wrote:
> What if the only byte received is a single 0x0A ? Is this enough to tell
> "not TLS"? How long would the server need to wait for more bytes?

You would be expecting to receive a handshake message, which would start
with 0x16 (ASCII: SYN). I cannot think of a reason that a Gopher
selector would start with a non-printable character, so you can most
likely check only a single byte.

Thanks,
Iain.

[Mateusz Viste]

On Thu, 01 Mar 2018 10:38:03 +0000, Iain R. Learmonth wrote:
> You would be expecting to receive a handshake message, which would start
> with 0x16 (ASCII: SYN). I cannot think of a reason that a Gopher
> selector would start with a non-printable character, so you can most
> likely check only a single byte.

Sounds good. You should definitely implement it then, so we can see how
it works.

Mateusz
--
mailing lists suck, right? come see us at comp.infosystems.gopher
why? gopher://gopher.viste.fr/0whyusenet.txt

[Michael Lazar]

On Thu, Mar 1, 2018 at 1:37 PM, Mateusz Viste <***@nospam.viste.fr>
wrote:
> On Thu, 01 Mar 2018 10:38:03 +0000, Iain R. Learmonth wrote:
>>
>> You would be expecting to receive a handshake message, which would start
>> with 0x16 (ASCII: SYN). I cannot think of a reason that a Gopher
>> selector would start with a non-printable character, so you can most
>> likely check only a single byte.
>
>Sounds good. You should definitely implement it then, so we can see how
>it works.

I thought this was a nice idea, so I tried implementing it on my python
gopher server. You can check it out here:

gopher://mozz.us:7005/1/demo-ssl

It was, in fact, very straightforward to accomplish using helpers from the
python standard library. Here's the code diff if you want to take a look:


https://github.com/michael-lazar/flask-gopher/commit/3777ef92516722f8aad7b858a0c6f931bd875f19#diff-d66156e6e2bebef5659c7aa5593e5ec6R628

- Michael Lazar

[Philipp Schafft]

Good morning,


On Thu, 2018-03-01 at 08:50 +0000, Iain R. Learmonth wrote:
> Hi,
>
> On 01/03/18 07:18, Alex Schroeder wrote:
> > Perhaps I'm wrong and it is in fact trivially easy? Can somebody post
> > some sample code, a few lines maybe, illustrating how it would work in a
> > simple Gopher server?
>
> Accept connection
> Read a few bytes, enough to know if it's a TLS client hello
> If it's TLS:
> Hand over to TLS library
> If it's not TLS:
> Handle the query directly
>
> This is not that hard.

Exactly.


> In Python you would use the socket.MSG_PEEK option on a recv() call to
> take a look at those bytes before passing it off to a TLS library or
> handling the request directly.

This is what everybody uses. The auto detection isn't anything new or
fancy. A lot software supports it.

Just for fun I implemented Gopher support in Icecast 2.5.x this morning.
It has some limitations: selectors always start with / (as they are the
same as on HTTP). There are also no menus as there are no templates for
menus. They can be added in web/ and admin/ directories later if needed.

If the listen socket is set to "auto" or "auto_no_plain" TLS mode it
will support TLS auto detection. If it is set to "rfc2818" the server
will expect TLS right from the start. RFC2817 mode ("rfc2817") is
currently not supported as I have not (yet) implemented
Upgrade-to-Gopher.

You can find gopher support in branch "ph3-gopher" in the official repo,
see: https://wiki.xiph.org/Icecast_Server/Git_workflow

Some examples:
$ printf "/example1.ogg\r\n" | openssl s_client -connect \
localhost:8905 -quiet | ogg123 -
$ printf "/example1.ogg\r\n" | netcat localhost 8905 | ogg123 -


Have fun.


With best regards,

--
Philipp.
(Rah of PH2)

[Christoph Lohmann]

Greetings,

On Thu, 01 Mar 2018 12:43:53 +0100 "Iain R. Learmonth" <***@fsfe.org> wrote:
> Hi,
>
> On 28/02/18 05:48, Christoph Lohmann wrote:
> > I propose that we should introduce »gophers://« schema which tells the
> > client to use »secure mode« for digging through gopherspace.
>
> Gopher menus do not use URLs so it's impossible to link to such a
> resource. It also creates a division between "secure" and "insecure"
> gopherspace where resources are not the same as they have different URLs.

You are wrong here. In my daily gopher usage my entry to gopherspace is
always some URI. On IRC URIs are exchanged and then given to the client
via some plumbing. After that this client session is started and fur‐
ther navigation hap‐ pens.

In my e‐mail further below I describe why I propose gophers://, which is
to tell the client to use a secure mode. The secure mode is then de‐
scribed below and is in conclusion just TLS probing. I did not want to
define the »TLS«, how it has to be implemented since this is a moving
target. All of this complexity should lie in the TLS library used. For
full security of course DNSSEC + DANE is needed or something likewise to
check fingerprints from a second source. But this is part of TLS, not
gopher.

I do not want to change the menu layout, since there is no advantage we
get. Why should there be a distinction in defined menus? It will make
development of applications using gopher menus harder. Instead if your
entry to gopherspace was marked to use secure mode and the client sup‐
ports it, the probing is done – automatically. Everyone hosting some old
gopher content will automagically have secure mode available, without
modification.

This is why having a distinct port for gophers is useless. I want to run
gopher and TLS on every port available, not just 7443 or what we define.

> This is, specifically, the thing that I would least like to see in the
> implementation of TLS for Gopher. Please stop looking at things that
> already exist that have known problems and instead think about solving
> those problems. These problems would be a lot more problematic in Gopher
> due to the lack of use of actual URLs.

As said above: I proposed exactly what you said in further e‐mails: Sim‐
ple TLS probing, TLS implementation is client‐side and backwards compat‐
ibility to the old gopher. For me the URI is a mark – as said in my pre‐
vious e‐mail.

With the proposal you can not just tell the client how to interpret the
content:
gopher://bitreich.org/I/stateless-misra.png
vs.
gopher://bitreich.org/0/stateless-misra.png

but also to try secure mode:
gophers://bitreich.org/I/stateless-misra.png

If probing is turned on by default by the client it is not needed. I
want to give freedom, not restriction.

Btw., for any incompatibility or »split« you are concerned about. If
someone wants content to only be served over TLS, then the plain old go‐
pher client will get an error, if the gopher server is implemented cor‐
rectly. We use a principle not known in the web world anymore: Inform
the user in plain text.

»Sorry, this content is only served using TLS. Please see ... for
details on TLS over gopher.«


Sincerely,

Christoph Lohmann

[Alex Schröder]

On Wed, Feb 28, 2018 at 5:48 AM, Christoph Lohmann <***@r-36.net> wrote:

> We do not need the »STARTTLS« I proposed. The simpler way is like some‐
> one said to just connect using TLS. The TLS header is an invalid ASCII
> selector and so any compliant server will give back an error. If this
> does not succeed the client should / must not try to connect us‐
> ing plain gopher. If a server supports TLS should be cached by the
> client.
>

Just to put a final nail into STARTTLS: https://tools.ietf.org/html/rfc8314
is about email but the principle is the same. Here's what they have to say:

3. Implicit TLS
>
> Previous standards for the use of email protocols with TLS used
> the STARTTLS mechanism: [RFC2595], [RFC3207], and [RFC3501]. With
> STARTTLS, the client establishes a cleartext application session
> and determines whether to issue a STARTTLS command based on
> server capabilities and client configuration. If the client
> issues a STARTTLS command, a TLS handshake follows that can
> upgrade the connection. Although this mechanism has been
> deployed, an alternate mechanism where TLS is negotiated
> immediately at connection start on a separate port (referred to
> in this document as "Implicit TLS") has been deployed more
> successfully. To encourage more widespread use of TLS and to also
> encourage greater consistency regarding how TLS is used, this
> specification now recommends the use of Implicit TLS for POP,
> IMAP, SMTP Submission, and all other protocols used between an
> MUA and an MSP.
>

Cheers
Alex

[Iain R. Learmonth]

Hi,

On 01/03/18 11:05, Alex Schröder wrote:
> Just to put a final nail into STARTTLS: https://tools.ietf.org/html/rfc8314
> is about email but the principle is the same.

The principle is *not* the same. For SMTP, the server sends data (the
banner) first as it can't know what the client is doing. For Gopher, the
client talks first and then the server can respond either using or not
using TLS based on what the client has sent.

STARTTLS was needed because the server would have already sent plaintext
to the client before it knew the client wanted TLS and would mess up the
handshake. This wouldn't happen for Gopher.

Thanks,
Iain.

[Kim Holviala]

On 27 Feb 2018, at 20.08, Christoph Lohmann <***@r-36.net> wrote:
>
> Ideal way:
>
> C->S: STARTTLS\r\n
> S->C: <TLS begins on both sides>
> C->S(in TLS): selector[\tsearch]\r\n
> S->C(in TLS): answer

Starttls is nice but requires gopher servers to be complied with OpenSSL or similar and will not work with inetd-based servers (because TLS would need to be initialized from scratch for each connection). So it will conpletely kill any simplicity gopher might have had. Dedicated TLS port (7443?) would work with TLS wrappers leaving the gopher server itself simple and the codebase understandable.

But I'm not going to veto STARTTLS either because yes, it does solve many problems.


- Kim

[Iain R. Learmonth]

Hi,

On 28/02/18 07:55, Kim Holviala wrote:
> Starttls is nice but requires gopher servers to be complied with OpenSSL or similar and will not work with inetd-based servers (because TLS would need to be initialized from scratch for each connection). So it will conpletely kill any simplicity gopher might have had. Dedicated TLS port (7443?) would work with TLS wrappers leaving the gopher server itself simple and the codebase understandable.

You could always have a TLS wrapper that is also happy to not do TLS.

This would allow clients to perform TLS if they wish, or not, and all on
the same port without complicating any codebases.

Thanks,
Iain.

[Kim Holviala]

On 28 Feb 2018, at 10.14, Iain R. Learmonth <***@fsfe.org> wrote:
>
>
> You could always have a TLS wrapper that is also happy to not do TLS.

True, although I don't know any such existing wrapper. Also as started in RFC3207 STARTTLS is less secure than dedicated port - an attacker can always strip the STARTTLS while dedicated port either connects securely or doesn't connect at all.

Since gopher hardcodes both server names and port numbers (all links are absolute) adding TLS is a pain without breaking the protocol. I know, I've been researching this for a couple of years now. Starttls is the easiest as it doesn't break the protocol but it's also no more secure than current plaintext connections while adding huge libraries to currently very simple servers.


- Kim

> This would allow clients to perform TLS if they wish, or not, and all on
> the same port without complicating any codebases.
>
> Thanks,
> Iain.
>

[R. A. Pavlov]

Why would one ever need a secure gopher if gopher client cannot send any
sensitive info by design? Do I miss something?

On 27.02.2018 21:08, Christoph Lohmann wrote:
> Greetings comrades.
>
> The idea of TLS for gopher has come up again. With rising popularity
> people seem to want to push all HTTP principles into gopher. Since I
> have control of geomyidae and some gopher clients for tryouts I want to
> propose the following to end the whole discussion:
>
> Ideal way:
>
> C->S: STARTTLS\r\n
> S->C: <TLS begins on both sides>
> C->S(in TLS): selector[\tsearch]\r\n
> S->C(in TLS): answer
>
> Compatibility:
>
> C->S: STARTTLS\r\n
> S->C: 3Some error .... (TLS on client side fails)
> # New connection
> C->S: selector\r\n
> S->C: answer
>
> I proposed to make every selector beginning with a char of uppercase
> ASCII (A‐Z) to be a special case. This will permit following historical
> compatibility:
>
> * Servers allowing HTTP transparent serving.
> * Maybe allow things like haproxy to work for gopher servers.
>
>
> ## Why no separate port?
> * We need to define some way in menus for TLS entries.
> * It will require not just an upgrade of clients and servers.
> * It will require gophers:// to be defined and introduced.
>
> ## What other proposals were there to add TLS support?
> * Using some separate tab parameter.
> * Will get in the way with gopher+.
> * Gopher+ is already a big enough burden.
> * Using some caps file.
> * Reimplementation of gopher+.
> * Adds metadata for fingerprinting.
> * Think of tor usage.
> * Adds roundtrips.
> * Using some special path like »/.starttls«.
> * Can overlap with real files.
> * My proposal of first checking for TLS data bytes.
> * Does not add the other compatibility I mention above.
>
>
> What do you people think of my proposal?
>
> Depending on the input I will start implementing this in geomyidae and
> sacc and write the RFC.
>
>
> Sincerely,
>
> Christoph Lohmann
>

[kroovy]

for example for ensuring data integrity


On 02/28/18 09:03, R. A. Pavlov wrote:
> Why would one ever need a secure gopher if gopher client cannot send any
> sensitive info by design? Do I miss something?

[Cameron Kaiser]

> > Although I would prefer gopher+TLS have a categorical port and only be
> > accessible that way, I don't object to probing/upgrading, as long as the
> > client is smart about it.
>
> The idea of requiring a specific port number sounds like a complete hack.
>
> Instead, can you not just cache that STARTTLS was successful and then
> immediately attempt TLS handshake when you connect next time instead of
> doing the STARTTLS upgrade?

I'm not aware of any TLS wrapper that knows how to do this (it would require
a lot of wire-level work to be even possible), and I'm sure many non-TLS
servers would not respond favourably. With a categorical port, you know
exactly what you're connecting to and what it's expecting.

--
------------------------------------ personal: http://www.cameronkaiser.com/ --
Cameron Kaiser * Floodgap Systems * www.floodgap.com * ***@floodgap.com
-- Eggheads unite! You have nothing to lose but your yolks. -- Adlai Stevenson

[Iain R. Learmonth]

Hi,

On 28/02/18 14:12, Cameron Kaiser wrote:
> I'm not aware of any TLS wrapper that knows how to do this (it would require
> a lot of wire-level work to be even possible), and I'm sure many non-TLS
> servers would not respond favourably. With a categorical port, you know
> exactly what you're connecting to and what it's expecting.

With a categorical port you recreate the problems of HTTP where you've
broken URLs when you enable TLS. Instead of http://blah/ you now have
https://blah/ and these are distinct resources. If you are running two
servers, they could have entirely different content and those who have
bookmarks/search indexes or anything referencing a URL would find it
broken when you enable TLS.

Just because such a wrapper doesn't currently exist it doesn't mean that
it wouldn't be possible to create one. It would likely be very easy.

See for example: http://57north.org.uk:443/

Here the server correctly responds to a plaintext HTTP request with a
plaintext HTTP response. It serves an error message explaining that
HTTPS was expected, but there's no reason that you couldn't just have it
serve content instead.

Anyone can just run an wrapper around anything and say you've created a
TLS version but you need to think about what it actually means
semantically in the context of the application. In the case of Gopher I
firmly believe that a separate port, even if not a "fixed" separate
port, is just recreating the problems of HTTP/HTTPS URL breakage and
that we can and should do better.

Thanks,
Iain.

[Zachary Lee Andrews]

On Wed, 28 Feb 2018, Iain R. Learmonth wrote:

> With a categorical port you recreate the problems of HTTP where you've
> broken URLs when you enable TLS. Instead of http://blah/ you now have
> https://blah/ and these are distinct resources. If you are running two
> servers, they could have entirely different content and those who have
> bookmarks/search indexes or anything referencing a URL would find it
> broken when you enable TLS.

This is not the case when protocol relative URLs are used... Which, BTW,
is the way that `lynx -source gopher://some.gopher.server' writes URLs.

see also: https://www.paulirish.com/2010/the-protocol-relative-url/

[Iain R. Learmonth]

Hi,

On 01/03/18 06:56, Zachary Lee Andrews wrote:
> This is not the case when protocol relative URLs are used... Which, BTW,
> is the way that `lynx -source gopher://some.gopher.server' writes URLs.
>
> see also: https://www.paulirish.com/2010/the-protocol-relative-url/

Except that "Gopher with TLS" shouldn't need to be a distinct protocol.
If it is then it's not Gopher anymore. If it requires a behaviour that
is not backwards compatible with existing Gopher clients/servers then it
is not Gopher and you've caused breakage of either URLs or clients.

If you just try to perform a TLS handshake when you talk to servers and
then cache failures and retry those with cleartext, then you've got
Gopher with TLS in a backwards compatible way without breaking URLs.

The main objection to this approach seems to be "it's hard" and that it
requires software which hasn't currently been written.

Thanks,
Iain.

[Nuno Silva]

On 2018-03-01 08:37 +0000, Iain R. Learmonth wrote:
> Hi,
>
> On 01/03/18 06:56, Zachary Lee Andrews wrote:
> > This is not the case when protocol relative URLs are used... Which, BTW,
> > is the way that `lynx -source gopher://some.gopher.server' writes URLs.
> >
> > see also: https://www.paulirish.com/2010/the-protocol-relative-url/
>
> Except that "Gopher with TLS" shouldn't need to be a distinct protocol.
> If it is then it's not Gopher anymore. If it requires a behaviour that
> is not backwards compatible with existing Gopher clients/servers then it
> is not Gopher and you've caused breakage of either URLs or clients.
>
> If you just try to perform a TLS handshake when you talk to servers and
> then cache failures and retry those with cleartext, then you've got
> Gopher with TLS in a backwards compatible way without breaking URLs.
>
> The main objection to this approach seems to be "it's hard" and that it
> requires software which hasn't currently been written.

But gopher with TLS is not gopher. The TLS handshake is not gopher. It
*is* a distinct protocol with a different behaviour. IMHO not making it
something clearly separated from gopher may cause more breakage.

You could also cache failures on the "gophers" port and retry with plain
gopher on port 70.

--
Nuno Silva

[Iain R. Learmonth]

Hi,

On 01/03/18 09:23, Nuno Silva wrote:
> But gopher with TLS is not gopher. The TLS handshake is not gopher. It
> *is* a distinct protocol with a different behaviour. IMHO not making it
> something clearly separated from gopher may cause more breakage.

If your server is going to crash when it sees something that is not a
well crafted Gopher selector, it shouldn't be on the Internet. Servers
should be resilient to all sorts of crazy inputs. It would be
interesting to see some fuzzing of Gopher servers/clients.

> You could also cache failures on the "gophers" port and retry with plain
> gopher on port 70.

Currently I have 65,535 options for my gopher port, but I'm only going
to have one for Gopher+TLS? That seems like a ridiculous restriction
that doesn't need to exist.

Thanks,
Iain.

[Menche]

On Wed, 28 Feb 2018 06:12:35 -0800 (PST)
Cameron Kaiser <***@floodgap.com> wrote:

>
> I'm not aware of any TLS wrapper that knows how to do this (it would
> require a lot of wire-level work to be even possible), and I'm sure
> many non-TLS servers would not respond favourably. With a categorical
> port, you know exactly what you're connecting to and what it's
> expecting.
>

Thought about this a bit; I do think that the STARTTLS suggestion would
be harder to write clients for, but I also don't like the idea of
defining specific ports for TLS and plaintext. TLS support can be
easily indicated in the URI by using "gophers://" in place of
"gopher://". In menus, what about specifying this using the host field
instead of the port? For example,
"1example<TAB>/example<TAB>tls://example.com<TAB>70"

Alternatively, there could be a new item-type that is to be interpreted
like the existing '+' mirror type: instead of specifying a duplicate of
the last entry it would specify a TLS version, possibly with the same
host/port.

[Zachary Lee Andrews]

Just curious if anyone has considered using CAPS.TXT instead of adding
more itemtypes or other weird stuff to signal that a server is TLS
capable.

I am still undecided about how to actually implement the TLS...

Best Regards,
Zach

[Kim Holviala]

On 2 Mar 2018, at 6.34, Zachary Lee Andrews <***@gmail.com> wrote:
>
> Just curious if anyone has considered using CAPS.TXT instead of adding more itemtypes or other weird stuff to signal that a server is TLS capable.

Gophernicus does that, unforunately the live example server is gone because it's part of a huuuuuge drug bust :-D

But that still doesn't solve the problem of how to add "TLS cabable" to the menu resources.


- Kim

[Christoph Lohmann]

Greetings.

On Fri, 02 Mar 2018 06:46:01 +0100 Kim Holviala <***@fastmail.com> wrote:
> On 2 Mar 2018, at 6.34, Zachary Lee Andrews <***@gmail.com> wrote:
> >
> > Just curious if anyone has considered using CAPS.TXT instead of adding more itemtypes or other weird stuff to signal that a server is TLS capable.
>
> Gophernicus does that, unforunately the live example server is gone because it's part of a huuuuuge drug bust :-D
>
> But that still doesn't solve the problem of how to add "TLS cabable" to the menu resources.

Adding such a flag or mark to menu items is just cosmetic. From the menu
you have no direct connection to the resource you are linking to. If
over time the resource/server changes the client will have to do probing
and handling of errors anyway. Here I am against adding incompatibili‐
ties for menu parsers.

My proposal states that you decide how you dig through gopher when you
start the client. Three modes are there, depending on how you like it:

* pure TLS – your client will stop/warn/ask you if there is some
resource which does not understand TLS/probing failed.
* try TLS – your client will probe for TLS, if not, it downgrades
* no TLS – old compatibility mode, client will not try TLS at all

It is up to you, the user, to decide, how you walk the gopher way.


Sincerely,

Christoph Lohmann

[Matt Owen (Jaruzel)]

Why are we not just following how the introduction of https did it ?

1. TLS enabled gopher links are prefixed by gophers://

2. gophers:// links are used in the gophermap file, and shared elsewhere
in the same manner https:// links are. If the connection to that server
is already TLS, then it is assumed that all local links in the gophermap
file that start with a '/' are also TLS.

3. Gopher clients are updated to understand that gophers:// is TLS and on
a default port of 7443 if no discrete port is specified in the link.

4. Legacy Gopher clients will get updated to support gophers://, or not.

5. Newer clients will be built as part of the growing support for TLS,
and eventually older clients with no TLS support will stop being used (in
the same way no-one uses NCSA Mosaic anymore).


When https was introduced, there wasn't any fancy way of letting older
browsers downgrade the connection - it just wasn't supported. If users
wanted to access https, then they had to upgrade their browsers. I don't
see why supporting TLS in gopherspace should be any different.

[Iain R. Learmonth]

Hi,

On 02/03/18 08:09, Matt Owen (Jaruzel) wrote:
> Why are we not just following how the introduction of https did it ?
>
> 1. TLS enabled gopher links are prefixed by gophers://

This is the mistake that was made with HTTP that we can avoid. This made
TLS support a special case, meant everyone changing URLs all over the
place, and a whole load of other problems. It implies that Gopher with
TLS is a separate protocol. You're still using the Gopher protocol just
that now it's running over TLS instead of directly over TCP.

HTTP did not repeat this mistake when they started to deploy QUIC+DTLS.
They didn't decide to have http+quic:// URLs, they reused the https://.

> 2. gophers:// links are used in the gophermap file, and shared elsewhere
> in the same manner https:// links are. If the connection to that server
> is already TLS, then it is assumed that all local links in the gophermap
> file that start with a '/' are also TLS.

gophermap doesn't use URLs, it has a host/port combination and a
selector. You literally cannot encode gophers:// in a gophermap unless
you're going to introduce a whole load of new item types, which is not
what item types are for.

> 3. Gopher clients are updated to understand that gophers:// is TLS and on
> a default port of 7443 if no discrete port is specified in the link.

Instead, how about gopher clients are updated to assume all servers
support TLS unless they see evidence to the contrary? I have previously
described a ridiculously simple method for multiplexing TLS and non-TLS
on the same port (using MSG_PEEK).

> When https was introduced, there wasn't any fancy way of letting older
> browsers downgrade the connection - it just wasn't supported. If users
> wanted to access https, then they had to upgrade their browsers. I don't
> see why supporting TLS in gopherspace should be any different.

There was. You use http:// instead. It was entirely supported and still
is to this day. To this day you're still connecting with HTTP first and
then getting a redirect to HTTPS. You really want to recreate that for
Gopher?

Thanks,
Iain.

[n***@sdf.org]

> Hi,
>
> On 02/03/18 08:09, Matt Owen (Jaruzel) wrote:
>> Why are we not just following how the introduction of https did it ?
>>
>> 1. TLS enabled gopher links are prefixed by gophers://
>
> This is the mistake that was made with HTTP that we can avoid. This made
> TLS support a special case, meant everyone changing URLs all over the
> place, and a whole load of other problems. It implies that Gopher with
> TLS is a separate protocol. You're still using the Gopher protocol just
> that now it's running over TLS instead of directly over TCP.
>
> HTTP did not repeat this mistake when they started to deploy QUIC+DTLS.
> They didn't decide to have http+quic:// URLs, they reused the https://.
>
>> 2. gophers:// links are used in the gophermap file, and shared elsewhere
>> in the same manner https:// links are. If the connection to that server
>> is already TLS, then it is assumed that all local links in the gophermap
>> file that start with a '/' are also TLS.
>
> gophermap doesn't use URLs, it has a host/port combination and a
> selector. You literally cannot encode gophers:// in a gophermap unless
> you're going to introduce a whole load of new item types, which is not
> what item types are for.
>
>> 3. Gopher clients are updated to understand that gophers:// is TLS and
>> on
>> a default port of 7443 if no discrete port is specified in the link.
>
> Instead, how about gopher clients are updated to assume all servers
> support TLS unless they see evidence to the contrary? I have previously
> described a ridiculously simple method for multiplexing TLS and non-TLS
> on the same port (using MSG_PEEK).
>
>> When https was introduced, there wasn't any fancy way of letting older
>> browsers downgrade the connection - it just wasn't supported. If users
>> wanted to access https, then they had to upgrade their browsers. I don't
>> see why supporting TLS in gopherspace should be any different.
>
> There was. You use http:// instead. It was entirely supported and still
> is to this day. To this day you're still connecting with HTTP first and
> then getting a redirect to HTTPS. You really want to recreate that for
> Gopher?
>
> Thanks,
> Iain.
>
I'm with Iain here. Gopher doesn't need to copy HTTP. That's the beauty of
Gopher: it's not HTTP. It's straightforward and beautifully simple. I
personally don't really care to use TLS with Gopher, but I don't want to
see a bunch of gophers:// or secgoph:// or something.

[Michael Lazar]

On Thu, Mar 1, 2018 at 1:37 PM, Mateusz Viste <***@nospam.viste.fr> wrote:
> On Thu, 01 Mar 2018 10:38:03 +0000, Iain R. Learmonth wrote:
>>
>> You would be expecting to receive a handshake message, which would start
>> with 0x16 (ASCII: SYN). I cannot think of a reason that a Gopher
>> selector would start with a non-printable character, so you can most
>> likely check only a single byte.
>
>Sounds good. You should definitely implement it then, so we can see how
>it works.

I thought this was a nice idea, so I tried implementing it on my python gopher
server. You can test it out here:

gopher://mozz.us:7005/1/demo-ssl

It was, in fact, very straightforward to accomplish using helpers from the
python standard library. Here's the code diff if you want to take a look:

https://github.com/michael-lazar/flask-gopher/commit/3777ef92516722f8aad7b858a0c6f931bd875f19#diff-d66156e6e2bebef5659c7aa5593e5ec6R628

- Michael Lazar

[Iain R. Learmonth]

Hi,

On 02/03/18 21:01, Michael Lazar wrote:
> I thought this was a nice idea, so I tried implementing it on my python gopher
> server. You can test it out here:
>
> gopher://mozz.us:7005/1/demo-ssl

Awesome. (:

> It was, in fact, very straightforward to accomplish using helpers from the
> python standard library. Here's the code diff if you want to take a look:
>
> https://github.com/michael-lazar/flask-gopher/commit/3777ef92516722f8aad7b858a0c6f931bd875f19#diff-d66156e6e2bebef5659c7aa5593e5ec6R628

Thanks very much for implementing this! This is exactly what I had in mind.

Thanks,
Iain.